[asaijo]

Hi, all.

I want to report the partiallly successful compilation of OpenFOAM on Cray XT5. As you know, Cray XT5
uses CNL for their node exectuion, so I had to compile the OpenFOAM statically. I could compile
almost all solvers and utilities (but some of them didn't work correctly.) I think it's easy to apply
these procedures to other static-linked only platforms.

Here is the recipe:

0. Switch to GNU compiler (for Cray XT5 module system)
$ module swap PrgEnv-pgi PrgEnv-gnu

1. Expand OpenFOAM-1.5.x.tar.gz (fetched from git repo) and ThirdParty.tar.gz into the CNL-accesible
directory and remove the needless libraries (malloc, zlib and openmpi) in ./ThirdParty. Edit
ThirdParty/Allwmake to comment out malloc and zlib build directives.

NOTE: if you want to use openmpi attached in ThirdParty.tar.gz, you must modify openmpi build
configure script in Allwmake.
Change "--enable-shared --disable-static" to "--disable-shared --enable-static",
and add "--without-memory-manager --without-libnuma".
cf. http://www.open-mpi.org/faq/?category=mpi-apps#static-mpi-apps

2. Edit etc/{bashrc,settings.sh}:
bashrc:
foamInstall=$YOUR_INSTALLED_DIR/$WM_PROJECT
WM_PROJECT_USER_DIR=$FOAM_INST_DIR/$USER-$WM_PROJECT_VERSION
WM_MPLIB=MPICH

settings.sh:
comilerInstall=OpenFOAM => System

case "$WM_MPLIB" in
...
MPICH) => MPICH_ORG)
....
MPI) => MPICH)
export MPI_ARCH_PATH=$MPICH_DIR
export FOAM_MPI_LIBBIN=$FOAM_LIBBIN

Finally, read all configurations.
$ . etc/bashrc

3. Change libso to libo in all Allwmake:
$ cd $FOAM_INST_DIR
$ find -name Allwmake -exec ruby -p -i.bak -e"gsub(/libso/, 'libo')" {} \;
And make solvers not to link indivisual libraries:
$ find -name options -exec ruby -p -i.bak -e"gsub(/EXE_LIBS/, 'EXE_LIBS_ORG')" {} \;

4. Comment out all codes relevant with libdl functions:
You can find them by executing this on shell:
$ find -name "*.C" -exec grep -nH dlfcn {} \;
./src/OpenFOAM/db/dlLibraryTable/dlLibraryTable.C:29:#include <dlfcn.h>
./src/OSspecific/Unix/printStack.C:36:#include <dlfcn.h>

And in $WM_PROJECT_DIR/wmake/rules/$WM_ARCH$WM_COMPILER/general,
replace "-ldl" by "-lz".

5. Make all libraries:
$ cd $WM_PROJECT_DIR
Comment out "(cd applications ..." in Allwmake.
$ export WM_NCOMPPROCS=2; ./Allwmake

6. Make the whole OpenFOAM library:
$ cd $FOAM_LIBBIN
$ ar rcvs libOpenFOAM.a lib*.o

7. Make solvers & utilities Edit $WM_PROJECT_DIR/wmake/rules/$WM_ARCH$WM_COMPILER/c++ to change the compiler and environment
variable:
CC = g++ => CC (or mpiCC)
LINK_LIBS = $(c++DBUG) => -static -Wl,--allow-multiple-definition

$ cd $FOAM_APP
$ ./Allwmake

Known Problems:
* The "functionObjectLibs" directive doesn't work because it loads shared library.
* Linker option "--allow-multiple-definition" suppresses errors of multiple definition of the lexers. I
don't have any ideas about what side-effects it bring on.

Cheers,
asaijo