Compiling OpenFOAM on hpc-fe.gbar.dtu.dk
 

Hi All.
I have successfully compiled OpenFOAM-1.7.1 on the DTU cluster hpc-fe.gbar.dtu.dk.
Follow the normal instructions for compiling openfoam, but pay attention to the following:

The default gcc version on the cluster is too old to use (most of openfoam compiles, but some libraries will fail), so you must configure openfoam to use gcc44 and g++44 instead of gcc and g++.

In OpenFOAM-1.7.1/etc/bashrc change the value of WM_CC and WM_CXX to gcc44 and g++44, respectively.

Further you must edit the two files, c and c++ in OpenFOAM-1.7.1/wmake/rules/linux64Gcc/, to change any occurrences of "gcc" and "g++" into "gcc44" and "g++44".
This information was found at:
http://consultancy.edvoncken.net/ind...lding_OpenFOAM

Furthermore you must configure openfoam to use the system mpi instead of the one that ships with OpenFOAM.
First tell openfoam to use the system mpi: in OpenFOAM-1.7.1/etc/bashrc
set the variable WM_MPLIB to SYSTEMOPENMPI. The line you should edit ends up looking like this:
: ${WM_MPLIB:=SYSTEMOPENMPI}; export WM_MPLIB

Next tell openfoam where to find the system MPI. In etc/settings.sh right after these two lines:
SYSTEMOPENMPI)
# use the system installed openmpi, get library directory via mpicc
mpi_version=openmpi-system
add these two lines:
export MPI_HOME=/opt/SUNWhpc/HPC8.2.1c/gnu/
export MPI_ARCH_PATH=/opt/SUNWhpc/HPC8.2.1c/gnu/

In order to use the system mpi you must call:

module load mpi

at the beginning of your session, i.e. either interactive session, or in your run file.
My run file to submit to the que looks like this (this I have inherited from NGJ):

#! /bin/sh
# <SUN Grid Engine>
# =======================
# standard start options, standardized job-name/output
#$ -S /bin/sh -cwd -j yes -N test1 -o log.run$JOB_ID
# =======================
# </SUN Grid Engine>
# <environ>
# ---------------------------------------------------------------------------
# : ${SGE_ROOT:=/opt/sge/sge6_2u5/}
# : ${SGE_CELL:=default}
# for i in $SGE_ROOT/$SGE_CELL/site/environ; do [ -f $i ] && echo "source $i" && . $i; done
# PATH=$HOME/bin:/home/ngja/OpenFOAM/ThirdParty/cmake-2.6.4/build/bin/:$PATH
FOAM_SIGFPE=1 #Halts on Nan/Floating Point Exceptions
module load mpi
. $HOME/OpenFOAM/OpenFOAM-1.7.1/etc/bashrc
# ---------------------------------------------------------------------------
# </environ>
APPLICATION=waveFoam17

WHICH_APP=`echo $APPLICATION | cut -f1 -d" "`; WHICH_APP=`which $WHICH_APP`;
caseName=${PWD##*/}
jobName=$caseName
# avoid generic names
case "$jobName" in
foam | OpenFOAM )
jobName=$(dirname $PWD)
jobName=$(basename $jobName)
;;
esac

cat<<PRINT
(**) job $jobName
(**) pwd ${PWD/$HOME/~}
(II) job_id $JOB_ID
(II) queue ${QUEUE:-NULL}
(II) host ${HOSTNAME:-NULL}
(II) slots ${NSLOTS:-1}
(II) foam ${WM_PROJECT_VERSION}
(II) App ${WHICH_APP/$HOME/~}
PRINT

# parallel/serial
if [ "$NSLOTS" -gt 1 ]; then
mpirun=mpirun
case "$PE" in
ompi)
echo "(II) mpi $PE-${OPENMPI_VERSION}"
# openmpi parameters
## mpirun="$mpirun --mca mpi_yield_when_idle 1"
;;
esac
mpirun="$mpirun --mca btl_openib_verbose 1 --mca btl ^tcp ${APPLICATION} -parallel"

echo "(II) mpirun /usr/mpi/gcc/openmpi-1.4.1/bin/mpirun"
echo "(II) cmd $mpirun"
exec $mpirun
else
echo "(II) cmd ${APPLICATION}"
${APPLICATION}
fi

Kind regards Kasper Kærgaard

Hi ,
I am using OpenFOAM on an Intel cluster . I compiled it successfully with the openfoam openmpi however it gave the following error on running a case :
I decided to use the above procedure but on making the following changes :
On changing $WM_MPLIB in etc/bashrc in the openfoam folder and adding the two lines etc/settings.sh as follows :
I get the following error on sourcing ~/.bashrc file :
Another , thing :
does not work . saying module : command not found .

Could u help me out in sorting the above problems ?

Thanks ,
Balkrishna .