CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

Compiling on ROCKS Cluster 5.4.3

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By ciberbob1987

Reply
 
LinkBack Thread Tools Display Modes
Old   February 19, 2012, 20:24
Default Compiling on ROCKS Cluster 5.4.3
  #1
New Member
 
Marco
Join Date: Feb 2012
Posts: 7
Rep Power: 5
ciberbob1987 is on a distinguished road
Hi, I'm a student of Politecnico di Torino, and I'm working with OpenFOAM on the HPC cluster of my university.
The system is equipped with ROCKS Cluster Linux Distribution 5.4.3 OS and SGE scheduler. I'm working via SSH and my account is obviously not SU.

Actually I don't know if this is the right way, anyway I have done these steps:

Code:
[~]snft3@cclix7$ cd Marco
[Marco]snft3@cclix7$ wget http://downloads.sourceforge.net/project/foam/foam/2.1.0/OpenFOAM-2.1.0.tgz?r=http%3A%2F%2Fopenfoam.org%2Fdownload%2Fsource.php&ts=1329690292&use_mirror=dfn
[1] 18979
[2] 18980
[Marco]snft3@cclix7$ --2012-02-20 00:05:08--  http://downloads.sourceforge.net/project/foam/foam/2.1.0/OpenFOAM-2.1.0.tgz?r=http%3A%2F%2Fopenfoam.org%2Fdownload%2Fsource.php
Resolving downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://garr.dl.sourceforge.net/project/foam/foam/2.1.0/OpenFOAM-2.1.0.tgz [following]
--2012-02-20 00:05:08--  http://garr.dl.sourceforge.net/project/foam/foam/2.1.0/OpenFOAM-2.1.0.tgz
Resolving garr.dl.sourceforge.net... 193.206.140.34, 2001:760:ffff:b0::34
Connecting to garr.dl.sourceforge.net|193.206.140.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 32518358 (31M) [application/x-tar]
Saving to: `OpenFOAM-2.1.0.tgz'

100%[================================================================>] 32,518,358  7.81M/s   in 4.2s    

2012-02-20 00:05:13 (7.31 MB/s) - `OpenFOAM-2.1.0.tgz' saved [32518358/32518358]


[1]-  Done                    wget http://downloads.sourceforge.net/project/foam/foam/2.1.0/OpenFOAM-2.1.0.tgz?r=http%3A%2F%2Fopenfoam.org%2Fdownload%2Fsource.php
[2]+  Done                    ts=1329690292
[Marco]snft3@cclix7$ tar -xzf OpenFOAM-2.1.0.tgz
[Marco]snft3@cclix7$ mkdir OpenFOAM
[Marco]snft3@cclix7$ mv OpenFOAM-2.1.0 ./OpenFOAM/
[Marco]snft3@cclix7$ cd OpenFOAM
[Marco]snft3@cclix7$ export FOAM_INST_DIR=/share/apps/snft/snft3/Marco/OpenFOAM
[Marco]snft3@cclix7$ foamDotFile=$FOAM_INST_DIR/OpenFOAM-2.1.0/etc/bashrc
[Marco]snft3@cclix7$ [ -f $foamDotFile ] && . $foamDotFile



[Marco]snft3@cclix7$ cd $WM_PROJECT_DIR
[OpenFOAM-2.0.1]snft3@cclix7$ chmod +x Allwmake
[OpenFOAM-2.0.1]snft3@cclix7$ ./Allwmake | tee ../compile.log
After about an hour I accidentally aborted the compiling process.
In the log there were some errors e.g.

Code:
make[2]: Entering directory `/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/applications/solvers/heatTransfer/buoyantPimpleFoam'
/bin/sh: /share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/wmake/platforms/linux64Gcc/wmkdep: No such file or directory
make[2]: Leaving directory `/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/applications/solvers/heatTransfer/buoyantPimpleFoam'
make[2]: Entering directory `/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/applications/solvers/heatTransfer/buoyantPimpleFoam'
SOURCE=buoyantPimpleFoam.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-100 -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/thermophysicalModels/basic/lnInclude -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/turbulenceModels/compressible/turbulenceModel -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/finiteVolume/lnInclude -IlnInclude -I. -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OpenFOAM/lnInclude -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/buoyantPimpleFoam.o
/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OpenFOAM/lnInclude/globalIndexAndTransformI.H: In member function 'Foam::labelList Foam::globalIndexAndTransform::transformIndicesForPatches(const Foam::labelHashSet&) const':
/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OpenFOAM/lnInclude/globalIndexAndTransformI.H:515: warning: converting to 'Foam::label' from 'double'
/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/finiteVolume/lnInclude/readTimeControls.H: In function 'int main(int, char**)':
/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/finiteVolume/lnInclude/readTimeControls.H:38: warning: unused variable 'maxDeltaT'
g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-100 -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/thermophysicalModels/basic/lnInclude -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/turbulenceModels/compressible/turbulenceModel -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/finiteVolume/lnInclude -IlnInclude -I. -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OpenFOAM/lnInclude -I/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/src/OSspecific/POSIX/lnInclude   -fPIC -Xlinker --add-needed -Xlinker --no-as-needed Make/linux64GccDPOpt/buoyantPimpleFoam.o -L/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64GccDPOpt/lib \
	     -lmeshTools -lbasicThermophysicalModels -lspecie -lcompressibleTurbulenceModel -lcompressibleRASModels -lcompressibleLESModels -lfiniteVolume -lOpenFOAM -ldl   -lm -o /share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64GccDPOpt/bin/buoyantPimpleFoam
/usr/bin/ld: cannot find -lmeshTools
collect2: ld returned 1 exit status
make[2]: *** [/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64GccDPOpt/bin/buoyantPimpleFoam] Error 1
make[2]: Leaving directory `/share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.1.0/applications/solvers/heatTransfer/buoyantPimpleFoam'
make[1]: *** [buoyantPimpleFoam] Error 2
Other ones are very similar.
I'm now trying to compile again, I'll post the log.
I want to know:
  1. what kind of errors are these?
  2. are they critical?
  3. how to solve them?

Thank you. Marco
chenxizh likes this.
ciberbob1987 is offline   Reply With Quote

Old   February 20, 2012, 05:31
Default
  #2
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,301
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings Marco and welcome to the forum!

AFAIK, Rocks 5.4 is based on CentOS 5.4, which means that you probably don't have decent Gcc compiler... or in other words, have a version older than Gcc 4.3.3.

In these situations where CentOS and relatives are involved, I generally point people to CentFOAM: http://sourceforge.net/apps/mediawik...itle=Main_Page

But if you want to waste more time trying to diagnose the issue:
  • Code:
    gcc -v
    will give you the version you've got installed. Remember, anything below 4.3.3 will not work out-of-the-box.
  • To really see what goes wrong:
    Code:
    ./Allwmake > make.log 2>&1
    This way the whole compilation is logged into the file "make.log".
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   February 22, 2012, 12:19
Default
  #3
New Member
 
Marco
Join Date: Feb 2012
Posts: 7
Rep Power: 5
ciberbob1987 is on a distinguished road
Hi Bruno, thank you for the reply. I have downloaded CentFOAM and made some tests, but something related to MPI don't work well.

In the user guide of the cluster (http://dauin-hpc.polito.it/?p=user-guide) is reported that the path of mpirun is /opt/openmpi/bin, but if I use it with

Code:
### Defining openmpi executable
MPIRUN=/opt/openmpi/bin/mpirun
#
#
### Set run directory
DIR_FOAM_RUN=/share/apps/snft/snft3/Marco/OpenFOAM/run
#
## Solver and case
SOLVER=snappyHexMesh
CASO=turbineSiting
#
source /share/apps/snft/snft3/Marco/OpenFOAM/OpenFOAM-2.0.x/etc/bashrc
cd $DIR_FOAM_RUN
$MPIRUN $SOLVER -case $CASO -parallel
without any changes in OF config the system returns an error:

Code:
[compute-0-0.local:32745] mca: base: component_find: unable to open /share/apps/snft/snft3/Marco/OpenFOAM/ThirdParty-2.0.x/platforms/linux64Gcc45/openmpi-1.5.3/lib/openmpi/mca_ess_slurmd: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
/opt/openmpi/bin/mpirun: symbol lookup error: /share/apps/snft/snft3/Marco/OpenFOAM/ThirdParty-2.0.x/platforms/linux64Gcc45/openmpi-1.5.3/lib/openmpi/mca_plm_rshd.so: undefined symbol: orte_plm_base_rsh_launch_agent_setup
So, I tried to set WM_MPLIB=SYSTEMOPENMPI in bashrc and edit the settings.sh file from:

Code:
SYSTEMOPENMPI)
    # Use the system installed openmpi, get library directory via mpicc
    export FOAM_MPI=openmpi-system

    # Set compilation flags here instead of in wmake/rules/../mplibSYSTEMOPENMPI
    export PINC="`mpicc --showme:compile`"
    export PLIBS="`mpicc --showme:link`"
    libDir=`echo "$PLIBS" | sed -e 's/.*-L\([^ ]*\).*/\1/'`

    # Bit of a hack: strip off 'lib' and hope this is the path to openmpi
    # include files and libraries.
    export MPI_ARCH_PATH="${libDir%/*}"

    if [ "$FOAM_VERBOSE" -a "$PS1" ]
    then
        echo "Using system installed MPI:"
        echo "    compile flags : $PINC"
        echo "    link flags    : $PLIBS"
        echo "    libmpi dir    : $libDir"
    fi

    _foamAddLib     $libDir
    unset libDir
    ;;
to

Code:
SYSTEMOPENMPI)
    # Use the system installed openmpi, get library directory via mpicc
    export FOAM_MPI=openmpi-system

    # Set compilation flags here instead of in wmake/rules/../mplibSYSTEMOPENMPI
    export PINC="`/opt/openmpi/bin/mpicc --showme:compile`"
    export PLIBS="`/opt/openmpi/bin/mpicc --showme:link`"
    libDir=/opt/openmpi/lib

    # Bit of a hack: strip off 'lib' and hope this is the path to openmpi
    # include files and libraries.
    export MPI_ARCH_PATH=/opt/openmpi

    if [ "$FOAM_VERBOSE" -a "$PS1" ]
    then
        echo "Using system installed MPI:"
        echo "    compile flags : $PINC"
        echo "    link flags    : $PLIBS"
        echo "    libmpi dir    : $libDir"
    fi

    _foamAddLib     $libDir
    unset libDir
    ;;
But also in this case the system reports some errors (one for each processor):
Code:
--> FOAM FATAL ERROR: 
Trying to use the dummy Pstream library.
This dummy library cannot be used in parallel mode

    From function UPstream::init(int& argc, char**& argv)
    in file UPstream.C at line 37.

FOAM exiting
Finally, I restored the default bashrc and settings.sh to use the OpenMPI provided with CentFOAM. Now the program runs (snappyHexMesh), but I can't use more than one node and before the execution some warnings are shown (see attached logs).
Attached Files
File Type: gz snappyHexMesh_log.tar.gz (31.1 KB, 3 views)
ciberbob1987 is offline   Reply With Quote

Old   February 22, 2012, 17:08
Default
  #4
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,301
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Marco,

Those errors almost look painful...

You were going in the right direction with "WM_MPLIB=SYSTEMOPENMPI". All that was missing was the dedicated Pstream and PTScotch libraries:
  1. Pstream:
    Code:
    foam
    wmake all src/Pstream
  2. PTScotch:
    Code:
    foam3rdParty
    ./Allwmake
Well, running Allwmake once more on the OpenFOAM folder should do the trick as well, perhaps even better:
Code:
foam
./Allwmake
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   February 22, 2012, 17:17
Default
  #5
New Member
 
Marco
Join Date: Feb 2012
Posts: 7
Rep Power: 5
ciberbob1987 is on a distinguished road
Hi Bruno,
I will try as soon as possible your solution, but I have gcc version 4.1.2 and flex is not installed. Is it sufficient to compile third party without errors?
ciberbob1987 is offline   Reply With Quote

Old   February 22, 2012, 17:35
Default
  #6
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,301
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Marco,

You should have gotten a shiny new gcc 4.4 or 4.5 with CentFOAM! For using strictly with Cent/OpenFOAM... i.e., while you have OpenFOAM's environment active (a shell where you're able to run OpenFOAM applications).

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   February 23, 2012, 07:22
Default
  #7
Senior Member
 
linnemann's Avatar
 
Niels Nielsen
Join Date: Mar 2009
Location: NJ - Denmark
Posts: 450
Rep Power: 15
linnemann will become famous soon enough
Hi

Why do you want to change the openmpi location?

centFOAM comes with its own openmpi install.

Youre trying to mix a version of openmpi (most likely compiled with gcc 4.1.2) with a newer version compiled with gcc 4.5 or higher.

This is bound to give you errors :-). If you want to use system openmpi you have to recompile OF.

If your cluster dont have infiniband, the provided openmpi in centFOAM should be just as good as any system openmpi.

EDIT: Just saw that your cluster indeed have inifiniband so you will have to recompile as Brunos suggest using the system openmpi.

Best
__________________
Linnemann

PS. I do not do personal support, so please post in the forums.
linnemann is offline   Reply With Quote

Old   February 23, 2012, 09:20
Default
  #8
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,301
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Linnemann,
Quote:
Originally Posted by linnemann View Post
EDIT: Just saw that your cluster indeed have inifiniband so you will have to recompile as Brunos suggest using the system openmpi.
If CentFOAM delivers a complete installation (including the "lnInclude" folders with the correct time stamps), then it should only be necessary to compile Pstream and PTScotch, due to OpenFOAM's building structure

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   February 25, 2012, 20:21
Default
  #9
New Member
 
Marco
Join Date: Feb 2012
Posts: 7
Rep Power: 5
ciberbob1987 is on a distinguished road
Hi everybody,I have re-compiled the PStream code and now all seems to work fine.
First simulation with posousSimpleFoam runned in parallel on 7 CPU with no errors.
Many thanks to all, you saved me!
ciberbob1987 is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to Run Parallel Jobs in Rocks Cluster asaha OpenFOAM Running, Solving & CFD 12 July 4, 2012 22:51
Rocks and linux for fluent cluster Far FLUENT 2 March 2, 2012 10:31
OpenFoam 2.0.0 installation + ICC + Rocks Cluster chinper OpenFOAM Installation 5 July 13, 2011 16:00
Compiling UDF on a cluster Adya FLUENT 1 December 14, 2009 12:30


All times are GMT -4. The time now is 14:19.