CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

openFOAM on Open MPI

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   February 13, 2012, 14:49
Default openFOAM on Open MPI
  #1
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
Hello everyone.

I've been trialling a case file on my computer using mhdFoam, and now I wanted to send it to the cluster. I would like to use many processors; however, I am wondering whether I absolutely need to decompose to run in parallel.

Also, I have developed a custom solver which I need to use on the cluster. Is it simply a case of uploading the .C files and compiling them on the cluster? I've never done this before... any instructions or things to know?

Thank you

Kindest Regards,

F

Last edited by fferroni; February 13, 2012 at 15:18.
fferroni is offline   Reply With Quote

Old   February 14, 2012, 03:48
Default
  #2
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
1) Yes, you need to decompose
2) Yes, copy the source to the cluster and compile
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Old   February 14, 2012, 07:01
Default
  #3
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
I did decomposePar and it seems to work for a conventional solver.

For the 2nd point, I uploaded my source code, but when I do wmake I get the error:

cmd=make -f /home/ff208/OpenFOAM/OpenFOAM-1.5/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps
make: *** No rule to make target `/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude/UPstream.H', needed by `EpotFoam.dep'. Stop.

I've compiled it on my computer and it works fine.
Any information helping a lowly undergrad would be highly appreciated!

Kindest Regards,

F
fferroni is offline   Reply With Quote

Old   February 14, 2012, 08:19
Default
  #4
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
Did you copy the entire OpenFOAM directory from your computer to the cluster or did you make a fresh install of OF and then just copy your custom solver? I'm guessing the former, and that your lnInclude links are pointing to invalid paths now.
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Old   February 15, 2012, 03:54
Default
  #5
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
Hi!

Thanks for that. I did a 'fresh' copy and this is the error message I get:


cmd=make -f /home/ff208/OpenFOAM/OpenFOAM-1.5/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps
Making dependency list for source file EpotFoam.C
SOURCE=EpotFoam.C ; g++ -m64 -Dlinux64 -DDP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-40 -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/incompressible/LES/LESModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/LES/LESdeltas/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels/incompressible/singlePhaseTransportModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/finiteVolume/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/sampling/lnInclude -IlnInclude -I. -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OSspecific/Unix/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOpt/EpotFoam.o
In file included from EpotFoam.C:22:
readEPISOControls.H:3:73: warning: no newline at end of file
g++ -m64 -Dlinux64 -DDP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-40 -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/incompressible/LES/LESModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/LES/LESdeltas/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels/incompressible/singlePhaseTransportModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/finiteVolume/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/sampling/lnInclude -IlnInclude -I. -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OSspecific/Unix/lnInclude -fPIC Make/linux64GccDPOpt/EpotFoam.o -L/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt \
-lincompressibleTurbulenceModel -lincompressibleLESModels -lincompressibleTransportModels -lfiniteVolume -lmeshTools -lOpenFOAM -liberty -ldl -lm -o /home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C
/usr/bin/ld: cannot open output file /home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C: Read-only file system
collect2: ld returned 1 exit status
make: *** [/home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C] Error 1


Read only file system...! Any way I can tell the compiler to place the compiled solver in another location, and then tell OF to search in that folder whenever I type "EpotFoam"?

Kindest Regards,

Francesco

Last edited by fferroni; February 15, 2012 at 04:19.
fferroni is offline   Reply With Quote

Old   February 15, 2012, 04:27
Default
  #6
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
Yes. You can modify the paths temporarily by changing environment variables or permanently by editing OF/etc/bashrc and OF/etc/config/settings.sh.
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Old   February 15, 2012, 05:16
Default
  #7
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
Great! That worked when it comes to compiling...

However, I run the solver, it reads all the case files, starts the time loop etc. and I get another error

Attempt to return dictionary entry as a primitive

file: /export71/home/ff208/system/fvSolution::U from line 30 to line 33.

From function ITstream& primitiveEntry::stream() const
in file db/dictionary/dictionaryEntry/dictionaryEntry.C at line 83.

FOAM aborting

#0 Foam::error:rintStack(Foam::Ostream&) in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so"
#1 Foam::IOerror::abort() in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so"
#2 Foam::dictionaryEntry::stream() const in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so"
#3 Foam::lduMatrix::solverPerformance Foam::solve<Foam::Vector<double> >(Foam::tmp<Foam::fvMatrix<Foam::Vector<double> > > const&) in "/home/ff208/EpotFoam"
#4 main in "/home/ff208/EpotFoam"
#5 __libc_start_main in "/lib64/libc.so.6"
#6 Foam::regIOobject::writeObject(Foam::IOstream::str eamFormat, Foam::IOstream::versionNumber, Foam::IOstream::compressionType) const in "/home/ff208/EpotFoam"
Aborted

fferroni is offline   Reply With Quote

Old   February 15, 2012, 05:49
Default
  #8
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
There is a problem with your fvSolution file. Is it is written to run with OF2 (the dictionary format changed sometime after 1.5 was released)?
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Old   February 15, 2012, 14:25
Default
  #9
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
Yes. The syntax was different. I should have spotted the /fvSolution message.. sorry!

Unfortunately () it gave me another error instead!

keyword PISO is undefined in dictionary "/export71/home/ff208/OpenFOAM/ff208-1.5/system/fvSolution"

file: /export71/home/ff208/OpenFOAM/ff208-1.5/system/fvSolution from line 20 to line 51.

From function dictionary::subDict(const word& keyword) const
in file db/dictionary/dictionary.C at line 271.

I've checked in fvSolution whether the PISO loop is defined any differently from the syntax of tutorials in the cluster OF version, but they look exactly the same.
I've changed $FOAM_APPBIN to another location where my custom compiled solver is. Could this be the reason? Perhaps it's trying to find another solver? Just throwing ideas... googled the error but no resolution.
Also I thought (maybe erroneously) that it could do with what libraries I've included when compiling? Maybe it doesn't understand what PISO is?

I have these:
EXE_INC = \
-I$(LIB_SRC)/turbulenceModels \
-I$(LIB_SRC)/turbulenceModels/incompressible/LES/LESModel \
-I$(LIB_SRC)/turbulenceModels/LES/LESdeltas/lnInclude \
-I$(LIB_SRC)/transportModels \
-I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
-I$(LIB_SRC)/finiteVolume/lnInclude \
-I$(LIB_SRC)/sampling/lnInclude

EXE_LIBS = \
-lincompressibleTransportModels \
-lfiniteVolume \
-lmeshTools

Not sure if it's relevant...

Thanks again for all the help! Hopefully one day I can answer questions rather than asking questions

F
fferroni is offline   Reply With Quote

Old   February 16, 2012, 04:05
Default
  #10
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
If you're unsure if it's using the correct executable, you can use the full path to run it. But I think there must be another error in the fvSolution file - post it here. By the way, why aren't you running it on the same version of OpenFOAM as on your local computer?

- Anton
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Old   February 17, 2012, 14:56
Default
  #11
Member
 
Join Date: Nov 2011
Posts: 44
Rep Power: 5
fferroni is on a distinguished road
Ok. I managed to get hold on the IT department and I'm able to load 2.0.0.

However, I get this error when running my compiled script:

/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE
/home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE

What kind of error is this? No idea...
fferroni is offline   Reply With Quote

Old   February 18, 2012, 10:40
Default
  #12
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Delft, Netherlands
Posts: 919
Rep Power: 17
akidess will become famous soon enough
Something wrong with libraries. Did you remove OpenFOAM-1.5 or is it still there? Did you compile OF-2.0.0 or install it from a package? And most importantly, did you clean the build directory of your solver (using wclean) before you tried to recompile?

You can use
Code:
ldd `which EpotFoam`
to check if the executable is picking up the correct libraries.
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
*Help define the OpenFOAM stackexchange Q&A site: http://area51.stackexchange.com/prop...oam-technology
akidess is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
2D Mesh Generation Tutorial for GMSH aeroslacker Open Source Meshers: Gmsh, Netgen, CGNS, ... 12 January 19, 2012 04:52
Installing OpenFOAM with SYSTEM MPI while using Third-Party libs m.culpo OpenFOAM Installation 5 July 3, 2011 03:48
OpenFOAM MPI coupling Ford Prefect OpenFOAM Programming & Development 3 August 25, 2010 03:18
MPI profiling OpenFOAM damBreak3D application mellanoxuser OpenFOAM Pre-Processing 0 April 13, 2008 23:15
Open Mpi on AMD 64 cluster nishant_hull OpenFOAM Running, Solving & CFD 6 February 4, 2008 19:20


All times are GMT -4. The time now is 23:53.