|
[Sponsors] |
February 13, 2012, 13:49 |
openFOAM on Open MPI
|
#1 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
Hello everyone.
I've been trialling a case file on my computer using mhdFoam, and now I wanted to send it to the cluster. I would like to use many processors; however, I am wondering whether I absolutely need to decompose to run in parallel. Also, I have developed a custom solver which I need to use on the cluster. Is it simply a case of uploading the .C files and compiling them on the cluster? I've never done this before... any instructions or things to know? Thank you Kindest Regards, F Last edited by fferroni; February 13, 2012 at 14:18. |
|
February 14, 2012, 02:48 |
|
#2 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
1) Yes, you need to decompose
2) Yes, copy the source to the cluster and compile
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
February 14, 2012, 06:01 |
|
#3 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
I did decomposePar and it seems to work for a conventional solver.
For the 2nd point, I uploaded my source code, but when I do wmake I get the error: cmd=make -f /home/ff208/OpenFOAM/OpenFOAM-1.5/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps make: *** No rule to make target `/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude/UPstream.H', needed by `EpotFoam.dep'. Stop. I've compiled it on my computer and it works fine. Any information helping a lowly undergrad would be highly appreciated! Kindest Regards, F |
|
February 14, 2012, 07:19 |
|
#4 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
Did you copy the entire OpenFOAM directory from your computer to the cluster or did you make a fresh install of OF and then just copy your custom solver? I'm guessing the former, and that your lnInclude links are pointing to invalid paths now.
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
February 15, 2012, 02:54 |
|
#5 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
Hi!
Thanks for that. I did a 'fresh' copy and this is the error message I get: cmd=make -f /home/ff208/OpenFOAM/OpenFOAM-1.5/wmake/Makefile MAKE_DIR=Make INCLUDE_DEPS=Make/linux64GccDPOpt/includeDeps Making dependency list for source file EpotFoam.C SOURCE=EpotFoam.C ; g++ -m64 -Dlinux64 -DDP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-40 -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/incompressible/LES/LESModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/LES/LESdeltas/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels/incompressible/singlePhaseTransportModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/finiteVolume/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/sampling/lnInclude -IlnInclude -I. -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OSspecific/Unix/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOpt/EpotFoam.o In file included from EpotFoam.C:22: readEPISOControls.H:3:73: warning: no newline at end of file g++ -m64 -Dlinux64 -DDP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-40 -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/incompressible/LES/LESModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/turbulenceModels/LES/LESdeltas/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/transportModels/incompressible/singlePhaseTransportModel -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/finiteVolume/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/sampling/lnInclude -IlnInclude -I. -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OpenFOAM/lnInclude -I/home/ff208/OpenFOAM/OpenFOAM-1.5/src/OSspecific/Unix/lnInclude -fPIC Make/linux64GccDPOpt/EpotFoam.o -L/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt \ -lincompressibleTurbulenceModel -lincompressibleLESModels -lincompressibleTransportModels -lfiniteVolume -lmeshTools -lOpenFOAM -liberty -ldl -lm -o /home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C /usr/bin/ld: cannot open output file /home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C: Read-only file system collect2: ld returned 1 exit status make: *** [/home/ff208/OpenFOAM/OpenFOAM-1.5/applications/bin/linux64GccDPOpt/EpotFoam.C] Error 1 Read only file system...! Any way I can tell the compiler to place the compiled solver in another location, and then tell OF to search in that folder whenever I type "EpotFoam"? Kindest Regards, Francesco Last edited by fferroni; February 15, 2012 at 03:19. |
|
February 15, 2012, 03:27 |
|
#6 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
Yes. You can modify the paths temporarily by changing environment variables or permanently by editing OF/etc/bashrc and OF/etc/config/settings.sh.
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
February 15, 2012, 04:16 |
|
#7 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
Great! That worked when it comes to compiling...
However, I run the solver, it reads all the case files, starts the time loop etc. and I get another error Attempt to return dictionary entry as a primitive file: /export71/home/ff208/system/fvSolution::U from line 30 to line 33. From function ITstream& primitiveEntry::stream() const in file db/dictionary/dictionaryEntry/dictionaryEntry.C at line 83. FOAM aborting #0 Foam::error:rintStack(Foam::Ostream&) in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so" #1 Foam::IOerror::abort() in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so" #2 Foam::dictionaryEntry::stream() const in "/home/ff208/OpenFOAM/OpenFOAM-1.5/lib/linux64GccDPOpt/libOpenFOAM.so" #3 Foam::lduMatrix::solverPerformance Foam::solve<Foam::Vector<double> >(Foam::tmp<Foam::fvMatrix<Foam::Vector<double> > > const&) in "/home/ff208/EpotFoam" #4 main in "/home/ff208/EpotFoam" #5 __libc_start_main in "/lib64/libc.so.6" #6 Foam::regIOobject::writeObject(Foam::IOstream::str eamFormat, Foam::IOstream::versionNumber, Foam::IOstream::compressionType) const in "/home/ff208/EpotFoam" Aborted |
|
February 15, 2012, 04:49 |
|
#8 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
There is a problem with your fvSolution file. Is it is written to run with OF2 (the dictionary format changed sometime after 1.5 was released)?
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
February 15, 2012, 13:25 |
|
#9 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
Yes. The syntax was different. I should have spotted the /fvSolution message.. sorry!
Unfortunately () it gave me another error instead! keyword PISO is undefined in dictionary "/export71/home/ff208/OpenFOAM/ff208-1.5/system/fvSolution" file: /export71/home/ff208/OpenFOAM/ff208-1.5/system/fvSolution from line 20 to line 51. From function dictionary::subDict(const word& keyword) const in file db/dictionary/dictionary.C at line 271. I've checked in fvSolution whether the PISO loop is defined any differently from the syntax of tutorials in the cluster OF version, but they look exactly the same. I've changed $FOAM_APPBIN to another location where my custom compiled solver is. Could this be the reason? Perhaps it's trying to find another solver? Just throwing ideas... googled the error but no resolution. Also I thought (maybe erroneously) that it could do with what libraries I've included when compiling? Maybe it doesn't understand what PISO is? I have these: EXE_INC = \ -I$(LIB_SRC)/turbulenceModels \ -I$(LIB_SRC)/turbulenceModels/incompressible/LES/LESModel \ -I$(LIB_SRC)/turbulenceModels/LES/LESdeltas/lnInclude \ -I$(LIB_SRC)/transportModels \ -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \ -I$(LIB_SRC)/finiteVolume/lnInclude \ -I$(LIB_SRC)/sampling/lnInclude EXE_LIBS = \ -lincompressibleTransportModels \ -lfiniteVolume \ -lmeshTools Not sure if it's relevant... Thanks again for all the help! Hopefully one day I can answer questions rather than asking questions F |
|
February 16, 2012, 03:05 |
|
#10 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
If you're unsure if it's using the correct executable, you can use the full path to run it. But I think there must be another error in the fvSolution file - post it here. By the way, why aren't you running it on the same version of OpenFOAM as on your local computer?
- Anton
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
February 17, 2012, 13:56 |
|
#11 |
Member
Join Date: Nov 2011
Posts: 44
Rep Power: 14 |
Ok. I managed to get hold on the IT department and I'm able to load 2.0.0.
However, I get this error when running my compiled script: /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE /home/ff208/EpotFoam: symbol lookup error: /home/ff208/EpotFoam: undefined symbol: _ZN4Foam10SLListBase12endConstIterE What kind of error is this? No idea... |
|
February 18, 2012, 09:40 |
|
#12 |
Senior Member
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29 |
Something wrong with libraries. Did you remove OpenFOAM-1.5 or is it still there? Did you compile OF-2.0.0 or install it from a package? And most importantly, did you clean the build directory of your solver (using wclean) before you tried to recompile?
You can use Code:
ldd `which EpotFoam`
__________________
*On twitter @akidTwit *Spend as much time formulating your questions as you expect people to spend on their answer. |
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[Gmsh] 2D Mesh Generation Tutorial for GMSH | aeroslacker | OpenFOAM Meshing & Mesh Conversion | 12 | January 19, 2012 03:52 |
Installing OpenFOAM with SYSTEM MPI while using Third-Party libs | m.culpo | OpenFOAM Installation | 5 | July 3, 2011 03:48 |
OpenFOAM MPI coupling | Ford Prefect | OpenFOAM Programming & Development | 3 | August 25, 2010 03:18 |
MPI profiling OpenFOAM damBreak3D application | mellanoxuser | OpenFOAM Pre-Processing | 0 | April 13, 2008 23:15 |
Open Mpi on AMD 64 cluster | nishant_hull | OpenFOAM Running, Solving & CFD | 6 | February 4, 2008 18:20 |