CFD Online Discussion Forums - How to make code run in parallel?

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)

- OpenFOAM Programming & Development (https://www.cfd-online.com/Forums/openfoam-programming-development/)

- - How to make code run in parallel? (https://www.cfd-online.com/Forums/openfoam-programming-development/76404-how-make-code-run-parallel.html)

How to make code run in parallel?

Hi guys,

I've recently adapted pimpleDyMFoam to form a code that solves for 1DOF flow induced rotational motion, and the code ("flapFoam" in the error message below) seems to work fine when running on a single core. However, when I tried to run it using mpirun, I received floating point exception errors.

Code:

[2] [5] #0  [1] #0  #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[3] #0  Foam::error::printStack(Foam::Ostream&) in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64Gc in c in "/home/cwang5/OpenFOAM/OpenFOAM-1."/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/DPOpt/ll6i.x/lib/linux6inux64GccDPOpt/libOpenFb4GccDPOpt/libOpenFOAM.so"OAM.so"

[1] #1  

[5] #1  Foam::sigFpe::sigFpeHandler(int)Foam::sigFpe::sigFpeHandler(int)OpenFOAM.so"

[2] #1  Foam::sigFpe::sigFpeHandler(int) in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64GccDPOpt/libOpenFOAM.so"

[5] #2   in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64GccDPOpt/libOpenFOAM.so"

[1] #2   in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64GccDPOpt/libOpenFOAM.so"

[2] #2  ?? in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64GccDPOpt/libOpenFOAM.so"

[3] #1  Foam::sigFpe::sigFpeHandler(int) in "/lib64/libc.so.6"

[5] #3  ?? in "/home/cwang5/OpenFOAM/OpenFOAM-1.6.x/lib/linux64GccDPOpt/libOpenFOAM.so"

[3] #2   in "/lib64/libc.so.6"

[1] #3  ????main in "/lib64/libc.so.6"

[3] #3  main in "/lib64/libc.so.6"

[2] #3   in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[1] #4  __libc_start_main in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[5] #4  __libc_start_mainmainmain in "/lib64/libc.so.6"

[5] #5   in "/lib64/libc.so.6"

[1] #5   in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[3] #4  __libc_start_main in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[2] #4  __libc_start_main__gxx_personality_v0 in "/lib64/libc.so.6"

[3] #5   in "/lib64/libc.so.6"

[2] #5  __gxx_personality_v0 in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[compute-02:00747] *** Process received signal ***

[compute-02:00747] Signal: Floating point exception (8)

[compute-02:00747] Signal code:  (-6)

[compute-02:00747] Failing at address: 0x3ec000002eb

[compute-02:00747] [ 0] /lib64/libc.so.6 [0x2b08b8c25c10]

[compute-02:00747] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2b08b8c25b95]

[compute-02:00747] [ 2] /lib64/libc.so.6 [0x2b08b8c25c10]

[compute-02:00747] [ 3] flapFoam [0x41b52e]

[compute-02:00747] [ 4] /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b08b8c13154]

[compute-02:00747] [ 5] flapFoam(__gxx_personality_v0+0xe9) [0x419a39]

[compute-02:00747] *** End of error message ***

__gxx_personality_v0--------------------------------------------------------------------------

mpirun noticed that process rank 1 with PID 747 on node compute-02 exited on signal 8 (Floating point exception).

--------------------------------------------------------------------------

__gxx_personality_v0 in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[compute-02:00751] *** Process received signal ***

[compute-02:00751] Signal: Floating point exception (8)

[compute-02:00751] Signal code:  (-6)

[compute-02:00751] Failing at address: 0x3ec000002ef

[compute-02:00751] [ 0] /lib64/libc.so.6 [0x2b51ae9a9c10]

[compute-02:00751] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2b51ae9a9b95]

[compute-02:00751] [ 2] /lib64/libc.so.6 [0x2b51ae9a9c10]

[compute-02:00751] [ 3] flapFoam [0x41b52e]

[compute-02:00751] [ 4] /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b51ae997154]

[compute-02:00751] [ 5] flapFoam(__gxx_personality_v0+0xe9) [0x419a39]

[compute-02:00751] *** End of error message ***

 in "/home/cwang5/OpenFOAM/cwang5-1.6.x/applications/bin/linux64GccDPOpt/flapFoam"

[compute-02:00749] *** Process received signal ***

[compute-02:00749] Signal: Floating point exception (8)

[compute-02:00749] Signal code:  (-6)

[compute-02:00749] Failing at address: 0x3ec000002ed

[compute-02:00749] [ 0] /lib64/libc.so.6 [0x2b1e9433ac10]

[compute-02:00749] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2b1e9433ab95]

[compute-02:00749] [ 2] /lib64/libc.so.6 [0x2b1e9433ac10]

[compute-02:00749] [ 3] flapFoam [0x41b52e]

[compute-02:00749] [ 4] /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b1e94328154]

[compute-02:00749] [ 5] flapFoam(__gxx_personality_v0+0xe9) [0x419a39]

[compute-02:00749] *** End of error message ***

The error seems to occur during the update.mesh() portion of the code based on the displayed information. Does anyone know what might have cause such error and if there's a way to work around/resolve the issue? Thanks

John

EDIT: btw, I'm using OpenFOAM 1.6.x, and is a complete noob when comes to C++ coding in OpenFOAM

Ercoftac centrifugal pump openFoam case study, bug

Hey,

I am trying to run the ERCOFTAC centrifugal pump OpenFOAM case-study shared on the openFoamwiki by people from Hydro Quebec and chalmers university of Technology.
I am using OpenFoam-1.6-EXT and the solver MRFSimpleFoam.
In the first iterations, the compilation is going well (I want to run the case on 5000 iterations) but later (at about 1400 iterations), there is a bug and the compilation is stopped.

Here is what is written on the log file:

smoothSolver: Solving for Ux, Initial residual = 1.4911964e-10, Final residual = 1.4911964e-10, No Iterations 0
smoothSolver: Solving for Uy, Initial residual = 8.9358568e-10, Final residual = 8.9358568e-10, No Iterations 0
GAMG: Solving for p, Initial residual = 0.98305543, Final residual = 0.0170324, No Iterations 1
time step continuity errors : sum local = 2.9330002e+41, global = 1.7750351e+29, cumulative = 1.7750353e+29
smoothSolver: Solving for epsilon, Initial residual = 6.4743276e-16, Final residual = 6.4743276e-16, No Iterations 0
smoothSolver: Solving for k, Initial residual = 6.2646515e-12, Final residual = 6.2646515e-12, No Iterations 0
ExecutionTime = 711.17 s ClockTime = 1455 s

GGI pair (GGI_INT, GGI_EXT) : 8.2404281e+25 2.921623e+26 Diff = 1.8559864e+25 or 22.522937 %
Time = 1413

smoothSolver: Solving for Ux, Initial residual = 2.7363363e-22, Final residual = 2.7363363e-22, No Iterations 0
smoothSolver: Solving for Uy, Initial residual = 4.2314383e-22, Final residual = 4.2314383e-22, No Iterations 0
GAMG: Solving for p, Initial residual = 4.3439669e-06, Final residual = 4.1146215e-08, No Iterations 1
time step continuity errors : sum local = 3.6948953e+68, global = -5.2860233e+65, cumulative = -5.2860233e+65
smoothSolver: Solving for epsilon, Initial residual = 1, Final residual = 0.05612422, No Iterations 4
[sycomore:01822] *** Process received signal ***
[sycomore:01822] Signal: Floating point exception (8)
[sycomore:01822] Signal code: (-6)
[sycomore:01822] Failing at address: 0x4cd0000071e
[sycomore:01822] [ 0] /lib64/libc.so.6 [0x3e19a30280]
[sycomore:01822] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x3e19a30215]
[sycomore:01822] [ 2] /lib64/libc.so.6 [0x3e19a30280]
[sycomore:01822] [ 3] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam19GaussSeidelSmoother6smoot hERNS_5FieldIdEERKNS_9lduMatrixERKS2_RKNS_10FieldF ieldIS1_dEERKNS_8UPtrListIKNS_17lduInterfaceFieldE EEhi+0x4b0) [0x2b67f41e0b90]
[sycomore:01822] [ 4] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam19GaussSeidelSmoother6smoo thERNS_5FieldIdEERKS2_hi+0x28) [0x2b67f41e0d38]
[sycomore:01822] [ 5] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam12smoothSolver5solveERNS_5 FieldIdEERKS2_h+0x2b0) [0x2b67f41da980]
[sycomore:01822] [ 6] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libfiniteVolume.so(_ZN4Foam8fvMatrixIdE5solveERKNS _10dictionaryE+0x16b) [0x2b67f337158b]
[sycomore:01822] [ 7] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libincompressibleRASModels.so(_ZN4Foam5solveIdEENS _9lduMatrix17solverPerformanceERKNS_3tmpINS_8fvMat rixIT_EEEE+0x52) [0x2b67f26a9e52]
[sycomore:01822] [ 8] /usr1/appli/openfoam/OpenFOAM-1.6-EXT/OpenFOAM-1.6-ext/lib/linux64GccDPOpt/libincompressibleRASModels.so(_ZN4Foam14incompress ible9RASModels8kEpsilon7correctEv+0x10fb) [0x2b67f269f6ab]
[sycomore:01822] [ 9] MRFSimpleFoam [0x41533a]
[sycomore:01822] [10] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e19a1d974]
[sycomore:01822] [11] MRFSimpleFoam(_ZNK4Foam11regIOobject11writeObjectE NS_8IOstream12streamFormatENS1_13versionNumberENS1 _15compressionTypeE+0xc1) [0x412859]
[sycomore:01822] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 1822 on node sycomore.localdomain exited on signal 8 (Floating point exception).

I have noticed that before the bug, the values of the amount of flux going through the GGI patched pair GGI_INT and GGI_ext are completly different. May be there is a problem with the mesh and GGI...

Does anybody have the same problem?
The ercofact case study has been run with OpenFoam-1.5-dev, so do you think i have a problem with the version of openFoam, as i am using the 1.6-ext?

I need your help!!

Ariane