CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM (https://www.cfd-online.com/Forums/openfoam/)
-   -   decomposePar, 4 processors (https://www.cfd-online.com/Forums/openfoam/107520-decomposepar-4-processors.html)

nakor September 29, 2012 16:31
decomposePar, 4 processors
 
Hello,
I have a case which is running just fine with 1 processor, 2 processor (decomposition method : scotch) but when I try to run it with 4 processors, it is not working anymore.
I did tried two differents decompositions methods without any success.

The solver is starting, it is not diverging, but after some iterations, it stops with this error :
Code:
[2] #0  Foam::error::printStack(Foam::Ostream&)[1] #0  Foam::error::printStack(Foam::Ostream&) in "/opt/openfoam211/platforms in "/opt/openfoam211/platforms/linux64GccDPO/linux64GccDPOpt/lib/libOpenpt/lib/libOpenFOAM.so"
[1] #1  FOAM.so"
[2] #1  Foam::sigFpe::sigHandler(int)Foam::sigFpe::sigHandler(int) in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #2  in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #3  Foam::GAMGSolver::scalingFactor(Foam::Field<double>&, Foam::Field<double> const&, Foam::Field<double> const&, Foam::Field<double> const&) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #2  in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #4  Foam::GAMGSolver::scalingFactor(Foam::Field<double>&, Foam::lduMatrix const&, Foam::Field<double>&, Foam::FieldField<Foam::Field, double> const&, Foam::UPtrList<Foam::lduInterfaceField const> const&, Foam::Field<double> const&, unsigned char) const in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #3  Foam::GAMGSolver::scalingFactor(Foam::Field<double>&, Foam::Field<double> const&, Foam::Field<double> const&, Foam::Field<double> const&) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #5  Foam::GAMGSolver::Vcycle(Foam::PtrList<Foam::lduMatrix::smoother> const&, Foam::Field<double>&, Foam::Field<double> const&, Foam::Field<double>&, Foam::Field<double>&, Foam::Field<double>&, Foam::PtrList<Foam::Field<double> >&, Foam::PtrList<Foam::Field<double> >&, unsigned char) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #4  Foam::GAMGSolver::scalingFactor(Foam::Field<double>&, Foam::lduMatrix const&, Foam::Field<double>&, Foam::FieldField<Foam::Field, double> const&, Foam::UPtrList<Foam::lduInterfaceField const> const&, Foam::Field<double> const&, unsigned char) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #6  Foam::GAMGSolver::solve(Foam::Field<double>&, Foam::Field<double> const&, unsigned char) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #5  Foam::GAMGSolver::Vcycle(Foam::PtrList<Foam::lduMatrix::smoother> const&, Foam::Field<double>&, Foam::Field<double> const&, Foam::Field<double>&, Foam::Field<double>&, Foam::Field<double>&, Foam::PtrList<Foam::Field<double> >&, Foam::PtrList<Foam::Field<double> >&, unsigned char) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #7  Foam::fvMatrix<double>::solve(Foam::dictionary const&) in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #6  Foam::GAMGSolver::solve(Foam::Field<double>&, Foam::Field<double> const&, unsigned char) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
[2] #8  in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #7  Foam::fvMatrix<double>::solve(Foam::dictionary const&)Foam::fvMatrix<double>::solve() in "/home/caelinux/OpenFOAM/caelinux-2.1.1/platforms/linux64GccDPOpt/bin/mychtMultiRegionSimpleFoam"
[2] #9 
 in "/opt/openfoam21[2]  in "/hom1e/caelinux/OpenFOAM/caelinux-2.1.1/platforms/linux64GccDPOpt/bin/mychtMultiRegionSimpleFoam"
[2] #10  __libc_start_main/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
[1] #8  in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #11  Foam::fvMatrix<double>::solve()
[2]  in "/home/caelinux/OpenFOAM/caelinux-2.1.1/platforms/linux64GccDPOpt/bin/mychtMultiRegionSimpleFoam"
[laptop:13776] *** Process received signal ***
[laptop:13776] Signal: Floating point exception (8)
[laptop:13776] Signal code:  (-6)
[laptop:13776] Failing at address: 0x3e8000035d0
 in "/home/caelinux/OpenFOAM/[romain-laptop:13776] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x364c0) [0x7f37f270f4c0]
[laptop:13776] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f37f270f445]
[laptop:13776] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x364c0) [0x7f37f270f4c0]
[laptop:13776] [ 3] /opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZNK4Foam10GAMGSolver13scalingFactorERNS_5FieldIdEERKS2_S5_S5_+0x77) [0x7f37f374b197]
[laptop:13776] [ 4] /opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZNK4Foam10GAMGSolver13scalingFactorERNS_5FieldIdEERKNS_9lduMatrixES3_RKNS_10FieldFieldIS1_dEERKNS_8UPtrListIKNS_17lduInterfaceFieldEEERKS2_h+0xa6) [0x7f37f374b426]
[romain-laptop:13776] [ 5] /opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZNK4Foam10GAMGSolver6VcycleERKNS_7PtrListINS_9lduMatrix8smootherEEERNS_5FieldIdEERKS8_S9_S9_S9_RNS1_IS8_EESD_h+0xfa0) [0x7f37f374dc90]
[laptop:13776] [ 6] /opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZNK4Foam10GAMGSolver5solveERNS_5FieldIdEERKS2_h+0x4c3) [0x7f37f374ec23]
[laptop:13776] [ 7] /opt/openfoam211/platforms/linux64GccDPOpt/lib/libfiniteVolume.so(_ZN4Foam8fvMatrixIdE5solveERKNS_10dictionaryE+0x137) [0x7f37f5875307]
[laptop:13776] [ 8] mychtMultiRegionSimpleFoam(_ZN4Foam8fvMatrixIdE5solveEv+0xc0) [0x432530]
[laptop:13776] [ 9] mychtMultiRegionSimpleFoam() [0x422498]
[laptop:13776] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f37f26fa76d]
[laptop:13776] [11] mychtMultiRegionSimpleFoam() [0x424c8d]
[laptop:13776] *** End of error message ***
caeli--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 13776 on node romain-laptop exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
Final Time 22:23:06
Simulation Time: 29 secondes
Simulation Time: 0 minutes
Any ideas why this could happen ? I am using OpenFoam 211.

Thanks for your help.

nimasam September 30, 2012 03:37
instead of GAMG solver, use another solver to calculate matrix

Bernhard September 30, 2012 04:32
Nimasam, I never had any problems with running GAMG on multiple processors. Why could it be the issue here?

Nakor, can you give more details about the solver and system you're using?

nakor September 30, 2012 15:22
Quote:
Originally Posted by nimasam (Post 384231)
instead of GAMG solver, use another solver to calculate matrix
Thanks, it works that way. Any explication about why it was working for 2 processors but not anymore with 4 ?

@Bernhard
The solver is a modified version of chtMultiRegionSimpleFoam, but I did also tried with chtMultiRegionFoam without any more success so it does not come from my modifications to the solver.

To describe my case in a few words, there are a solid domain, and two liquids domains, and I am mainly interested by the thermal aspect of the simulation since most of the fluid does not move (only convection near the heated walls)


All times are GMT -4. The time now is 14:04.