multiphaseEulerFoam (OF2.3.0) : Courant number explodes when running in parallel
 

Hi,

I am doing simulations using "multiphaseEulerFoam" (OpenFOAM 2.3.0).
I started by the tutorial (bubble column) before trying more complicated geometry.

My problem is that when I use a fine mesh (of course I reduce the time step "delta t"), OpenFoam craches (the courant number increases suddenly to ~ 1e+100 !!) after a certain time (after permanent flow is well established) when running in parallel.

The strange thing is that when I don't run the "multiphaseEulerFoam" solver in parallel, everything is going well and the Courant number remains < 0.4

Do you have an idea about the origin of the problem ?
How can I fix this ?

Thank you for your help.

Mehrez


PS : Copy of the end of the log file :
__________________________________________________ __________________


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 0.0197451 max: 0.0684915
Time = 9.625

MULES: Solving for alpha.air
air volume fraction, min, max = 0.465748 0.000119881 1
MULES: Solving for alpha.water
water volume fraction, min, max = 0.53486 0 0.999981
Phase-sum volume fraction, min, max = 1.00061 0.0042213 1.72497
MULES: Solving for alpha.air
air volume fraction, min, max = 0.465796 0.000120519 1
MULES: Solving for alpha.water
water volume fraction, min, max = 0.534812 0 0.999981
Phase-sum volume fraction, min, max = 1.00061 0.00422589 1.71204
GAMG: Solving for p, Initial residual = 0.00139639, Final residual = 5.81202e-06, No Iterations 3
time step continuity errors : sum local = 2.41125e-08, global = -1.30472e-08, cumulative = -0.000127697
GAMG: Solving for p, Initial residual = 0.00773661, Final residual = 4.7937e-10, No Iterations 14
time step continuity errors : sum local = 1.99335e-12, global = 2.62437e-13, cumulative = -0.000127697
ExecutionTime = 798.17 s


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 0.0198142 max: 0.536377
Time = 9.6275

MULES: Solving for alpha.air
air volume fraction, min, max = 0.465856 0.000121156 1
MULES: Solving for alpha.water
water volume fraction, min, max = 0.53476 0 0.999981
Phase-sum volume fraction, min, max = 1.00062 0.00423059 1.56304
MULES: Solving for alpha.air
air volume fraction, min, max = 0.465919 0.000121796 1
MULES: Solving for alpha.water
water volume fraction, min, max = 0.534714 0 0.999981
Phase-sum volume fraction, min, max = 1.00063 0.00423533 1.56304
GAMG: Solving for p, Initial residual = 0.0102849, Final residual = 3.65886e-05, No Iterations 3
time step continuity errors : sum local = 1.08056e-07, global = 5.29818e-08, cumulative = -0.000127644
GAMG: Solving for p, Initial residual = 0.528984, Final residual = 3.99465e-10, No Iterations 20
time step continuity errors : sum local = 2.48252e-12, global = -4.20296e-13, cumulative = -0.000127644
ExecutionTime = 798.4 s


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 0.0442704 max: 69.0571
Time = 9.63

MULES: Solving for alpha.air
air volume fraction, min, max = 0.467749 -0.335673 23.0504
MULES: Solving for alpha.water
water volume fraction, min, max = 0.534738 -1.39541 1.65974
Phase-sum volume fraction, min, max = 1.00249 -0.335673 23.6134
MULES: Solving for alpha.air
air volume fraction, min, max = 0.468363 -603.363 190.594
MULES: Solving for alpha.water
water volume fraction, min, max = 0.535464 -11.6784 27.1617
Phase-sum volume fraction, min, max = 1.00383 -603.236 217.755
GAMG: Solving for p, Initial residual = 0.93761, Final residual = 0.0042504, No Iterations 3
time step continuity errors : sum local = 0.00868465, global = 0.00218753, cumulative = 0.00205988
GAMG: Solving for p, Initial residual = 0.597729, Final residual = 7.29283e-10, No Iterations 18
time step continuity errors : sum local = 9.40381e-09, global = -2.04498e-09, cumulative = 0.00205988
ExecutionTime = 798.63 s


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 2.31929 max: 2036.06
Time = 9.6325

MULES: Solving for alpha.air
air volume fraction, min, max = -3.06138 -234880 453771
MULES: Solving for alpha.water
water volume fraction, min, max = 0.368301 -29517.2 14142.4
Phase-sum volume fraction, min, max = -2.69308 -235405 467914
MULES: Solving for alpha.air
air volume fraction, min, max = 102602 -4.0891e+08 2.16309e+09
MULES: Solving for alpha.water
water volume fraction, min, max = 1570.57 -2.35245e+07 4.54948e+07
Phase-sum volume fraction, min, max = 104172 -4.32434e+08 2.1622e+09
GAMG: Solving for p, Initial residual = 0.999469, Final residual = 0.00694096, No Iterations 3
time step continuity errors : sum local = 207305, global = -113118, cumulative = -113118
GAMG: Solving for p, Initial residual = 7.21206e-06, Final residual = 2.57156e-10, No Iterations 9
time step continuity errors : sum local = 2.27507e+06, global = 1.12627e+06, cumulative = 1.01315e+06
ExecutionTime = 798.83 s


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 2.39491e+11 max: 3.23227e+14
Time = 9.635

MULES: Solving for alpha.air
air volume fraction, min, max = -1.68871e+17 -2.39138e+22 1.91865e+22
MULES: Solving for alpha.water
water volume fraction, min, max = -4.84919e+16 -5.94328e+21 4.46871e+21
Phase-sum volume fraction, min, max = -2.17363e+17 -2.9857e+22 2.36552e+22
MULES: Solving for alpha.air
air volume fraction, min, max = 5.25141e+41 -2.25881e+45 9.02144e+45
MULES: Solving for alpha.water
water volume fraction, min, max = 1.79355e+45 -5.82929e+42 3.03974e+49
Phase-sum volume fraction, min, max = 1.79408e+45 -5.481e+39 3.03952e+49
GAMG: Solving for p, Initial residual = 1, Final residual = 0.00883942, No Iterations 3
time step continuity errors : sum local = 4.98206e+57, global = -1.53054e+55, cumulative = -1.53054e+55
GAMG: Solving for p, Initial residual = 2.89028e-16, Final residual = 2.89028e-16, No Iterations 0
time step continuity errors : sum local = 4.49754e+93, global = -1.81398e+77, cumulative = -1.81398e+77
ExecutionTime = 799.01 s


fieldAverage fieldAverage1 output:
Calculating averages

Courant Number mean: 2.41532e+93 max: 1.40595e+97
Time = 9.6375

MULES: Solving for alpha.air
air volume fraction, min, max = 7.57427e+121 -1.03072e+142 1.05773e+142
MULES: Solving for alpha.water
water volume fraction, min, max = 6.52985e+118 -6.83465e+138 6.66015e+138
Phase-sum volume fraction, min, max = 1.99642e+122 -1.03006e+142 1.05705e+142
[0] #0 Foam::error::printStack(Foam::Ostream&) in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #1 Foam::sigFpe::sigHandler(int) in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #2 in "/lib64/libc.so.6"
[0] #3 Foam::multiply(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&) in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #4 Foam::tmp<Foam::GeometricField<double, Foam::fvsPatchField, Foam::surfaceMesh> > Foam::operator*<Foam::fvsPatchField, Foam::surfaceMesh>(Foam::GeometricField<double, Foam::fvsPatchField, Foam::surfaceMesh> const&, Foam::tmp<Foam::GeometricField<double, Foam::fvsPatchField, Foam::surfaceMesh> > const&) in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/multiphaseEulerFoam"
[0] #5 Foam::fv::gaussConvectionScheme<double>::flux(Foam ::GeometricField<double, Foam::fvsPatchField, Foam::surfaceMesh> const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&) const in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
[0] #6 Foam::tmp<Foam::GeometricField<double, Foam::fvsPatchField, Foam::surfaceMesh> > Foam::fvc::flux<double>(Foam::tmp<Foam::GeometricF ield<double, Foam::fvsPatchField, Foam::surfaceMesh> > const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::word const&) in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so"
[0] #7 Foam::multiphaseSystem::solveAlphas() in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so"
[0] #8 Foam::multiphaseSystem::solve() in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so"
[0] #9
[0] in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/multiphaseEulerFoam"
[0] #10 __libc_start_main in "/lib64/libc.so.6"
[0] #11
[0] in "/chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/multiphaseEulerFoam"
[tem110:04262] *** Process received signal ***
[tem110:04262] Signal: Floating point exception (8)
[tem110:04262] Signal code: (-6)
[tem110:04262] Failing at address: 0x2c50b000010a6
[tem110:04262] [ 0] /lib64/libc.so.6(+0x326a0)[0x2b064384a6a0]
[tem110:04262] [ 1] /lib64/libc.so.6(gsignal+0x35)[0x2b064384a625]
[tem110:04262] [ 2] /lib64/libc.so.6(+0x326a0)[0x2b064384a6a0]
[tem110:04262] [ 3] /chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam8multiplyERNS_5FieldIdEERKN S_5UListIdEES6_+0xd5)[0x2b0642a15a05]
[tem110:04262] [ 4] multiphaseEulerFoam(_ZN4FoammlINS_13fvsPatchFieldE NS_11surfaceMeshEEENS_3tmpINS_14GeometricFieldIdT_ T0_EEEERKS7_RKS8_+0x16c)[0x47c6dc]
[tem110:04262] [ 5] /chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so(_ZNK4Foam2fv21gaussConvectionSc hemeIdE4fluxERKNS_14GeometricFieldIdNS_13fvsPatchF ieldENS_11surfaceMeshEEERKNS3_IdNS_12fvPatchFieldE NS_7volMeshEEE+0x28)[0x2b0641a75018]
[tem110:04262] [ 6] /chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so(_ZN4Foam3fvc4fluxIdEENS_3tm pINS_14GeometricFieldIT_NS_13fvsPatchFieldENS_11su rfaceMeshEEEEERKNS2_INS3_IdS5_S6_EEEERKNS3_IS4_NS_ 12fvPatchFieldENS_7volMeshEEERKNS_4wordE+0x74)[0x2b063ff221c4]
[tem110:04262] [ 7] /chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so(_ZN4Foam16multiphaseSystem1 1solveAlphasEv+0xa3d)[0x2b063ff0f8bd]
[tem110:04262] [ 8] /chemeng/ttreerat/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libmultiphaseSystem.so(_ZN4Foam16multiphaseSystem5 solveEv+0x517)[0x2b063ff11007]
[tem110:04262] [ 9] multiphaseEulerFoam[0x466893]
[tem110:04262] [10] /lib64/libc.so.6(__libc_start_main+0xfd)[0x2b0643836d5d]
[tem110:04262] [11] multiphaseEulerFoam[0x427231]
[tem110:04262] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 4262 on node tem110 exited on signal 8 (Floating point exception).

__________________________________________________ __________________

Hi,
without knowing the contents of your controlDict, you could try setting your maxCo (the maximum Courant number) to a reasonably small value and using adjustTimeStep instead of a fixed time step.

The parallel vs. linear behavior sounds strange, a bit similar(?) to this post: http://www.cfd-online.com/Forums/ope...ty-result.html

Good luck!

Dear mnikku,

Thank you for your answer.
Using adaptable time step to keep the Co number at a reasonable small value (0.01) didn't improve the stability...
Mehrez

Hi, I face some stability problems with multiphaseEulerFoam, too, and I have difficulties to find its source. Any news so far?

OK it was my rheology model and the difference of solving for p_rgh or p.
Now it works fine, I modified the code to allow non-newtonian rheology models and coupled the solver to lagrangian particles using kinematicCollidingCloud, in case someone is interested.

Hi
i faced with this problem with twoPhaseEulerFoam in OP 2.2.
i don't know it is about parallel calculating or not but i know Courant overshooting could be affected by type of Geometry, for example with tri meshing it is sometimes more sensitive to diverge another parameter is using maxCo option in controldict , u can find it in many tutorials of OP and copy it to your controlDict.

vonboett, I sent you a private message about your code modifications, if you may, please take a look :)

Hello,
I fixed my problem by using snappyHexMesh to create hex meshes. It seems that any thing other than hex leads to unstability.
I also recommend OF3 instead of OF2.3. There was a bug report about the solver that has been fixed in OF3. http://bugs.openfoam.org/view.php?id=1379
Mhrz

Hi Mehrez,

are you able to produce pure Hex meshes with SnappyHexMesh? I'm also using twoPhaseEulerFoam but so far i have difficulties to obtain pure hexadral meshes, which the Euler-Euler class solvers seem to require..

You cannot produce pure hex meshes with snappyHexMesh if your geometry is complex (by the way, can you share your geometry to have an idea). But if you have some polyhedral elements in your mesh, doesn't mean that your simulation will become unstable (from my experience).
What I suggest you, if you are comfortable with snappyHexMesh (SHM), is to use as initial hex mesh a mesh that is slightly closer to your geometry by creating it with Salome or gmsh. I think that starting the meshing with SHM from a box (meshed with hex elements using blockMesh) is not the best choice to do if your geometry is complex.
Mehrez

Yes, i agree. Full hexa mesh is not possible with SHM. I haven't even figured out how to produce sharp (90°) edges with it, for example the edges on a rectangular tank. My geometry is basically a tank with a pipe as inlet. Nothing to fency but the round pipe inlet stuff is what creates wedges, tets and so one, which twophaseeulerfoam is known to not like.
Maybe i can find some time to look into Salome, which sounds promissing at first glance..