[sharonyue]

Hi guys,

I talked with many guys about this issue, they have faced this problem once or twice when we are running some cases. to put in simple, that is when u succeed in running a case with only a single core in your machine, but it will blow up in parallel.

I happened to get into this trouble these day, 2 yrs ago when I was doing some training on OpenFOAM I accidentally run into this problem, but I neglected it. However, when I wanna speed up my simulation, I can not use a single core.

This is the log file:

single core:

PIMPLE: Operating solver in PISO mode


Starting time loop

Courant Number mean: 0.0394037 max: 0.49847
Max Ur Courant Number = 0.0929619
deltaT = 0.000726744
Time = 5.00073

MULES: Solving for alpha.air
MULES: Solving for alpha.air
alpha.air volume fraction = 0.00199999  Min(alpha1) = 5.25086e-05  Max(alpha1) = 0.0298173
DILUPBiCG:  Solving for TY, Initial residual = 0.000198921, Final residual = 2.11904e-17, No Iterations 8
GAMG:  Solving for p, Initial residual = 0.00891682, Final residual = 3.51213e-05, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.00150613, Final residual = 1.24395e-05, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.000241716, Final residual = 7.33671e-08, No Iterations 5
smoothSolver:  Solving for epsilonm, Initial residual = 2.29257e-05, Final residual = 6.22539e-08, No Iterations 4
smoothSolver:  Solving for km, Initial residual = 3.06159e-05, Final residual = 7.12566e-08, No Iterations 6
ExecutionTime = 2.49 s

Courant Number mean: 0.0395294 max: 0.499942
Max Ur Courant Number = 0.095134
deltaT = 0.000726744
Time = 5.00145

MULES: Solving for alpha.air
MULES: Solving for alpha.air
alpha.air volume fraction = 0.00199999  Min(alpha1) = 5.2494e-05  Max(alpha1) = 0.0298275
DILUPBiCG:  Solving for TY, Initial residual = 0.000200717, Final residual = 1.09388e-17, No Iterations 8
GAMG:  Solving for p, Initial residual = 0.00903888, Final residual = 3.94824e-05, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.000881921, Final residual = 6.50462e-06, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.000136874, Final residual = 6.05967e-08, No Iterations 5
smoothSolver:  Solving for epsilonm, Initial residual = 1.21255e-05, Final residual = 5.58432e-08, No Iterations 4
smoothSolver:  Solving for km, Initial residual = 2.34679e-05, Final residual = 7.04817e-08, No Iterations 6
ExecutionTime = 4.27 s

parallel:

PIMPLE: Operating solver in PISO mode


Starting time loop

Courant Number mean: 0.0394037 max: 0.49847
Max Ur Courant Number = 0.0929619
deltaT = 0.000726744
Time = 5.00073

MULES: Solving for alpha.air
MULES: Solving for alpha.air
alpha.air volume fraction = 0.00199999  Min(alpha1) = 5.25086e-05  Max(alpha1) = 0.0298173
DILUPBiCG:  Solving for TY, Initial residual = 0.000198921, Final residual = 5.953e-17, No Iterations 9
GAMG:  Solving for p, Initial residual = 0.00918711, Final residual = 3.26779e-05, No Iterations 4
GAMG:  Solving for p, Initial residual = 0.001776, Final residual = 7.50224e-06, No Iterations 4
GAMG:  Solving for p, Initial residual = 0.000296505, Final residual = 3.69827e-08, No Iterations 7
smoothSolver:  Solving for epsilonm, Initial residual = 2.41189e-05, Final residual = 7.0696e-08, No Iterations 4
smoothSolver:  Solving for km, Initial residual = 3.18021e-05, Final residual = 7.4419e-08, No Iterations 6
ExecutionTime = 1.19 s

Courant Number mean: 0.0395316 max: 0.499996
Max Ur Courant Number = 0.0951141
deltaT = 0.000726744
Time = 5.00145

MULES: Solving for alpha.air
MULES: Solving for alpha.air
alpha.air volume fraction = 0.00199999  Min(alpha1) = 5.24938e-05  Max(alpha1) = 0.0298275
DILUPBiCG:  Solving for TY, Initial residual = 0.00020167, Final residual = 6.63252e-14, No Iterations 4001
GAMG:  Solving for p, Initial residual = 0.00912649, Final residual = 7.89893e-05, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.00105189, Final residual = 4.65768e-06, No Iterations 4
GAMG:  Solving for p, Initial residual = 0.000162372, Final residual = 4.39752e-08, No Iterations 6
^Cmpirun: killing job...

Actually it didnot blow up in parallel after this time step. but Im sure it will. Iterations number is way too high its not a common sense.

Additional information is :
model: two fluid model
solver: tailor-made solver.
fvsolutions for TY:

    {
        solver          PBiCG;
        preconditioner  DILU;
        tolerance       1e-17;//
        relTol          0;
    }

Any comments is appreciated.Thanks

[sharonyue]

I found some threads on this:

http://www.cfd-online.com/Forums/ope...tml#post246561

http://www.cfd-online.com/Forums/ope...terations.html

looks like DILUPBiCG is incompatible with parallel running sometimes.. I should use GAMG I think