[basneb]

Hello everybody,

I'm running simpleFoam simulations in double precision and in order to speed them up, I would like to switch to single precision. This works fine in the beginning of the simulation and the simulation is about 40% quicker (in terms of simulation time per iteration), but suddenly the simulation diverges. I get an error message that states that there is a "floating point exception". I don't have bounding epsilon or k before the simulation blows up. Does anyone of you know, what could be the reason? It would help me so much.

Best regards,
Bastian.

[luca_g]

Dear Bastian,

it is definitely worth running single precision but with turbulence model it is likely that occasional underflow/overflow happens in single precision.
This causes exception and ends a computation which was otherwise going fine. To avoid it edit the file bashrc in folder etc and comment the line
export FOAM_SIGFPE=
so that exception will not be raised anymore.

Regards,

Luca

[basneb]

Dear Luca,

that sounds great, but will the change influence the result of the computations in some way or is this just a thing to trick the computer?

Thx already and best regards,

Bastian

[luca_g]

Dear Bastian,

In my experience it will have absolutely no effect (apart from the slight difference you might anyway find switching from double to float), but you might want to look at the particular turbulence model you are using and figure out which part of it (likely damping function or similars) is causing it, to be sure it will not compromise the results.

Regards,

Luca

[basneb]

Dear Luca,

ahh okay, then I will just run some cases in single precision and check out how the results change or if they are still acceptable.

Best,

Bastian

[basneb]

Hi Luca,

I tried the solution, you supposed, but I still have the same problem. The simulation stops, because of a floating point exception. In the following I attach the error-message, which I get in the log-file. Hopefully somebody else had a similar problem already.

Regards,

Bastian

Here comes the error message:

 
Time = 191
DILUPBiCG:  Solving for Ux, Initial residual = 0.0183009, Final residual = 0.00139391, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.0260962, Final residual = 0.00202694, No Iterations 2
DILUPBiCG:  Solving for Uz, Initial residual = 0.0598852, Final residual = 0.00420837, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.833212, Final residual = 0.0341513, No Iterations 2
time step continuity errors : sum local = 0.0358474, global = 4.17163e-05, cumulative = 0.0421732
DILUPBiCG:  Solving for epsilon:  solution singularity
bounding epsilon, min: 2.45278e-09 max: 1.18469e+18 average: 3.41778e+11
DILUPBiCG:  Solving for k, Initial residual = 0.419389, Final residual = 3.7141e-15, No Iterations 25
bounding k, min: -1810.15 max: 2.95828e+10 average: 311408
ExecutionTime = 1847.54 s  ClockTime = 1874 s
Time = 192
DILUPBiCG:  Solving for Ux, Initial residual = 0.0130778, Final residual = 0.000781722, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.0188701, Final residual = 0.000931275, No Iterations 2
DILUPBiCG:  Solving for Uz, Initial residual = 0.0457371, Final residual = 0.00249969, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.780956, Final residual = 0.0524846, No Iterations 2
time step continuity errors : sum local = 0.0444382, global = 0.000165461, cumulative = 0.0423386
[3] #0  [1] #0  [2] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[0] #1  Foam::sigFpe::sigFpeHandler(int) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[3] #1  Foam::sigFpe::sigFpeHandler(int) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[1] #1  Foam::sigFpe::sigFpeHandler(int) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[2] #1  Foam::sigFpe::sigFpeHandler(int) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[0] #2   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[3] #2   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[1] #2   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[2] #2  ???????? in "/lib64/libc.so.6"
[0] #3  Foam::PBiCG::solve(Foam::Field<float>&, Foam::Field<float> const&, unsigned char) const in "/lib64/libc.so.6"
[3] #3  Foam::PBiCG::solve(Foam::Field<float>&, Foam::Field<float> const&, unsigned char) const in "/lib64/libc.so.6"
[1] #3  Foam::PBiCG::solve(Foam::Field<float>&, Foam::Field<float> const&, unsigned char) const in "/lib64/libc.so.6"
[2] #3  Foam::PBiCG::solve(Foam::Field<float>&, Foam::Field<float> const&, unsigned char) const in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[0] #4  Foam::fvMatrix<float>::solve(Foam::Istream&) in "/vcc/a in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[1] #4  Foam::fvMatrix<float>::solve(Foam::Istream&)ns/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[3] #4  Foam::fvMatrix<float>::solve(Foam::Istream&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so"
[2] #4  Foam::fvMatrix<float>::solve(Foam::Istream&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libfiniteVolume.so"
[0] #5  Foam::lduMatrix::solverPerformance Foam::solve<float>(Foam::tmp<Foam::fvMatrix<float> > const&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libfiniteVolume.so"
[3] #5  Foam::lduMatrix::solverPerformance Foam::solve<float>(Foam::tmp<Foam::fvMatrix<float> > const&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libfiniteVolume.so"
[1] #5  Foam::lduMatrix::solverPerformance Foam::solve<float>(Foam::tmp<Foam::fvMatrix<float> > const&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libfiniteVolume.so"
[2] #5  Foam::lduMatrix::solverPerformance Foam::solve<float>(Foam::tmp<Foam::fvMatrix<float> > const&) in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[0] #6  Foam::incompressible::RASModels::realizableKE::correct() in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[3] #6  Foam::incompressible::RASModels::realizableKE::correct() in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[1] #6  Foam::incompressible::RASModels::realizableKE::correct() in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[2] #6  Foam::incompressible::RASModels::realizableKE::correct() in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[0] #7   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[3] #7   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[1] #7   in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so"
[2] #7  mainmainmainmain in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam"
[0] #8  __libc_start_main in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam"
[3] #8  __libc_start_main in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam"
[1] #8  __libc_start_main in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam"
[2] #8  __libc_start_main in "/lib64/libc.so.6"
[3] #9  Foam::regIOobject::readIfModified() in "/lib64/libc.so.6"
[0] #9  Foam::regIOobject::readIfModified() in "/lib64/libc.so.6"
[1] #9  Foam::regIOobject::readIfModified() in "/lib64/libc.so.6"
[2] #9  Foam::regIOobject::readIfModified() in "/vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam"
[gbwcs7-21:17607] *** Process received signal ***
[gbwcs7-21:17607] Signal: Floating point exception (8)
[gbwcs7-21:17607] Signal code:  (-6)
[gbwcs7-21:17607] Failing at address: 0x1a1aae000044c7
[gbwcs7-21:17607] [ 0] /lib64/libc.so.6 [0x2ba6ff829c30]
[gbwcs7-21:17607] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2ba6ff829bb5]
[gbwcs7-21:17607] [ 2] /lib64/libc.so.6 [0x2ba6ff829c30]
[gbwcs7-21:17607] [ 3] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libOpenFOAM.so(_ZNK4Foam5PBiCG5solveERNS_5FieldIfEERKS2_h+0xdf9) [0x2ba6fec1b5f9]
[gbwcs7-21:17607] [ 4] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libfiniteVolume.so(_ZN4Foam8fvMatrixIfE5solveERNS_7IstreamE+0x164) [0x2ba6fdecdd44]
[gbwcs7-21:17607] [ 5] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so(_ZN4Foam5solveIfEENS_9lduMatrix17solverPerformanceERKNS_3tmpINS_8fvMatrixIT_EEEE+0x50) [0x2ba6fd42ac80]
[gbwcs7-21:17607] [ 6] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/lib/linux64GccSPOpt/libincompressibleRASModels.so(_ZN4Foam14incompressible9RASModels12realizableKE7correctEv+0x1c29) [0x2ba6fd455849]
[gbwcs7-21:17607] [ 7] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam [0x415716]
[gbwcs7-21:17607] [ 8] /lib64/libc.so.6(__libc_start_main+0xf4) [0x2ba6ff817184]
[gbwcs7-21:17607] [ 9] /vcc/ans/cfd/of/1.5-090529-vcc.1/OpenFOAM-1.5.x/applications/bin/linux64GccSPOpt/simpleFoam(_ZN4Foam11regIOobject14readIfModifiedEv+0x1a9) [0x413be9]
[gbwcs7-21:17607] *** End of error message ***

[luca_g]

Dear Bastian,

I think you have a different problem here: iteration 191 shows that solution for k and epsilon is alerady diverging so that the successive failing is not a big surprise to me.
My experience was for sudden floating point error in a well converging solution.
It might be that the switch to single precision is influencing your solution if you are using fine near-wall mesh for k-eps. I do not have much experience on this, but I guess you should check your solution starting from a few iterations before it stops.

Regards,

Luca

[basneb]

Dear Luca,

I agree that timestep #191 is already looking strange and that it is not a big surprise that I get divergence. However, timestep #189 looks still perfectly fine. The solution diverges more or less in 2-3 timesteps. Below you can see the last few timesteps.

Best regards,

Bastian

 
Time = 187
DILUPBiCG:  Solving for Ux, Initial residual = 0.00332962, Final residual = 0.0001476, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.00133389, Final residual = 9.25218e-06, No Iterations 4
DILUPBiCG:  Solving for Uz, Initial residual = 0.00284297, Final residual = 8.63122e-05, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.062382, Final residual = 0.00186959, No Iterations 2
time step continuity errors : sum local = 0.000670455, global = -8.67323e-06, cumulative = 0.0421226
DILUPBiCG:  Solving for epsilon, Initial residual = 0.00140286, Final residual = 4.18191e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.0017069, Final residual = 3.07222e-15, No Iterations 18
ExecutionTime = 1725.62 s  ClockTime = 1753 s
Time = 188
DILUPBiCG:  Solving for Ux, Initial residual = 0.00330507, Final residual = 0.000115972, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.00130667, Final residual = 7.23427e-06, No Iterations 4
DILUPBiCG:  Solving for Uz, Initial residual = 0.00280713, Final residual = 7.8791e-05, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.062679, Final residual = 0.00186556, No Iterations 2
time step continuity errors : sum local = 0.000663852, global = -2.72414e-06, cumulative = 0.0421199
DILUPBiCG:  Solving for epsilon, Initial residual = 0.00139674, Final residual = 4.17446e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.00169037, Final residual = 9.02943e-15, No Iterations 17
ExecutionTime = 1732.3 s  ClockTime = 1759 s
Time = 189
DILUPBiCG:  Solving for Ux, Initial residual = 0.00327983, Final residual = 0.000119982, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.00127992, Final residual = 1.599e-05, No Iterations 4
DILUPBiCG:  Solving for Uz, Initial residual = 0.00277252, Final residual = 8.75667e-05, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.0622074, Final residual = 0.00188927, No Iterations 2
time step continuity errors : sum local = 0.00066771, global = 3.24493e-06, cumulative = 0.0421232
DILUPBiCG:  Solving for epsilon, Initial residual = 0.00139073, Final residual = 4.16778e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.00167427, Final residual = 1.96518e-15, No Iterations 18
ExecutionTime = 1739.06 s  ClockTime = 1766 s
Time = 190
DILUPBiCG:  Solving for Ux, Initial residual = 0.00325495, Final residual = 0.000104864, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.00125293, Final residual = 8.50088e-06, No Iterations 4
DILUPBiCG:  Solving for Uz, Initial residual = 0.00273899, Final residual = 8.8469e-05, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.0614532, Final residual = 0.00188187, No Iterations 2
time step continuity errors : sum local = 0.000662112, global = 8.30823e-06, cumulative = 0.0421315
DILUPBiCG:  Solving for epsilon, Initial residual = 0.00138482, Final residual = 4.16147e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.00165835, Final residual = 13.8498, No Iterations 1001
bounding k, min: -6.48451e+10 max: 6.10749e+10 average: 84.3805
ExecutionTime = 1840.75 s  ClockTime = 1868 s
Time = 191
DILUPBiCG:  Solving for Ux, Initial residual = 0.0183009, Final residual = 0.00139391, No Iterations 2
DILUPBiCG:  Solving for Uy, Initial residual = 0.0260962, Final residual = 0.00202694, No Iterations 2
DILUPBiCG:  Solving for Uz, Initial residual = 0.0598852, Final residual = 0.00420837, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.833212, Final residual = 0.0341513, No Iterations 2
time step continuity errors : sum local = 0.0358474, global = 4.17163e-05, cumulative = 0.0421732
DILUPBiCG:  Solving for epsilon:  solution singularity
bounding epsilon, min: 2.45278e-09 max: 1.18469e+18 average: 3.41778e+11
DILUPBiCG:  Solving for k, Initial residual = 0.419389, Final residual = 3.7141e-15, No Iterations 25
bounding k, min: -1810.15 max: 2.95828e+10 average: 311408
ExecutionTime = 1847.54 s  ClockTime = 1874 s

[luca_g]

Dear Bastian,

I think the problem is iteration 190: you see that k equation failed to solve, doing 1001 iterations (the default limit) with diverging residual.
This is something that sometimes happens when you run in parallel, even in double precision, using DILUPBiCG (I'm quite sure that it would be ok if you could run on a single cpu).
I suggest you require a small relative tolerance and either limit the max iter to a small value (5 or 10) which will prevent diverging or you better switch to a smoothSolver for turbulence fields.

Regards,

Luca

[basneb]

Dear Luca,

thx for the answer, I will try your suggestions immediately. Running on a single CPU, however, is impossible, since the mesh is rather big (13M cells).

I will keep you updated about the progress.

Best regards,

Bastian