Different results between 3 processors simulation and 7 processors simulation
 

Dear all,

I added the following line in parcel.C file. (after the last line of updateParcelProperties member function)

Info << " d() = " << d() << endl;

Then I run dieselFoam with 3 decompositions and got the following output in log file:

Evolving Spray
d() = 1.12798e-05
d() = 1.12798e-05
d() = 1.57086e-05
d() = 1.57086e-05
d() = 1.54595e-05
.....
....
Solving chemistry
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for Ux, Initial residual = 0.00170744, Final residual = 4.95922e-07, No Iterations 5
DILUPBiCG: Solving for Uy, Initial residual = 0.00181109, Final residual = 3.96033e-07, No Iterations 5
DILUPBiCG: Solving for Uz, Initial residual = 0.000120342, Final residual = 1.81973e-07, No Iterations 4
DILUPBiCG: Solving for C7H16, Initial residual = 0.820013, Final residual = 9.75537e-07, No Iterations 8
DILUPBiCG: Solving for O2, Initial residual = 0.000302092, Final residual = 3.62254e-07, No Iterations 3
DILUPBiCG: Solving for CO2, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for H2O, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for h, Initial residual = 0.000105396, Final residual = 4.97253e-07, No Iterations 3
DICPCG: Solving for p, Initial residual = 0.19397, Final residual = 8.81985e-10, No Iterations 58
DICPCG: Solving for p, Initial residual = 9.91074e-10, Final residual = 9.91074e-10, No Iterations 0
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.28234e-13, global = -2.93383e-14, cumulative = -2.93383e-14
DICPCG: Solving for p, Initial residual = 0.0156706, Final residual = 9.4495e-10, No Iterations 56
DICPCG: Solving for p, Initial residual = 9.90869e-10, Final residual = 9.90869e-10, No Iterations 0
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.2799e-13, global = 3.14686e-14, cumulative = 2.13031e-15
DILUPBiCG: Solving for epsilon, Initial residual = 0.002522, Final residual = 7.4453e-07, No Iterations 4
DILUPBiCG: Solving for k, Initial residual = 0.000132955, Final residual = 6.47361e-07, No Iterations 3

Number of parcels in system.... | 50
Injected liquid mass........... | 27.9464 mg
Liquid Mass in system.......... | 0.00560216 mg
SMD, Dmax...................... | 16.8812 mu, 53.4697 mu
Added gas mass................. | 27.9408 mg
Evaporation Continuity Error... | 4.44956e-06 mg
ExecutionTime = 1.21 s ClockTime = 1 s

However, when I run with 7 decompositions I got the following:

Evolving Spray
Solving chemistry
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for Ux, Initial residual = 0.00153395, Final residual = 4.14953e-07, No Iterations 5
DILUPBiCG: Solving for Uy, Initial residual = 0.00163918, Final residual = 3.44191e-07, No Iterations 5
DILUPBiCG: Solving for Uz, Initial residual = 9.21614e-05, Final residual = 6.42257e-07, No Iterations 3
DILUPBiCG: Solving for C7H16, Initial residual = 0.828965, Final residual = 5.45344e-07, No Iterations 10
DILUPBiCG: Solving for O2, Initial residual = 0.000294897, Final residual = 4.85681e-07, No Iterations 3
DILUPBiCG: Solving for CO2, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for H2O, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for h, Initial residual = 9.22275e-05, Final residual = 4.16487e-07, No Iterations 3
DICPCG: Solving for p, Initial residual = 0.184617, Final residual = 7.51892e-10, No Iterations 59
DICPCG: Solving for p, Initial residual = 0.0012862, Final residual = 9.94569e-10, No Iterations 47
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.33238e-13, global = -5.25939e-15, cumulative = -5.25939e-15
DICPCG: Solving for p, Initial residual = 0.010484, Final residual = 9.81269e-10, No Iterations 53
DICPCG: Solving for p, Initial residual = 0.000409979, Final residual = 9.88064e-10, No Iterations 44
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.32206e-13, global = 2.41525e-14, cumulative = 1.88931e-14
DILUPBiCG: Solving for epsilon, Initial residual = 0.00246708, Final residual = 4.9188e-07, No Iterations 4
bounding epsilon, min: -9.6188 max: 2.27937e+07 average: 31722.1
DILUPBiCG: Solving for k, Initial residual = 8.8404e-05, Final residual = 5.15251e-07, No Iterations 3

Number of parcels in system.... | 59
Injected liquid mass........... | 27.9464 mg
Liquid Mass in system.......... | 0.00657278 mg
SMD, Dmax...................... | 16.638 mu, 53.4697 mu
Added gas mass................. | 27.9408 mg
Evaporation Continuity Error... | 0.000977339 mg
ExecutionTime = 0.79 s ClockTime = 1 s

The lines with "d()" is not shown any more in 7 processors simulation.

Any idea why this happened?

Fairus

Just figured out that I supposed to use "Pout" instead of "Info" to get output from each processor. Now, for 3 processors, I got the following:

Evolving Spray
[0] d() = 1.12798e-05
[0] d() = 1.12798e-05
[0] d() = 1.57086e-05
[0] d() = 1.57086e-05
[0] d() = 1.54595e-05
[0] d() = 1.54595e-05
[0] d() = 1.8351e-05
[0] d() = 1.8351e-05
[0] d() = 1.99194e-05
.....
....
....
Solving chemistry
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for Ux, Initial residual = 0.0014768, Final residual = 1.15586e-07, No Iterations 6
DILUPBiCG: Solving for Uy, Initial residual = 0.00138362, Final residual = 6.95235e-07, No Iterations 5
DILUPBiCG: Solving for Uz, Initial residual = 5.60855e-05, Final residual = 9.75065e-07, No Iterations 3
DILUPBiCG: Solving for C7H16, Initial residual = 0.114131, Final residual = 5.25429e-07, No Iterations 7
DILUPBiCG: Solving for O2, Initial residual = 0.000130773, Final residual = 7.78804e-07, No Iterations 3
DILUPBiCG: Solving for CO2, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for H2O, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for h, Initial residual = 0.000274324, Final residual = 4.24009e-07, No Iterations 3
DICPCG: Solving for p, Initial residual = 0.0512225, Final residual = 7.91496e-10, No Iterations 56
DICPCG: Solving for p, Initial residual = 8.46768e-10, Final residual = 8.46768e-10, No Iterations 0
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.0986e-13, global = 1.1918e-14, cumulative = 1.40483e-14
DICPCG: Solving for p, Initial residual = 0.0107209, Final residual = 7.19084e-10, No Iterations 53
DICPCG: Solving for p, Initial residual = 7.75078e-10, Final residual = 7.75078e-10, No Iterations 0
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.00267e-13, global = 4.93066e-15, cumulative = 1.8979e-14
DILUPBiCG: Solving for epsilon, Initial residual = 0.00250506, Final residual = 8.27057e-07, No Iterations 4
bounding epsilon, min: -7211.69 max: 2.28754e+07 average: 31824.2
DILUPBiCG: Solving for k, Initial residual = 0.000115688, Final residual = 7.84806e-07, No Iterations 3

And I got the following with 7 processors:

Evolving Spray
[2] d() = 1.12798e-05
[2] d() = 1.12798e-05
[2] d() = 1.57086e-05
[2] d() = 1.57086e-05
[2] d() = 1.54595e-05
[2] d() = 1.54595e-05
[2] d() = 1.8351e-05
[2] d() = 1.8351e-05
[2] d() = 1.99194e-05
[2] d() = 1.99194e-05
[1] d() = 1.99194e-05
......
......
......
Solving chemistry
[2] d() = 3.20949e-05
[2] d() = 3.20949e-05
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for Ux, Initial residual = 0.00153395, Final residual = 4.14953e-07, No Iterations 5
DILUPBiCG: Solving for Uy, Initial residual = 0.00163918, Final residual = 3.44191e-07, No Iterations 5
DILUPBiCG: Solving for Uz, Initial residual = 9.21614e-05, Final residual = 6.42257e-07, No Iterations 3
DILUPBiCG: Solving for C7H16, Initial residual = 0.828965, Final residual = 5.45344e-07, No Iterations 10
DILUPBiCG: Solving for O2, Initial residual = 0.000294897, Final residual = 4.85681e-07, No Iterations 3
DILUPBiCG: Solving for CO2, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for H2O, Initial residual = 0, Final residual = 0, No Iterations 0
DILUPBiCG: Solving for h, Initial residual = 9.22275e-05, Final residual = 4.16487e-07, No Iterations 3
DICPCG: Solving for p, Initial residual = 0.184617, Final residual = 7.51892e-10, No Iterations 59
DICPCG: Solving for p, Initial residual = 0.0012862, Final residual = 9.94569e-10, No Iterations 47
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.33238e-13, global = -5.25939e-15, cumulative = -5.25939e-15
DICPCG: Solving for p, Initial residual = 0.010484, Final residual = 9.81269e-10, No Iterations 53
DICPCG: Solving for p, Initial residual = 0.000409979, Final residual = 9.88064e-10, No Iterations 44
diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 1.32206e-13, global = 2.41525e-14, cumulative = 1.88931e-14
DILUPBiCG: Solving for epsilon, Initial residual = 0.00246708, Final residual = 4.9188e-07, No Iterations 4
bounding epsilon, min: -9.6188 max: 2.27937e+07 average: 31722.1
DILUPBiCG: Solving for k, Initial residual = 8.8404e-05, Final residual = 5.15251e-07, No Iterations 3

Apparently, 3 processors decomposition employs a serial calculation for lagrangian particle tracking and 7 processors decomposition employs 2 processors for lagrangian particle tracking. That's why "Info" works for 3 processors decomposition and not for 7 processors decomposition (please correct me if I'm wrong).

Good lesson on Monday morning :p.

Fairus