[christian]

I'm running LES on internal flow passing an orifice plate. There are 3 million hexahedral cells. AMG is used with the following settings:

p GAMG
{
p GAMG
{
tolerance 1e-04;
relTol 0;
smoother GaussSeidel;
nCellsInCoarsestLevel 30;
mergeLevels 1;
agglomerator faceAreaPair;
cacheAgglomeration on;
// nPreSweeps 0;
// nPostSweeps 2;
// nFinestSweeps 2;
// scaleCorrection yes;
// directSolveCoarsest no;
};
pFinal GAMG
{
tolerance 1e-04;
relTol 0;
smoother GaussSeidel;
nCellsInCoarsestLevel 30;
mergeLevels 1;
agglomerator faceAreaPair;
cacheAgglomeration on;
// nPreSweeps 0;
// nPostSweeps 2;
// nFinestSweeps 2;
// scaleCorrection yes;
// directSolveCoarsest no;

I'm running on a dual core machine. I've compared running on 1 CPU and running on 2 CPU:s, see below:

-------------- 1 CPU -----------------------

Time = 0.0127

Courant Number mean: 0.029355 max: 1.3072
DILUPBiCG: Solving for Ux, Initial residual = 0.00181139, Final residual = 3.63695e-06, No Iterations 2
DILUPBiCG: Solving for Uy, Initial residual = 0.0207766, Final residual = 3.83961e-06, No Iterations 3
DILUPBiCG: Solving for Uz, Initial residual = 0.0209138, Final residual = 3.63663e-06, No Iterations 3
GAMG: Solving for p, Initial residual = 0.247526, Final residual = 9.88771e-05, No Iterations 62
time step continuity errors : sum local = 2.27516e-08, global = -5.17908e-09, cumulative = 1.71557e-07
GAMG: Solving for p, Initial residual = 0.0506679, Final residual = 9.04641e-05, No Iterations 12
time step continuity errors : sum local = 2.07179e-08, global = -3.83544e-09, cumulative = 1.67722e-07
ExecutionTime = 37679.3 s ClockTime = 45218 s

Time = 0.01275

Courant Number mean: 0.0293701 max: 1.3049
DILUPBiCG: Solving for Ux, Initial residual = 0.00182424, Final residual = 3.75133e-06, No Iterations 2
DILUPBiCG: Solving for Uy, Initial residual = 0.0208073, Final residual = 2.84017e-07, No Iterations 3
DILUPBiCG: Solving for Uz, Initial residual = 0.0209474, Final residual = 3.01873e-07, No Iterations 3
GAMG: Solving for p, Initial residual = 0.249727, Final residual = 9.40309e-05, No Iterations 61
time step continuity errors : sum local = 2.15569e-08, global = -4.70248e-09, cumulative = 1.63019e-07
GAMG: Solving for p, Initial residual = 0.0506867, Final residual = 9.95558e-05, No Iterations 12
time step continuity errors : sum local = 2.30218e-08, global = -4.36871e-09, cumulative = 1.58651e-07
ExecutionTime = 37848.4 s ClockTime = 45388 s

-------------------------------------------

------------- 2 CPU -----------------------

Time = 0.0142

Courant Number mean: 0.0298694 max: 1.25606
DILUPBiCG: Solving for Ux, Initial residual = 0.00415434, Final residual = 3.0531e-06, No Iterations 2
DILUPBiCG: Solving for Uy, Initial residual = 0.026599, Final residual = 1.81261e-07, No Iterations 3
DILUPBiCG: Solving for Uz, Initial residual = 0.0327937, Final residual = 9.15543e-06, No Iterations 2
GAMG: Solving for p, Initial residual = 0.27006, Final residual = 9.92377e-05, No Iterations 178
time step continuity errors : sum local = 3.20367e-08, global = -3.3541e-09, cumulative = -1.08914e-08
GAMG: Solving for p, Initial residual = 0.0598454, Final residual = 9.88563e-05, No Iterations 37
time step continuity errors : sum local = 3.10378e-08, global = -3.38161e-09, cumulative = -1.4273e-08
ExecutionTime = 2829.56 s ClockTime = 2969 s

Time = 0.01425

Courant Number mean: 0.0298782 max: 1.21613
DILUPBiCG: Solving for Ux, Initial residual = 0.00416074, Final residual = 1.55868e-06, No Iterations 2
DILUPBiCG: Solving for Uy, Initial residual = 0.0264671, Final residual = 3.28098e-07, No Iterations 3
DILUPBiCG: Solving for Uz, Initial residual = 0.0325682, Final residual = 3.72951e-07, No Iterations 3
GAMG: Solving for p, Initial residual = 0.277771, Final residual = 9.83602e-05, No Iterations 178
time step continuity errors : sum local = 3.08924e-08, global = 3.23832e-09, cumulative = -1.10347e-08
GAMG: Solving for p, Initial residual = 0.0579332, Final residual = 9.59878e-05, No Iterations 37
time step continuity errors : sum local = 3.10846e-08, global = 3.44964e-09, cumulative = -7.58509e-09
ExecutionTime = 3115 s ClockTime = 3269 s

-------------------------------------------

1) How come the AMG solver needs to do more iterations when running parallel? This is not what I've seen before on this machine and a similar case. Then there was a little speed-up when running on 2 CPUs. In this case, I'm losing time running parallel.

2) What are recommended settings for the AMG solver? I'm especially interested in the number of cells in the coarsest level when running serial and parallel, respectively.

3) In general, isn't it possible to get a scaling of 2 when running on two CPUs with AMG? What kind of case will give me a scaling of two?

4) I'm using Crank-Nicholson for time discretisation. From your experience, what is the gain in accuracy (compared to all sources of errors in the computation) and increase in computational time compared to Backward?

5) What would you say to be a reasonable convergence criteria (for p)? I don't want a situation where p is converged to much to drown in all other errors.

6) For convection, both midpoint and linear discretisation give me the velocity field in the plot below (after a few time steps). I assume what is seen are wiggles??? I used vanLeer for the RANS solution to start from. Do you believe vanLeer to be to diffusive for LES?



Best regards,
Christian