[ciberbob1987]

Hi,
I'm running a simulation with porousSimpleFoam.
The simulation executed on 7 cores converges after 1295 iterations, but if I try to run the same simulation on 40 cores it diverges after 96 iterations.

Is there any mathematical reason for this?
If yes, there is a way to calculate the max number of cores that can be used?

Thanks to all.
Marco

[wyldckat]

Greetings Marco,

My experience on this is limited, but AFAIK, there are several hypothesis for the crash, since you didn't give any specifics about the case. Some examples:

1. There could be communication problems if you are using the 7 cores case in a single machine, but the 40 cores in several machines. Did you check with 7 cores, one or two per machine?
2. Analyze how the mesh was decomposed into several parts. You might find some weird situation where patches got improperly divided.
3. Certain boundary conditions have a few issues with parallel processing. Usually the guilty ones are the cyclic patches, where their division between sub-domains might lead to problems. Bug reports as examples:
   • http://www.openfoam.org/mantisbt/view.php?id=302
   • http://www.openfoam.org/mantisbt/view.php?id=387
4. Usually the rule of thumb is to have at least 50k cells per core, but this varies with core power and communication stack (number of machines vs bandwidth).

Best regards,
Bruno

[arjun]

Originally Posted by ciberbob1987

Hi,
Is there any mathematical reason for this?
If yes, there is a way to calculate the max number of cores that can be used?

numerically possible. So yes there is a mathematical reason for it.

But I am not sure if it is the case. What was your pressure correction solver.
If it was multigrid then it is possible to crash or diverge. If it was Kyrlov based solver like conjugate gradient then less likely but still depends on preconditioner used.