CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (http://www.cfd-online.com/Forums/openfoam-solving/)
-   -   porousSimpleFoam crash on 40 cores (http://www.cfd-online.com/Forums/openfoam-solving/97827-poroussimplefoam-crash-40-cores.html)

ciberbob1987 February 26, 2012 15:56

porousSimpleFoam crash on 40 cores
 
Hi,
I'm running a simulation with porousSimpleFoam.
The simulation executed on 7 cores converges after 1295 iterations, but if I try to run the same simulation on 40 cores it diverges after 96 iterations.

Is there any mathematical reason for this?
If yes, there is a way to calculate the max number of cores that can be used?

Thanks to all.
Marco

wyldckat February 26, 2012 17:09

Greetings Marco,

My experience on this is limited, but AFAIK, there are several hypothesis for the crash, since you didn't give any specifics about the case. Some examples:
  1. There could be communication problems if you are using the 7 cores case in a single machine, but the 40 cores in several machines. Did you check with 7 cores, one or two per machine?
  2. Analyze how the mesh was decomposed into several parts. You might find some weird situation where patches got improperly divided.
  3. Certain boundary conditions have a few issues with parallel processing. Usually the guilty ones are the cyclic patches, where their division between sub-domains might lead to problems. Bug reports as examples:
  4. Usually the rule of thumb is to have at least 50k cells per core, but this varies with core power and communication stack (number of machines vs bandwidth).
Best regards,
Bruno

arjun February 26, 2012 18:22

Quote:

Originally Posted by ciberbob1987 (Post 346360)
Hi,
Is there any mathematical reason for this?
If yes, there is a way to calculate the max number of cores that can be used?

numerically possible. So yes there is a mathematical reason for it.

But I am not sure if it is the case. What was your pressure correction solver.
If it was multigrid then it is possible to crash or diverge. If it was Kyrlov based solver like conjugate gradient then less likely but still depends on preconditioner used.


All times are GMT -4. The time now is 11:11.