Decomposition Methods: which ones are best for large scale simulations?
 

What decomposition method (Metis, Scotch, simple, hierarchical) you guys favor for large scaled simulations?

Metis or scotch is what we use, since we generally don't have a better idea ourselves of how to decompose the domain any better (nor the time to figure it out anew for each geometry / cpu count combination).

Simply try out metis/scotch and see if you are happy with the results. You can view the separate processor* domains in paraview (eg, via paraFoam) to see if it matches up with your expectations.

BTW: I believe that future versions of OpenFOAM might be switching to scotch instead of metis due to licensing aspects. I found that the scotch decomposition was fairly similar to metis anyhow.

Regarding licensing I think you may use Metis from 1.5 for testing purposes only. Otherwise, you need to have a license for that. Afaik this is the main reason why it changed to scotch in 1.6.
What I miss is a parallel partitioning method freeing us from the need to have machines with very large memory.

Regards

Hi.

I'm using also scotch or metis since they do not require any manual input. I checked the subdomains in paraview and they seem to be convenient. But the problem is that I get a terrible speed-up (scaling) if I increase the processor number. In other words, nearly all computational effort is invested on solving the pressure-poisson equation in PISO loop, which does not scale well with increased processor number.

Are you satisfied with parallelisation of OpenFOAM? I'm asking because considering this computational performance I can not simulate anything big. Maybe I'm doing something wrong.

Here https://projects.coin-or.org/BuildTo...Metis?rev=1294 it states
The restriction appears to apply to the vendor, not the end-user - http://glaros.dtc.umn.edu/gkhome/met...faq#distribute

HI Mark,

can you please tell me how to load the separate processor domains into ParaView?

Best regards,

Kate

Nothing special is needed Kate. From within each processor file make the
VTK conversion, then open Paraview and load as you normally would.
Does this help?

Sorry, somehow I overlooked the constant directory in the processor files.

Have a nice day,

Kate

Hi,
When it comes to parallel programming, there is a sweet spot between the no of processors you can use and your domain. Say your domain has 10000 nodes/points, then you have to find out what is the optimum no of processors you can add such that the communications are minimal. Vtune amplifier will be helpful in this analysis.
Hope this helps.

Regards,
Koushik

Sent from my MI 5 using CFD Online Forum mobile app

Hi everyone.
Just wanted to share that I was having trouble with parallel processing. I have found that aspect ratio is very crucial for partitioning interfaces.