[derekm]

This is the reverse of the usual question, i have the configuration already determined, the question how many cells can it cope with, aiming for a 24hr run to simulate 1 to 3 hrs. The problem space is the heat flow in insect colonies using CHTmultiregionfoam or CHTMultiregionSimpleFoam.

The config is 4 servers with dual opteron 8381 (shanghai quad core 2.5GHZ) with 2 servers with quad 8381 opteron, total 64 cores and total 64Gb memory (800mhz). Gigabit interconnect.

[wyldckat]

Greetings Derek,

Well, the proportions of "mesh size to RAM" may vary depending on the simulation software and the cell types in question. Because sometimes it's the vertex count that matters, others it's the face count and others is the cell count. Then it depends on the number of equations to be solved, if there are special mesh interfaces, how many subdomains there are (usually 1 subdomain per processor) and so on.

In your case, it's OpenFOAM and my guess is that the mesh is made mostly of hexahedral cells... although an ant colony isn't exactly easily meshed with a purely hexahedral mesh...
The rule of thumb I found a few years ago was that for the "cavity" tutorial, using blockMesh and icoFoam, the proportion seemed to be 1 GB of RAM for each 1 million cells. But as mentioned above, it's a purely hex mesh and a rather simple solver.

Therefore, in your case, my suggestion is that you should do a quick study of mesh proportion to occupied RAM, using the 1 GB/Mcell as reference for building 2 cases, e.g.: one with 16 million cells and another with 32 million cells. Then launch each case for a couple of iterations, to see how much total RAM it uses in each one.

Best regards,
Bruno

[derekm]

looks like 1.5e9 bytes per 1e6 cells for the chtmultipleregionfoam.

the next thing is to fathom a reasonable value per cpu

[derekm]

Looking at the posts it seems 50k cells is a minimum number of cells per core and 2 core per numa is probably optimal. this would give 50k x 16 x 2
or 1.6E6 so that means the range would be 1.6E6 to 40E6 cells. with the processors being 2009 era i'm thinking closer to the 1.6E6 boundary.