[Venky_94]

Hello,

This is my first time running openFoam on a cluster and I noticed that my simulation was running much slower on the cluster using multiple processors than on my local PC.

I did a scaling analysis by noting down the execution time for around 50 iterations and averaged them (for different processor counts). I noticed that the performance sharply deteriorates after just increasing the processor count to 4. I've attached a photo of the timings and efficiency.

For comparison, the execution time is less than a second on my local PC while using 8 cores. A single node on the cluster has 40 processors, and I'd like to make use of the available computing power to speed up my simulation. I'm using scotch method of decomposition and a custom solver based on interFoam. I'm also attaching the case setup. Please let me know what I could change to improve the performance.

[Santiago]

Originally Posted by Venky_94

Hello,

This is my first time running openFoam on a cluster and I noticed that my simulation was running much slower on the cluster using multiple processors than on my local PC.

I did a scaling analysis by noting down the execution time for around 50 iterations and averaged them (for different processor counts). I noticed that the performance sharply deteriorates after just increasing the processor count to 4. I've attached a photo of the timings and efficiency.

For comparison, the execution time is less than a second on my local PC while using 8 cores. A single node on the cluster has 40 processors, and I'd like to make use of the available computing power to speed up my simulation. I'm using scotch method of decomposition and a custom solver based on interFoam. I'm also attaching the case setup. Please let me know what I could change to improve the performance.

Pretty common when running unstructured codes on single nodes. This is basically due to the number of channels and the L2/L1 cache of your blade. My suggestion: run the same performance analysis, but using the node as the smallest "cpu unit". That is: use 40, 80, 120 processors.

[Venky_94]

Originally Posted by Santiago

Pretty common when running unstructured codes on single nodes. This is basically due to the number of channels and the L2/L1 cache of your blade. My suggestion: run the same performance analysis, but using the node as the smallest "cpu unit". That is: use 40, 80, 120 processors.

Thanks for your suggestion. I tried it out and noticed that while all 3 options are faster than my local PC, I had the best results while using a single node. I apologize for the delayed response. I had quite a large wait time for the jobs to run.

I ran all three options for 24 hours and the completed runtimes for the case were 2.5s on single node (40 processors), 2s on 80 processors and 1.9s on 120 processors.

While the performance of single node is good enough for my case currently, I'm genuinely curious about the results.

1. Why is it that the performance was so poor for 20 cores but much improved for 40 cores? Is it because the complete system is available at its disposal enabling openfoam to have access to more memory?
2. Also why is 40 cores giving a better performance than higher core count? Is it because of higher communication times between the cores (not enough cells/cores situation)
3. And finally what would I have to do if I need to further quicken my simulation?