[giovanni.medici]

Originally Posted by andy_

andy

2x AMD Epyc 9555 (2x 64 cores ) 12x 64GB 6400MT/s, 4 SSD RAID5

openFOAM.org (v12)

I ran the motorBikeSteady of the tutorial and only modified the decomposeParDict.
Managed the runs through the script suggested by andy_ , without changing the iterations (which are set to 500) nor the system/blockMeshDict.

# cores   Wall time (s):
------------------------
1 267.619
2 162.446
4 73.0929
8 38.0247
16 20.6077
32 12.69
64 9.62739
96 11.0211
128 9.36561

I wonder : is the lack of improvement with 96 and 128 cores is due to the relatively small mesh of the tutorial or to other aspects?

Is the motorbikeSteady approach the good to go also for openfoam.org ? Thanks

[andy_]

Originally Posted by giovanni.medici

I wonder : is the lack of improvement with 96 and 128 cores is due to the relatively small mesh of the tutorial or to other aspects?

The benchmark is a bit small to be representative of CFD runs on current processors with 100+ cores. Increasing the mesh size will reduce the ratio of communications to number crunching and lead to better parallel efficiencies.

If you only have 12 memory chips rather than 24 that will lead to parallel efficiency dropping earlier with number of cores.

[giovanni.medici]

Originally Posted by giovanni.medici

2x AMD Epyc 9555 (2x 64 cores ) 12x 64GB 6400MT/s, 4 SSD RAID5

openFOAM.org (v12)

I ran the motorBikeSteady of the tutorial and only modified the decomposeParDict.

[...]
Thanks

Following _andy's suggestion, I've increased a bit the test case: 500 iterations instead of 100 and bigger blockmeshdict.

Moreover tested the effect of 12x 64GB and 24x 64GB.

2x AMD Epyc 9555 (2x 64 cores ) 4 SSD RAID5

24x 64GB (bigger case)

# cores   Wall time (s):
------------------------
1 2079.66
2 1098.44
4 456.518
8 216.782
16 108.372
32 64.466
64 42.7124
96 38.4144
128 32.9732

12x 64GB (bigger case)

# cores   Wall time (s):
------------------------
1 2099.09
2 1167.66
4 451.095
8 216.72
16 109.641
32 69.4607
64 49.9208
96 46.9687
128 41.3233

There is clearly an effect, nevertheless I was still expecting a bit more, possibly suboptimal BIOS settings (I checked and SMT is on) ?

[andy_]

Originally Posted by giovanni.medici

there is clearly an effect, nevertheless i was still expecting a bit more, possibly suboptimal bios settings (i checked and smt is on) ?

If we compare the parallel efficiencies (normalised by the average of the two single processor results) they appear to be what one might expect. The cache improvement is essentially the same until the number of cores exceeds the number of memory channels. Is the difference on 1 and 2 cores due to the benchmark not fitting in a single memory stick? Using half the memory channels is leading to a 20% reduction in speed when running a job on the whole machine.

  1  1.005  0.995
  2  0.951  0.895
  4  1.144  1.158
  8  1.205  1.205
 16  1.205  1.191
 32  1.013  0.940
 64  0.764  0.654
128  0.495  0.395

[superkelle]

I ran the benchmark on a mid tier laptop "LENOVO IDEAPAD PRO 5 14", with an R7 8845HS 32 GB LPDDR5x Ram @6400MT/s. The results are better than the 5 years passed R7 2700X Desktop runs that I made and that in such a small form factor without any tweaking and with so little cooling. Advances in computer technology are so fast in these days ... and I am getting old


# cores Wall time (s):
------------------------
1 2 4 6 8
Meshing Times:
1 461.80
2 330.67
4 202.80
6 158.15
8 137.43

Flow Calculation:
1 602.91
2 302.44
4 185.01
6 159.69
8 152.37

Originally Posted by superkelle

Update with optimised memory timings:

Ryzen 2700X eight core CPU @ stock, 32Gb RAM (4 x 8Gb in dual channel) @ 3200MHz 16-18-20-36 (+ optimises subsettings by DRAM Calculator for Ryzen v1.7.0 by 1usmus), Ubuntu 19.10, OF1912

# cores   Wall time (s):
------------------------
1 699.57
2 414.2
4 257.01
6 227.03
8 221.71
********************
12 216.02
16 223.53

More than 6 threads seem to be not beneficial. Additionaly like assumed, 12 and 16 threaded are not really faster, but for sake of interest I included the results.