[Styralf]

Hello,

We bought an new computer for CFX calculations. It has an AMD Threadripper 32 core processor and 256Gb 3600Mhz DDR4 Ram. When I test the speed performance of each simulation, it is the same using 32 partitions or 64, even 20 partitions runs at the same speed. I have tried to disable SMT mode and this did not help either.

I also tested to run two simulations on the same time, each using 32 cores each. This resulted that each simulation would take 2x longer time.

Anyone know what the problem might be?

[Opaque]

Check with ANSYS CFX support and they should be able to stir you in the right direction.

Warning: do not expect linear scalability (for any software in general) to the full number of cores on these processors. I do not recall if the Threadripper has 64 real CPU cores, 32 real ones plus 32 virtual ones. In addition, if the memory bandwidth is not there, any software will waste time moving data between memory banks. It is not about the total memory available, but how it is distributed and allocated.

[Styralf]

Originally Posted by Opaque

Check with ANSYS CFX support and they should be able to stir you in the right direction.

Warning: do not expect linear scalability (for any software in general) to the full number of cores on these processors. I do not recall if the Threadripper has 64 real CPU cores, 32 real ones plus 32 virtual ones. In addition, if the memory bandwidth is not there, any software will waste time moving data between memory banks. It is not about the total memory available, but how it is distributed and allocated.

Thanks but we dont pay for the support any more so thats why Im here =)

Threadripper has 32 cores, and 64 virtual. I turned of the virtual ones but this did not help.

[ghorrocks]

If you do a speed-up test from 1x, 2x, 4x, 8x, 16x, 32x you will probably find that it speeds up nicely to 16x, but 32x does not give you much extra. This is pretty normal for running lots of cores in a single workstation. You will get better speedups at 16x and 32x by running across multiple workstations - but note you will need a high-end network interconnect to get good speed up here too. Ethernet is not good enough, 10G ethernet not much better; you will need infiniband or myranet or one of the those specialist interconnects.

In short - getting good speed up beyond about 16x requires careful planning and testing. You cannot just buy off the shelf stuff.

[Styralf]

Originally Posted by ghorrocks

If you do a speed-up test from 1x, 2x, 4x, 8x, 16x, 32x you will probably find that it speeds up nicely to 16x, but 32x does not give you much extra. This is pretty normal for running lots of cores in a single workstation. You will get better speedups at 16x and 32x by running across multiple workstations - but note you will need a high-end network interconnect to get good speed up here too. Ethernet is not good enough, 10G ethernet not much better; you will need infiniband or myranet or one of the those specialist interconnects.

In short - getting good speed up beyond about 16x requires careful planning and testing. You cannot just buy off the shelf stuff.

Thank you, I will test this ASAP, I also think that my memory latency speed is to slow and I think it has to do with the Uncore Frequency. It has happend alot since I build my own computer 20 years ago so I am a little noob still.

[Opaque]

There is a reason why supercomputers (or high-end cluster) providers exist, it is not trivial (nor cheap) to build a high-performance computer machine.