CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > ANSYS > CFX

Number of CPU cores used does not increase speed!

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes
  • 2 Post By ghorrocks

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   September 24, 2021, 08:47
Default Number of CPU cores used does not increase speed!
  #1
New Member
 
adad
Join Date: Sep 2021
Posts: 3
Rep Power: 2
Styralf is on a distinguished road
Hello,

We bought an new computer for CFX calculations. It has an AMD Threadripper 32 core processor and 256Gb 3600Mhz DDR4 Ram. When I test the speed performance of each simulation, it is the same using 32 partitions or 64, even 20 partitions runs at the same speed. I have tried to disable SMT mode and this did not help either.

I also tested to run two simulations on the same time, each using 32 cores each. This resulted that each simulation would take 2x longer time.

Anyone know what the problem might be?
Styralf is offline   Reply With Quote

Old   September 24, 2021, 11:17
Default
  #2
Senior Member
 
Join Date: Jun 2009
Posts: 1,452
Rep Power: 27
Opaque will become famous soon enough
Check with ANSYS CFX support and they should be able to stir you in the right direction.

Warning: do not expect linear scalability (for any software in general) to the full number of cores on these processors. I do not recall if the Threadripper has 64 real CPU cores, 32 real ones plus 32 virtual ones. In addition, if the memory bandwidth is not there, any software will waste time moving data between memory banks. It is not about the total memory available, but how it is distributed and allocated.
__________________
Note: I do not answer CFD questions by PM. CFD questions should be posted on the forum.
Opaque is offline   Reply With Quote

Old   September 24, 2021, 14:37
Default
  #3
New Member
 
adad
Join Date: Sep 2021
Posts: 3
Rep Power: 2
Styralf is on a distinguished road
Quote:
Originally Posted by Opaque View Post
Check with ANSYS CFX support and they should be able to stir you in the right direction.

Warning: do not expect linear scalability (for any software in general) to the full number of cores on these processors. I do not recall if the Threadripper has 64 real CPU cores, 32 real ones plus 32 virtual ones. In addition, if the memory bandwidth is not there, any software will waste time moving data between memory banks. It is not about the total memory available, but how it is distributed and allocated.
Thanks but we dont pay for the support any more so thats why Im here =)

Threadripper has 32 cores, and 64 virtual. I turned of the virtual ones but this did not help.
Styralf is offline   Reply With Quote

Old   September 24, 2021, 21:17
Default
  #4
Super Moderator
 
Glenn Horrocks
Join Date: Mar 2009
Location: Sydney, Australia
Posts: 16,835
Rep Power: 132
ghorrocks is just really niceghorrocks is just really niceghorrocks is just really niceghorrocks is just really nice
If you do a speed-up test from 1x, 2x, 4x, 8x, 16x, 32x you will probably find that it speeds up nicely to 16x, but 32x does not give you much extra. This is pretty normal for running lots of cores in a single workstation. You will get better speedups at 16x and 32x by running across multiple workstations - but note you will need a high-end network interconnect to get good speed up here too. Ethernet is not good enough, 10G ethernet not much better; you will need infiniband or myranet or one of the those specialist interconnects.

In short - getting good speed up beyond about 16x requires careful planning and testing. You cannot just buy off the shelf stuff.
Opaque and aero_head like this.
__________________
Note: I do not answer CFD questions by PM. CFD questions should be posted on the forum.
ghorrocks is offline   Reply With Quote

Old   September 25, 2021, 11:41
Default
  #5
New Member
 
adad
Join Date: Sep 2021
Posts: 3
Rep Power: 2
Styralf is on a distinguished road
Quote:
Originally Posted by ghorrocks View Post
If you do a speed-up test from 1x, 2x, 4x, 8x, 16x, 32x you will probably find that it speeds up nicely to 16x, but 32x does not give you much extra. This is pretty normal for running lots of cores in a single workstation. You will get better speedups at 16x and 32x by running across multiple workstations - but note you will need a high-end network interconnect to get good speed up here too. Ethernet is not good enough, 10G ethernet not much better; you will need infiniband or myranet or one of the those specialist interconnects.

In short - getting good speed up beyond about 16x requires careful planning and testing. You cannot just buy off the shelf stuff.
Thank you, I will test this ASAP, I also think that my memory latency speed is to slow and I think it has to do with the Uncore Frequency. It has happend alot since I build my own computer 20 years ago so I am a little noob still.
Styralf is offline   Reply With Quote

Old   September 27, 2021, 12:03
Default
  #6
Senior Member
 
Join Date: Jun 2009
Posts: 1,452
Rep Power: 27
Opaque will become famous soon enough
There is a reason why supercomputers (or high-end cluster) providers exist, it is not trivial (nor cheap) to build a high-performance computer machine.
__________________
Note: I do not answer CFD questions by PM. CFD questions should be posted on the forum.
Opaque is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[snappyHexMesh] snappyHexMesh stuck when snap is turned on yukuns OpenFOAM Meshing & Mesh Conversion 3 February 2, 2021 13:05
[mesh manipulation] Mesh Refinement Luiz Eduardo Bittencourt Sampaio (Sampaio) OpenFOAM Meshing & Mesh Conversion 42 January 8, 2017 12:55
decomposePar -allRegions stru OpenFOAM Pre-Processing 2 August 25, 2015 03:58
simpleFoam parallel AndrewMortimer OpenFOAM Running, Solving & CFD 12 August 7, 2015 18:45
SigFpe when running ANY application in parallel Pj. OpenFOAM Running, Solving & CFD 3 April 23, 2015 14:53


All times are GMT -4. The time now is 08:54.