CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

Comments and questions regarding second workstation

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By flotus1

Reply
 
LinkBack Thread Tools Display Modes
Old   October 25, 2017, 10:41
Default Comments and questions regarding second workstation
  #1
New Member
 
Håvard B. Refvik
Join Date: Jun 2015
Location: Norway
Posts: 16
Rep Power: 5
havref is on a distinguished road
First a brief description:
We are looking to expand our computing capabilities by investing in another CFD-specific computer, mainly used for interFoam and interDyMFoam in marine applications. Today we have a dual processor rack with two 16-core Intel E5-2697A V4, which is in more or less constant use. In addition to having an increasing amount of projects lining up, simulations with complex geometries and/or dynamic meshes are often taking too long.

A typical mesh consists of 3-8 million cells, depending on the size and accuracy of the turbulence modelling. Our current memory usage for such a typical simulation with 6 million cells is in the range of 10-15 GB.

Potential workstation number 2
We don't really have a specified budget, but are aiming towards the $ 15k-30k region. A setup we've looked quite a lot at is the following:

Poweredge R940-server
4 x Intel Xeon Gold 6154 3.0GHz
8 x 16GB RDIMM 2666MT/s Dual Rank
1 x 960GB SSD SATA
Price ~ $26k


Questions:
1)
Do you have any comments regarding this setup? I'm thinking of potential bottlenecks or better computing per dollar alternatives.
2)
How much "better" will a 4 x Intel Gold 6154 perform than a dual Intel Gold 6154 setup? Can we expect the 80-90% increase in computing power that we are hoping for?
3)
I've wondered how much memory is really needed for simulations where we are rarely expecting to use more than 20 GB. Is it the memory channels we should pay most attention to? These processors have 6 memory channels. Does that mean we should use multiples of 6 when buying memory? Thus decrease to 6 or increase to 12?
4)
Is there any speed difference between 16x 8GB RAM, 8x 16GB RAM or 4 x 32? I've noticed it does not affect the cost much, so is it purely a reliability and space issue?

Thank you!
havref is offline   Reply With Quote

Old   October 25, 2017, 10:56
Default
  #2
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,848
Rep Power: 29
flotus1 will become famous soon enoughflotus1 will become famous soon enough
For this setup, you MUST use 24 DIMMs, 6 DIMMs per CPU. If memory channels are not populated, the CPU can not use their additional memory bandwidth. With only 8 or 12 DIMMs you will waste more than half of the performance this system can have for CFD.
With that fixed, the setup should be more than twice as fast compared to your old setup. But of course only if your simulations are not i/o-bound and scale well on such a large amount of cores.
If it does not scale so well, you can settle for a lower core count processor model and save a few bucks. Or if money is not that much of an issue use less cores per CPU but spread evenly among the sockets.

Edit: if you haven't done it yet, you can see how your jobs scale on your old system by running the same simulation on a single core and on an increasing amount of cores, typically doubling the core count each time. Then put the speedup factor in a chart...
When you are unsure if you are spending those 26000$ wisely, this might help you to make the right decision.

And I have to ask: are you sure that your old workstation has its memory populated correctly?
Simbelmynë likes this.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   October 26, 2017, 01:09
Default
  #3
New Member
 
Håvard B. Refvik
Join Date: Jun 2015
Location: Norway
Posts: 16
Rep Power: 5
havref is on a distinguished road
Ah, thanks! Now I understand a bit more. I've been scratching my head regarding the memory channels/memory bandwitdth question, but could not find an answer stating it clearly enough until now. Maybe it is too obvious?

I'll do a check of both speedup factor and number of DIMMs in the old system. Thanks again.
havref is offline   Reply With Quote

Old   November 17, 2017, 03:38
Default
  #4
SLC
New Member
 
Join Date: Jul 2011
Posts: 26
Rep Power: 9
SLC is on a distinguished road
Quote:
Originally Posted by flotus1 View Post
For this setup, you MUST use 24 DIMMs, 6 DIMMs per CPU. If memory channels are not populated, the CPU can not use their additional memory bandwidth. With only 8 or 12 DIMMs you will waste more than half of the performance this system can have for CFD.
With that fixed, the setup should be more than twice as fast compared to your old setup. But of course only if your simulations are not i/o-bound and scale well on such a large amount of cores.
If it does not scale so well, you can settle for a lower core count processor model and save a few bucks. Or if money is not that much of an issue use less cores per CPU but spread evenly among the sockets.

Edit: if you haven't done it yet, you can see how your jobs scale on your old system by running the same simulation on a single core and on an increasing amount of cores, typically doubling the core count each time. Then put the speedup factor in a chart...
When you are unsure if you are spending those 26000$ wisely, this might help you to make the right decision.

And I have to ask: are you sure that your old workstation has its memory populated correctly?
If you were to go with a dual-socket setup (say 2 x Intel Xeon Gold 6154 3.0GHz), would the optimum setup be with 12 DIMMS then? 6 DIMMS per CPU still?
SLC is offline   Reply With Quote

Old   November 17, 2017, 04:04
Default
  #5
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,848
Rep Power: 29
flotus1 will become famous soon enoughflotus1 will become famous soon enough
Exactly. Each memory channel needs its own DIMM (dual-rank preferably). Using two or three DIMMs per channel should usually be avoided unless it is the only way to get the memory capacity you need.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Reply

Tags
hardware, xeon

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Wall functions - questions about implementation AlmostSurelyRob OpenFOAM Programming & Development 38 August 12, 2017 16:36
workstation + master node naffrancois Hardware 6 April 5, 2017 03:10
Adding a second workstation Bazinga Hardware 4 March 7, 2017 04:31
Some questions about Laplacian with nonorthogonal correction koderer OpenFOAM 3 December 12, 2012 17:30
Questions in the BouyantBoussinesqPisoFoam waku2005 OpenFOAM 6 April 27, 2011 14:50


All times are GMT -4. The time now is 14:44.