CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

Small cluster for ANSYS Fluent

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   October 16, 2017, 13:24
Question Small cluster for ANSYS Fluent
  #1
New Member
 
flyever
Join Date: Mar 2012
Posts: 5
Rep Power: 8
lasersim is on a distinguished road
Dear CFD users,

I work in a company and we are trying to scale up our CFD capabilities from a server to a small cluster. Our major CFD tool is ANSYS Fluent. We have 3 HPC pack licenses and that is good for running Fluent on 128 cores. I have got 2 quotes from different cluster suppliers based on the same budget. I am not familiar with cluster stuff and so I am here to seek suggestions/comments. I want to know which quote is better and how to optimize them. Here I list the details of these 2 clusters:

Cluster 1:
8 compute nodes @ 128 cores
Processor: Intel XEON e5-2667 v4 CPU (3.2GHz/3.6GHz HT) (16 X 8c)
Chassis: 4 x 1U x 2 Node chassis (1000 Watt power supply)
Memory: 8 x 128 GB DDR4-2400MHz LRDIMM (8 X 16 GB)
OS Data: 8 x 2 x 1TB SATA 7200 RPM 6Gbp/s
Interconnect: 8 x 56 Gbps Infiniband

Interconnect Switch: 12 port Mellanox FDR switch
Operating System: CENTOS 7

Cluster 2:

8 compute nodes @ 192 cores
Processor: Intel Scalable Xeon SKL-SP GOLD 5118 12C/24T 2.3G 16.5M 10.4GT UPI
Head Node: 8 cores (2 x SILVER 4112), 96GB Memory (6 x 16GB) DDR4 ECC RDIMM, 32TB Storage (8 x 4TB) RAW-RAID 10/5/6 all available 9SAS3 Discrete RAID)
Each Compute Node: 24 cores ( 2x GOLD 5118), 192 GB Memory (12x 16GB) DDR4 ECC RDIMM 2666MHz

Interconnect: 36 port EDR InfiniBand Switch from Mellanox
24 Port Management Switch
Operating System: Read Hat Enterprise Linux

Any suggestions or comments are welcome! Thank you in advance!
lasersim is offline   Reply With Quote

Old   October 19, 2017, 16:12
Default
  #2
Senior Member
 
Erik
Join Date: Feb 2011
Location: Earth (Land portion)
Posts: 792
Rep Power: 14
evcelica is on a distinguished road
CFD is usually all about memory bandwidth.

The Gold CPUs have 6 memory channels, where the v4 only have 4. So cluster 2 will have 150% of the memory bandwidth, and should be faster. Looks like that CPU only supports 2400 MHz Ram, not 2666 as you show.
The XEON Gold 6126 is higher CPU clock and supports the 2666MHz memory, with a modest price increase ($450 per CPU)
Here is a list of the new INTEL CPus: https://ark.intel.com/products/serie...ble-Processors

People have been looking at the AMD EPYC CPUs recently as well, and there have been some discussions on this forum.
evcelica is offline   Reply With Quote

Old   October 19, 2017, 16:33
Default
  #3
New Member
 
flyever
Join Date: Mar 2012
Posts: 5
Rep Power: 8
lasersim is on a distinguished road
Thank you, Erik! I will discuss with the vendor about XEON Gold 6126.
lasersim is offline   Reply With Quote

Old   October 20, 2017, 17:03
Default
  #4
Senior Member
 
Onno
Join Date: Jan 2012
Location: Germany
Posts: 120
Rep Power: 9
Kaskade is on a distinguished road
Some notes since I am also looking into getting hardware for Fluent and 3 HPC-Packs:

- FDR will not be significantly slower than EDR, but much cheaper
- Omnipath offers the bandwidth of EDR at the price of FDR
- go with Skylake-SP because of the number of memory lanes
- I am going with 16x Xeon Gold 6130. They run at high frequencies when only 8 of the available 16 cores are in use. But if I rent a fourth HPC-Pack I can unlock 50 to 75 % extra performance by running on all available 256 cores.
- one 4-socket system should be slightly cheaper than two 2-socket system. Plus it is easier to handle.
Kaskade is offline   Reply With Quote

Old   October 21, 2017, 01:00
Default
  #5
New Member
 
flyever
Join Date: Mar 2012
Posts: 5
Rep Power: 8
lasersim is on a distinguished road
Thank you, Kaskade!
lasersim is offline   Reply With Quote

Old   December 29, 2017, 04:23
Default
  #6
New Member
 
MN
Join Date: Dec 2017
Posts: 1
Rep Power: 0
bela541 is on a distinguished road
Thank you, Erik!
bela541 is offline   Reply With Quote

Reply

Tags
ansys fluent, cluster

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Fluent Computer cluster issues foamer FLUENT 0 November 11, 2016 06:03
Running UDF with Supercomputer roi247 FLUENT 4 October 15, 2015 14:41
Fluent 14.0 file not running in parallel mode in cluster tejakalva FLUENT 0 February 4, 2015 08:02
Fluent Cluster Computing np1409 FLUENT 0 December 9, 2014 03:09
[Solution] Fluent hangs on exit in a 2 nodes cluster ghost82 FLUENT 0 November 9, 2014 10:11


All times are GMT -4. The time now is 10:21.