CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

Suggestions for StarCCM+ Workstation configuration

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   April 12, 2018, 07:39
Default Suggestions for StarCCM+ Workstation configuration
  #1
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Hello everyone!,

I wish any of you could give me his/her opinion on how to choose the right workstation configuration. The machine will be mostly used for CFD computation (starCCM+) but also FEM from time to time. The simulations types will be mostly external and internal non-compressible flow (marine applications including 6DOF) and not very often though, heat transfer analysis. I do not expect to have mesh sizes larger than 40M (usually smaller) cells. I have received some configurations options and the quotations along. I am wondering if any of you could let me know which option is better or if you could suggest even a better one according to the job that the machine will be performing.

The budget for the workstation is max 12k USD

So far the options received (within the budget) are:

MAIN SPECS

Option 1)

Motherboard:Supermicro X11DAi-N - Intel C621 Chipset - Socket
LGA 3647 Workstation Motherboard
CPU / Processor:2x Intel Xeon Gold 6138 SP 2.0GHz (3.7GHz
Turbo) 125W 27.5MB Cache (40 Coress (80 Threads)
Memory:128GB (8 x 16GB) 288-Pin DDR4 SDRAM 2666
(PC4-21300) ECC Server Memory
Video Card:NVIDIA Quadro P4000 8GB GDDR5 PCIe 3.0 x16
Workstation Video Card
M.2 / U.2 NVMe Drive:WD Black 512GB M.2 NVMe PCIe 3.0 x4
Solid State Drive - read/write 2050/800 MBps
SATA Drive 1:Seagate 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s
3.5" Internal Hard Drive

Option 2)

Motherboard:Supermicro X11DAi-N - Intel C621 Chipset - Socket
LGA 3647 Workstation Motherboard
CPU / Processor:2x Intel Xeon Gold 6140 SP 2.3GHz (3.7GHz
Turbo) 125W 27.5MB Cache (36 Cores / 72 Threads)
Memory:256GB (8 x 32GB) 288-Pin DDR4 SDRAM 2666
(PC4-21300) ECC Server Memory
Video Card:NVIDIA Quadro P4000 8GB GDDR5 PCIe 3.0 x16
Workstation Video Card
M.2 / U.2 NVMe Drive:WD Black 512GB M.2 NVMe PCIe 3.0 x4
Solid State Drive - read/write 2050/800 MBps
SATA Drive 1:Seagate 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s
3.5" Internal Hard Drive

The option 2 has a slightly higher price however, still within the budget.

Has anyone experience with the above mentioned CPUs?

Thank you very much!
ifil is offline   Reply With Quote

Old   April 13, 2018, 04:07
Default
  #2
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Which retailer put together these configurations?
Skylake-SP CPUs have 6 memory channels each. By using only 8 DIMMs total, a large portion of the performance is wasted. Rendering the choice of CPU more or less irrelevant.
Both workstations should be equipped with 12 identical DIMMs (dual-rank if possible) instead of 8. And after they have been shipped I would check if they put them in the right slots.
flotus1 is offline   Reply With Quote

Old   April 13, 2018, 04:28
Default
  #3
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Hi flotus1,
thank you for your prompt reply, well, I have to admit that am not a super expert in hardware architecture. I believe they may have just made a mistake then, I can still ask to rectify.
Nevertheless if the config can be corrected, what is your opinion on the two options? Do you have a better suggestion that would still fall withing that budget?

Thank you
ifil is offline   Reply With Quote

Old   April 13, 2018, 05:15
Default
  #4
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
The most versatile choice would probably be a Xeon Gold 6146 thanks to its very high clock speeds. It is a little on the expensive side though, so settling for a 6136 instead yields a better price/performance ratio.
Here is a very useful article that also lists all-core turbo frequencies for these CPUs wich are more relevant than base clock and single-core turbo https://www.pugetsystems.com/labs/hp...rs-Guide-1077/

AMD Epyc is also an option. I have not seen any benchmarks with CCM+ directly, but results for other CFD solvers look very promising.

Quote:
I believe they may have just made a mistake then
In my opinion, this is not a simple mistake from the retailer. When I buy directly from a retailer and pay the premium they charge, I expect them to know what they are doing and not cripple performance to last-gen levels. Otherwise I could just build the system myself and save a few thousand euros.
flotus1 is offline   Reply With Quote

Old   April 13, 2018, 09:59
Default
  #5
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Thank you very much again,

I will investigate with the retailer the issue you mentioned for sure then.

Besides that, I see that also the 6148 comes out to be a good option from the article you linked (very interesting btw), but still rather expensive...

I have been going through the benchmark between intel and AMD for fluent and it shows that AMD is a very good choice too! I don't know though how much difference there might be between fluent and CCM+...this may be a tricky.

Do you happen to have an idea of what could be a good choice with AMD Epyc then?

thank you
ifil is offline   Reply With Quote

Old   April 13, 2018, 10:08
Default
  #6
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
The 16-core models offer the best price/performance ratio for CFD. More cores do not increase performance very much, they mostly use more licenses.
With your budget Epyc 7351 seems to be the only reasonable choice since it is the fastest CPU with 16 cores and only costs around 1100$ each.
Notice to myself: consider selling reasonably configured workstations as a side job

Last edited by flotus1; April 15, 2018 at 08:40.
flotus1 is offline   Reply With Quote

Old   April 18, 2018, 04:49
Default
  #7
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Thank you very much Flotus1!
I will have some dealer preparing some quotations comparing the Epyc procs. I am very curious with what they will come out with.

Thank you!
ifil is offline   Reply With Quote

Old   April 25, 2018, 11:59
Default
  #8
Senior Member
 
Lucky
Join Date: Apr 2011
Location: Orlando, FL USA
Posts: 5,112
Rep Power: 60
LuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura about
Wow, I had to verify and see it for myself. Sure enough, a Supermicro X11DAi-N really only has 2 banks of 4 DIMMS.

I would dump both setups and find a budget single node Epyc with 8 DIMMS or a better motherboard that isn't crippled. A $12k budget is respectable and you deserve much more than this.
LuckyTran is offline   Reply With Quote

Old   April 25, 2018, 12:15
Default
  #9
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Are you sure about that? The images and descriptions on SMs homepage suggest it has enough slots https://www.supermicro.com/products/...0/X11DAi-N.cfm
AFAIK, the only dual-socket boards for socket 3647 that only have 8 DIMM slots total are Supermicro X11DPL and Asus Z11PA-D8.
flotus1 is offline   Reply With Quote

Old   April 25, 2018, 12:48
Default
  #10
Senior Member
 
Lucky
Join Date: Apr 2011
Location: Orlando, FL USA
Posts: 5,112
Rep Power: 60
LuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura about
Ahh I only counted and did not notice the black bars. So it supports 2 banks of quad channel ram or 1 bank of six channel ram. Well that is a lot better in terms of specs. But I still have to call out the retailer for not offering an option that exploits the six memory channels when there's plenty of room.

So is it possible to negotiate a setup with
192GB (12 x 16GB) 288-Pin DDR4 SDRAM 2666 ??

Or you could buy either, find 4 more RAM sticks and throw them in.

The real difference between OPTION 1 and 2 is the overall RAM capacity. I generally say 4GB for every million cells if your cases are simple, more if you plan to run a lot of models. So I would decide based on planned usage. If you plan on doing a lot of big cases, go for option 2. Otherwise option 1 will save you some money. In either case, you could always add more RAM later if you find you need it.
LuckyTran is offline   Reply With Quote

Old   May 3, 2018, 00:40
Default
  #11
Member
 
Guillaume Jolly
Join Date: Dec 2015
Posts: 57
Rep Power: 9
trampoCFD is on a distinguished road
Send a message via Skype™ to trampoCFD
I can do https://trampocfd.com/collections/wo...es-workstation
with 2x AMD EPYC 7351 processors for your budget.

gui@trampo
trampoCFD is offline   Reply With Quote

Old   October 30, 2018, 04:50
Default After some time we found the budget for the workstation
  #12
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Hi Alex,

few months have passed but we finally got the budget to finally go ahead with the workstation.

We managed to find these main specs falling within our budget:

Supermicro 4U CHASSIS 8X3.5 3X5.25 SAS3 1280W (1+1) PLATINUM 7FP EATX (CSE-745BAC-R1K28B2)
Supermicro H11DSI-NT EPYC7000 DDR4 M2 VGA 2X10GBE 10XSATA RETAIL (CSE-745BAC-R1K28B2)
8 x Kingston Ram KSM26RD8/16HAI (16GB 2666MHz DDR4 ECC Reg CL19 DIMM 2Rx8 Hynix A IDT)
2 x AMD EPYC 24-CORE 7451 3.2GHZ SKT SP3 64MB CACHE 180W WOF
RADEON PRO DUO 32GB GDDR5 PCIE 3.0 16X 3X DP HDMI RETAIL
2 x Supermicro 4U Active CPU Heat Sink Socket OLGA4094 (SNK-P0064AP4)
Seagate ST2000NM0125 3.5 2TB, SATA 6Gb/s, 128MB, 512E
Samsung 1TB NVMe SSD 970 Pro Series M.2 PCI-Express [MZ-V7P1T0BW]



Any comment?

Do you think this may be a good option?

Thank you in Advance!
ifil is offline   Reply With Quote

Old   October 30, 2018, 05:49
Default
  #13
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Yes, I have a few comments.

Quote:
8 x Kingston Ram KSM26RD8/16HAI (16GB 2666MHz DDR4 ECC Reg CL19 DIMM 2Rx8 Hynix A IDT)
With 2 AMD Epyc CPUs, you need 16 DIMMs to populate all memory channels. So either double the amount of memory to 16x16GB or step down to 16x8GB.

Quote:
RADEON PRO DUO 32GB GDDR5 PCIE 3.0 16X 3X DP HDMI RETAIL
Are you 100% certain that any of your software benefits from such an exotic dual GPU? And that it plays nicely with the rest of the software you use?
Otherwise I would play it safe and chose a midrange Quadro card, e.g. P4000. Or at least a single-GPU from AMD.

Quote:
Seagate ST2000NM0125 3.5 2TB, SATA 6Gb/s, 128MB, 512E
No need for more local storage? A single 2TB HDD seems rather small.

Be warned that these machines can be awfully loud. The supermicro chassis uses small fans with up to 9400rpm, the power supplies have an even smaller fan with even higher rpm and their CPU coolers are not on the quiet side either. Not an issue if it is sitting in a separate room, but you probably would not want this sitting under your desk if you are sensitive to noise while working
flotus1 is offline   Reply With Quote

Old   October 30, 2018, 06:04
Default DIMMs population issue
  #14
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Hi Alex,

thank you for your prompt reply! Well regarding the Graphic card: we are planning to use the workstation also for some graphic rendering every now and then so this is the reason why we decided to go for such a exotic one.
Regarding the memory I will contact the dealer and see what he can do.

Do you think that the DIMMs population may heavily affect the computing performance? I am sorry for asking this question but I am not an expert on this specific matter.

Thank you again.

Cheers
ifil is offline   Reply With Quote

Old   October 30, 2018, 06:05
Default DIMMs population issue
  #15
New Member
 
Phil
Join Date: Aug 2013
Posts: 8
Rep Power: 11
ifil is on a distinguished road
Sorry I almost forgot,

regarding the memory, regarding the storage we will use an external server to store the executed jobs. So this is the reason why we didn't go for a very large one.

Thanks
ifil is offline   Reply With Quote

Old   October 30, 2018, 06:09
Default
  #16
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,210
Rep Power: 44
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by ifil View Post
Do you think that the DIMMs population may heavily affect the computing performance?
Using all 16 channels vs. only 8 will effectively double the performance of the machine for bandwidth-limited applications like CFD or FEM. With 48 cores total you can easily max out the memory bandwidth of all 16 channels.
flotus1 is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
workstation configuration pippo2013 Hardware 24 November 21, 2017 04:55
Low budget Workstation . Second Hand suggestions well received. Fer_Arus Hardware 4 April 10, 2017 12:20
Xeon workstation: suggestions needed Blanco Hardware 23 September 2, 2016 17:29
workstation configuration harsha_kulkarni Hardware 0 December 30, 2014 05:46
Yet another workstation configuration question RSE Hardware 6 October 17, 2012 18:15


All times are GMT -4. The time now is 15:32.