CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Hardware

EPYC based workstation for OpenFOAM simulations

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes
  • 1 Post By flotus1
  • 1 Post By flotus1

Reply
 
LinkBack Thread Tools Display Modes
Old   October 25, 2017, 08:51
Default EPYC based workstation for OpenFOAM simulations
  #1
Member
 
Johan Roenby
Join Date: May 2011
Location: Denmark
Posts: 45
Rep Power: 14
roenby will become famous soon enough
Dear hardware experts

I am about to buy a workstation for OpenFOAM simulations and have tried to learn from the threads in this forum. It is all quite confusing for a non-hardware expert like me, but as far as I have understood, memory band width is the main bottleneck with CFD.

The type of simulations I will do is transient interFoam (interfacial flows, with a VOF field, a pressure field and a velocity field to be solved for) and I will probably rarely go over 2-5M cells (if I need more, my plan is to look for a rented cluster).

I am considering buying a workstation with the new EPYC 7301 processor which gives me 16 cores, 8 memory channels (DDR4-2666), 64 MB L3 cache [1] at a reasonable price and with possibility to double all this at a later time.

I have found a similar configuration at a price of around 3.650 USD [2].

Here are the specs:

System: Supermicro 4023S-TRT Tower
CPU: 1 x AMD EPYC 7301 Processor
Memory: 8 x 4GB DDR4 2666MHz RAM
HDD 1: 256 SSD disk (unspecified)
HDD 2: 1T HDD (unspecified)
Power: 1280W Redundant High-efficiency Digital Power Supplies w/ PMBus 1.2

Now, can any of you give me an educated guess on how risky this configuration is, given that the EPYC processors are brand new and there are still no proper OpenFOAM CFD benchmarks available yet.

Could some architectural detail make the apparently very promising EPYC CPU specs disappoint with regards to CFD?

Would it be safer to make a similarly priced configuration with one of the new Intel SP processors?

Any advice or guidance will be much appreciated.

Kind regards,
Johan

[1] https://en.wikipedia.org/wiki/Epyc
[2] https://www.nextron.no/en/main.php3?...info&CID=40382
roenby is offline   Reply With Quote

Old   October 25, 2017, 10:38
Default
  #2
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,621
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
I am currently waiting until Epyc CPUs and motherbards are finally available to build a similar setup. However, I am building it myself. So if I should be disappointed by the performance all I have to do is swap the motherboard and CPU(s).
But I guess the risk of Epyc performing much worse than anticipated is rather small. All benchmarks I have seen so far indicate that it meets the expectations.
What I would change is using at least 8GB DIMMs. They they should offer a better price/GB ratio

A word of warning: the workstation in the second link you posted will be loud as hell. Those supermicro cases and fans are not designed to be particularly silent. But the biggest problem is the power supply. In order to cram it into a small standard form factor, it is equipped with extremely noisy 40mm fans that will drive you crazy if this machine sits under your desk.
roenby likes this.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   October 25, 2017, 15:02
Default
  #3
Member
 
Johan Roenby
Join Date: May 2011
Location: Denmark
Posts: 45
Rep Power: 14
roenby will become famous soon enough
Thanks a lot for your quick and informative reply, flotus1!

I plan to stow the computer away in a technical installation shaft in my home but would still prefer that it is not ridiculously noisy.

I had a look again at you post here, and if I remove 1 CPU and its RAM and fan, then I get down to 2940$:

AMD Epyc:
E-ATX Case (~100$)
Power supply, e.g. Bequiet Dark Power Pro P11 550W (~130$)
Dual-socket board, e.g. Supermicro H11DSi (~550$)
1x AMD Epyc 7301, 16 cores (~1x 920$)
1x Noctua NH-U14S TR4-SP3 (~1x 80$)
8x8GB DDR4-2666 dual-rank (~800$)
GTX 1060 6GB (~270$)
SSD 250GB, e.g. Samsung 850 Evo (~90$)
+as much storage as you need, maybe even an m.2 SSD with 1-2 TB if you need fast file I/O for your transient simulations

With my current max budget 4000-4500$ this looks quite interesting with the option of adding a cpu at a later time.

I wouldn't mind building the thing myself. I had fun building my first computer many years ago and I played with Lego as a kid, so I guess it is doable :-)

In my communication with Velocity Micro, I was told that they are "against 2666MHz RAM, because in testing it is not as stable as the 2400MHz".

Any comment on that?

Kind regards,
Johan
roenby is offline   Reply With Quote

Old   October 25, 2017, 15:52
Default
  #4
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,621
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
If you already built PCs yourself and you don't need support and warranty for the whole system you can build it yourself for a lower price.
The only caveat is that you will have to wait a few more weeks until boards and CPUs become available from retailers.

For the memory...
You can have a look at the motherboard manuals at supermicro: they officially support RDIMM DDR4-2400 dual-rank and RDIMM DDR4-2666 single-rank. This is identical to the official memory support of AMD Ryzen and Threadripper with one DIMM per channel. I am still waiting for a response from their support people to clear up a few of the questions I had.
But then again, dual-rank DIMMs have proven to deliver better performance than single-rank DIMMs with higher speeds, especially for AMDs new Zen architecture.
So based on the little information I have my recommendation is: if you can get dual-rank DDR4-2400 DIMMs significantly cheaper than dual-rank DDR4-2666 you can buy them. Otherwise just get DDR4-2666 dual-rank. Running it at lower speeds until memory support catches up is always possible.
Speaking of memory support: if you absolutely want to avoid any issues you can stick to the memory QVL of your motherboard. Other modules might work as well, but there is no guarantee.
roenby likes this.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   October 25, 2017, 17:19
Default
  #5
Member
 
Johan Roenby
Join Date: May 2011
Location: Denmark
Posts: 45
Rep Power: 14
roenby will become famous soon enough
I can wait a few weeks - no problem.

I had a look at the tested memory blocks here and there are only 16 and 32 GB blocks with 2666 MHz which is overkill if I am to fill out all the 8 slots (around 185$ per Micron 16 GB block). With 2400 MHz they have a Micron 8GB block but it is only single rank. Similar story with the other Super Micro EPYC 7000 series rmotherboards here.

Please let us know if you hear anything regarding supported dual-rank 2666 MHz memory from the support people while we wait for the boards to become available.

Thanks so much,
Johan
roenby is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Strange residuals of the Density Based Solver Pat84 FLUENT 0 October 22, 2012 15:59
amd opteron based workstation needed ztdep Hardware 1 December 5, 2010 07:23
incorrect temperature in pressure based solution Kian FLUENT 1 July 6, 2009 05:59
Absorption coeff:Cell or domain based for boiler DEV FLUENT 1 October 1, 2008 06:57
Pressure based and Density based Solver Xobile Siemens 1 November 30, 2004 22:13


All times are GMT -4. The time now is 18:29.