CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

OpenFOAM server for many small runs

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 3, 2021, 04:19
Default OpenFOAM server for many small runs
  #1
New Member
 
Join Date: Jun 2021
Posts: 1
Rep Power: 0
awld is on a distinguished road
Hi all,


I'm planning to purchase some hardware to run OpenFOAM simulations. These will be fairly small run-of-the-mill steady state RANS runs with grid sizes mostly below 1 million. However, there will be lots of these runs, i.e. many thousands. So this is less of an on-demand machine and more of a 24/7 number crunching device. Setup-wise, no regard needs to be placed on multi-user access.



Considering the small size of these runs, memory bandwidth is probably more important than CPU freq or core count. So, a dual Epyc configuration with 2x32 cores and 8 DDR3200 DIMMs per socket is my baseline, assuming that the sub-million grid point cases are never too large for this.


I've read up a lot on benchmarking and general setup, both in this board and elsewhere. With the background taken care of, I'd really appreciate some input regarding the following:

  • Does my baseline make sense?
  • Is is generally advisable to look at Epyc models with larger L3, i.e. 7532 (or 7543) versus 7452?
I'm looking at different ways to maximize throughput:

  • Is there substantial benefit in going to a 2x64 core configuration? Obviously much more expensive, but I try to understand the options
  • Is it maybe beneficial to go to multiple compute nodes with lower core counts (i.e. several of the 2x32 baseline boxes)?
  • If so, what is the best system setup for these? Run them independently next to each other and do the coordination of runs via custom scripts, or set up a whole HPC environment with SLURM/PBS/whatnot controlled from a master node?
And finally some general points:

  • As for general advice, any opinions on whether a tower or rack is better now and in the long run? I'm thinking about expandability, but maybe there are other factors. One could possibly add a storage solution to a rack, while this is more difficult with a tower.
  • We are purchasing from Germany, any EU members on here able to offer advice on system integrators they had good experiences with?


That's a lot of questions. I'm aware that some of these are a matter of personal preference, but I'm looking for general input.


Any insight is much appreciated. Regards,
awld
awld is offline   Reply With Quote

Old   June 3, 2021, 16:25
Default
  #2
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 2,865
Rep Power: 40
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Does my baseline make sense?
I would think so
Quote:
Is is generally advisable to look at Epyc models with larger L3, i.e. 7532 (or 7543) versus 7452?
More L3 per core is better, but it is an expensive way to squeeze out just a bit more performance. I would generally only recommend that for codes with expensive per-core licenses, i.e. not OpenFOAM.
Also, 3rd gen Epyc is already released. I would only buy second gen now if there was a substantial discount.

Quote:
Is there substantial benefit in going to a 2x64 core configuration? Obviously much more expensive, but I try to understand the options
Two boxes with 2x32 cores will provide significantly more throughput than a single box with 2x64 cores. Scaling won't be great beyond 32 cores per CPU because memory bandwidth.
Quote:
Is it maybe beneficial to go to multiple compute nodes with lower core counts (i.e. several of the 2x32 baseline boxes)?
See above. And here: General recommendations for CFD hardware [WIP]
Quote:
If so, what is the best system setup for these? Run them independently next to each other and do the coordination of runs via custom scripts, or set up a whole HPC environment with SLURM/PBS/whatnot controlled from a master node?
For maximizing throughput on "many thousand" small runs, it is probably best to run each simulation on a single core only. And without having to manage resource usage for multiple users, some basic scripts should get you pretty far.

Quote:
As for general advice, any opinions on whether a tower or rack is better now and in the long run? I'm thinking about expandability, but maybe there are other factors. One could possibly add a storage solution to a rack, while this is more difficult with a tower.
Depends on a lot of factors. Do you have a server room? Plans for colocation? Do you have enough budget for several nodes? Massively expanding capacity later sure is easier in a rack, but you probably don't want a server running in your office. Not sure what to tell you due to lack of information.
Quote:
We are purchasing from Germany, any EU members on here able to offer advice on system integrators they had good experiences with?
Never bought from them myself, but their prices seem fair, and the configurations are generally reasonable: https://www.deltacomputer.com/workstation/amd-epyc.html
From their FAQ, they don't seem to do on-site support though. So if that's what you need, better look elsewhere.

Last edited by flotus1; June 4, 2021 at 04:29.
flotus1 is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[OpenFOAM.com] Installing/compiling OpenFOAM on a server without admin privileges foadsf OpenFOAM Installation 2 March 2, 2018 13:14
OpenFOAM file management (on Server) lifeinhand OpenFOAM Running, Solving & CFD 7 January 10, 2017 03:51
OpenFOAM Training Jan-Jul 2017, Virtual, London, Houston, Berlin CFDFoundation OpenFOAM Announcements from Other Sources 0 January 4, 2017 07:15
Install openFOAM from with proxy server Dhruval OpenFOAM Installation 3 October 18, 2014 16:38
Install OpenFoam on Redhat server maolongliu OpenFOAM Installation 4 May 25, 2010 03:47


All times are GMT -4. The time now is 14:42.