Recommendation for new system for CFD and DEM
 

Hello everyone!


I am currently in the proces of choosing a new system that will be used mostly for CFD and DEM analysis.

1.Which software do you intend to use?
Ansys Fluent/Rocky
2.Are you limited by license constraints? I.e. does your software license only allow you to run on N threads?
Ordinary licence on 4 core. Will probably get HPC1 or HPC2 next year.
3.What type of simulations do you want to run? And what's the maximum cell count?
CFD and DEM of complex processes in fluid bed driers, coaters, mixers etc., with up to 5 million cells.
4.If there is a budget, how high is it?
5500€ exVAT.
5.What kind of setting are you in? Hobbyist? Student? Academic research? Engineer?
Academic research.
6.Where can you source your new computer? Buying a complete package from a large OEM? Assemble it yourself from parts? Are used parts an option? Our institution policy is only new fully functional computers with warranty of 3 years so unfortunately no DIY (as much as I would like that) or used systems.
7.Which part of the world are you from? It's cool if you don't want to tell, but since prices and availability vary depending on the region, this can sometimes be relevant. Particularly if it's not North America or Europe.
Europe.


I would appreciate any feedback on the matter as I am currently going in circles between different configurations that I made after going through web sites. I made some configurations listed below considering information I got from the other threads in this Hardware section of the forum. Configurations:



1. 1U 1P Epyc 7003 system

• AMD EPYC™ 72F3 Processor 8-core 3.70GHz 256MB Cache
• 8x16GB PC4-25600 3200MHz DDR4 ECC RDIMM (128GB)
• 2TB SSD

2. Workstation with Threadripper Pro 5000x processors

• AMD Threadripper Pro 5975ex (32 cores)
• 128GB DDR5 4800MHz ECC RDIMM
• 2TB SSD

3. 1U 1P Epyc 9004 system

• AMD EPYC™ 9224 Processor 24-core 2.50GHz 64MB Cache
• 12x16GB PC5-38400 4800MHz DDR5 ECC RDIMM (192GB)
• 2TB SSD

4. 1U 2P Epyc 7003 system

• 2xAMD EPYC™ 7313 Processor 16-core 3.00GHz 128MB Cache
• 16x8GB PC4-25600 3200MHz DDR4 ECC RDIMM (128GB)
• 2TB SSD

5. 1U 1P Epyc 7003 system

• AMD EPYC™ 7543P Processor 32-core 2.80GHz 256MB Cache
• 8x16GB PC4-25600 3200MHz DDR4 ECC RDIMM (128GB)
• 2TB SSD

All the configurations are in the same ballpark regarding price around 5500EUR. Which configuration would be best performing with 4 cores, but would also represent the best investment for when HPC1 and or HPC2 would be available. Budget constraints rule out 3DV cache of either variants of Epyc 7003 or 9004 series as much as I would like to include them.


Am I correct assuming that configuration 3 (Epyc 9004 series) although its CPU having much higher memory bandwith and Z4 cores, would be negatively influenced by the lower L3 cache (64MB) and lower frequencies of CPU cores, which would negate any advantage of 9004 series in this case as compared to other configurations.


So now I am going in circles between 32 core configurations, since for my case seems to represent the best ratio of price/performance (configs 2., 4. and 5.), but cannot decide. Any feedback would be greatly appreciated. If I am greatly mistaken in my logic regarding the configurations I would appreciate any alternative options that I missed.

Hi
Will you have access to a compute cluster through your institution?
Is GPU computing included in your rocky license?

You could go with 1x 9124 on a dual-processor motherboard, and upgrade your processors with 2x 9454 or similar + RAM when your institution decides to go ahead with HPC packs and processor prices drop. Make sure your warranty allows you to open your computer.

64MB L3 is not sufficient to fully load 16 cores, but it's plenty for 4 cores :-)
Genoa is fast: a 32-cores Genoa processor is 27% faster than 64-cores MilanX

This is a good option but you should check with the dual CPU motherboard manufacturer if it could operate with a single CPU only. Not all dual CPU motherboards would run with an unpopulated CPU socket.

Thank you on your feedback.


Our DEM work involves irregular polyhedral particle shapes and as such needs double precision. We looked at the GPUs on the market and due to the severe constraints in double precision capabilities of the consumer and workstation lines of GPU available and our the budget constraints we never really thought about GPU computing as a serious alternative. Database level GPUs with good DP capabilities are way up there with price (pci cards AMD MI210, A100, H100 and their lower tier variants). The cheapest single such card is above the budget we have available.


Anyhow the best consumer or workstation cards (RTX 30x0, 40x0, A4500-6000 and 5000-6000 ADA) have DP compute capabilities in range of a good middle range single Epyc CPU (0,7 to 1,4 Tflops fp64). In view of that we think just focusing on CPU side is a more universal solution and dare I say better under our constraints.

Perhaps for spherical particles one could just use a much cheaper system with consumer grade GPU like 4090 with its massive 83Tflos of fp32.


But I did look at options to expand upper configurations with GPUs that fit the budget. I came up with a configuration for a workstation with the Threadripper Pro 5945wx with 2x RTX A4500:

• AMD Threadripper Pro 5945wx (12 cores) 64MB L3
• 128GB DDR5 4800MHz ECC RDIMM
• 2TB SSD
• 2x Nvidia RTX A4500 20GB (with NVlink)

Combined fp64 from both GPU is around 1,5T flops in this system, which I think is fp64 compute performance roughly equal to Genoa-X (32-cores). But on the other hand it has only 40GB of video RAM.


We will take into consideration the option regarding starting with 1x9124 and then switching after some time. Also that warranty regarding opening your computer is spot on. I added that to the spec list directly. Thanks.


Spec regarding two CPU MB working only when only one CPU is installed is also noted. Thanks.

u can get an amd instinct mi50 for a song (<150 euros) on ebay. these are chinese ex-mining cards but who cares. they will give you 3.5 fp64 tflops when using dgemm easy but forget about the 6.6 tflops due to bandwidth limitations. you will not find cpus doing this much at any sane price.