[Freewill1]

Hi, all!

I am looking to buy an EPYC (9004 or 9005) workstation for CFD-like modelling, using a FEM mantle convection code, ASPECT (details: https://aspect-documentation.readthedocs.io/en/latest/), which supports MPI-based parallelism quite well.

My budget is 5,000–10,000 USD.

Here is my typical situation:

1. 1000x1000 2-D FEM mesh (i.e., 1M mesh);
2. 200x200x200 3-D FEM mesh (i.e., 8M mesh);
3. Focus primarily on Stokes flow (or creeping flow, solving the simplified Navier-Stokes equations) or Rayleigh-Benard convection (i.e., thermal convection);
4. Nonlinear iterations are required and can significantly slow the modeling due to nonlinear rheology of the mantle rock as it convects (20x time required compared with a linear, isoviscous case)

To maximize the performance to the fullest extent possible for the given budget, I'm not sure about the following:

• Dual-socket vs. single-socket: Does a dual-socket system run twice as fast as a single-socket one? Is there a bottleneck in inter-socket communication for the 9004 or 9005 series CPUs? A dual-socket system costs roughly twice as much as a single-socket one, and I don't want to waste the budget.
  
• How many cores per memory channel is most cost-effect? For both a dual-socket and a single-socket machine, how many CPU cores can 12 or 24 memory channels sustain—32, 64, or more? I remember, as a rule of thumb, each memory channel can saturate only 2–3 cores for the older 9001/9002/9003 EPYC CPUs (3200 MT/s, DDR4). Not sure whether the newer 9004 or 9005 can sustain more cores due to higher memory frequency (4800/6400 MT/s, DDR5) and other technological improvements.
  
• 9004 with 3D V-Cache vs. 9005: Does a 9004 CPU with 4800 MT/s RAM support and a large 3D L3 cache (e.g., 768 MB for EPYC 9384X) perform better than a 9005 CPU with 6400 MT/s RAM support? My ~1M FEM mesh is not large and can be entirely stored in the L3 cache; would each run thus speed up?

Here are my tentative single-socket hardware configurations:

Option-1:

1. CPU: EPYC 9384X, 32 core
2. RAM: 12 × 16 GB = 192 GB, 4800 MT/s
3. Motherboard: Supermicro H14SSL-NT (with 6400 MT/s RAM and 10 Gbps LAN supports)
4. ...

Option-2:

1. CPU: EPYC 9355, 32 core
2. RAM: 12 × 16 GB = 192 GB, 6400 MT/s
3. Motherboard: Supermicro H14SSL-NT (with 6400 MT/s RAM and 10 Gbps LAN supports)
4. ...

Any suggestion for cost-effect configuration(s)?

Best,

Freewill

[wkernkamp]

I think both are good choices for CFD single socket configurations. With CFD it is all about bandwidth. However, the 3D cache reduces cache misses and, therefore, the need for bandwidth. My guess would be that the fourth gen X processor will outperform the 5th gen non-X even though the latter has 33% more bandwidth. To talk about cost effectiveness, I would need to know what the price is for each of these processors. (I am assuming about equal now).


If later on you want to increase your capacity with a dual socket motherboard. Your 5th generation selection is a P part, so can only run on single socket active config.

[avareed]

For a ~10 million element FEM mesh, both the EPYC 9004 with large L3 cache and the 9005 are strong options, but it depends on your workload. If your solver is highly cache-sensitive, the 9004 with 3D V-Cache can offer excellent performance. However, for memory-bound workloads that benefit from higher memory bandwidth, more cores, and newer architecture improvements, the 9005 is likely the better choice. For most large-scale FEM simulations, I’d lean toward the 9005 for a more balanced and future-proof setup.