[Jimbo]

Hello all,

I am looking to build a dedicated CFD (primarily) and FEA (secondly) machine for hydrodynamics and structural consulting purposes in the naval architectural industry, but I am very green on the hardware front as I've always just had to use what we've had at uni and/or work, so have not thought about it much.

The CFD software I will be using is Simerics MP+ for Marine, which has a specialised templatised setup for vessel simulations. The license will be limited to 16 cores, so I want to optimise the machine build to maximise performance with this core number.

Any larger requirements and I'll start to look at cloud or cluster options.

I will be using Abaqus for FEA.

I would thoroughly appreciate any direction/suggestions.

So far I have (interpreted) the following information from various sources (please correct, add info as seen fit);

1). CFD will only use 2, 4, 8, 16, 32, 64 cores, so core arrangements outside of these (say 10, 12, 18 etc), won't utilise all cores?

2). It is generally better to spend money on a CPU with a higher clock speed than more cores.
(Even though I am trying to maximise with 16 cores, if the best performance achieved is with fewer cores, then I am all ears).

3). The performance of CPU's with increasing cores tend to scale poorly in CFD due to memory bottlenecking from the low number of memory channels, i.e. 2? From what I can see this tends to be anything less than 20 cores.

4). From what I have found, most, if not all CPU's with around 16 cores (my CFD license limit), only have 2 memory channels.

5). Given the above points, I understand that a dual setup of 2 x 8 core CPU's is the best solution so 2 x 2 memory channels can be utilised.
(I believe cooling also becomes less of the issue?)

The Intel CPU's appear to have higher clock speeds and cache than similar 8 core AMD offerings, but I am agnostic about a particular brand.

So final questions:

Any recommendations on CPU model? i9 with high cache and high clock speed?

More Cache or higher clock speed, or balance between?

What am I missing, have incorrect, or what else should I consider before committing to a setup?

I am also completely interested to hear about other hardware considerations other than CPU setups...!

NB: I realise there are a multitude of factors associated with the CAD/mesh model that may affect things.

Thanks in advance
Jimbo

[flotus1]

First things first: General recommendations for CFD hardware [WIP]

1). CFD will only use 2, 4, 8, 16, 32, 64 cores, so core arrangements outside of these (say 10, 12, 18 etc), won't utilise all cores?

In general, you can use any amount of threads you want. You are not limited to powers of 2. Scaling tests are often done with increments power of 2, so that might be where this misconception is coming from. But that's mostly for convenience and because programmers really like powers of 2
I have written quick&dirty implementations in the past that worked best for powers of 2. But any general purpose software will work fine with other thread counts.

2). It is generally better to spend money on a CPU with a higher clock speed than more cores. (Even though I am trying to maximise with 16 cores, if the best performance achieved is with fewer cores, then I am all ears).

By and large, that's an irrelevant question in the early stages of picking a CPU. Even more so since you already know the maximum amount of cores you can use.
It becomes relevant for software that can't utilize a large number of cores by design, or is held back by a large serial portion, where one thread quickly limits total performance. Which is not the case here.

3). The performance of CPU's with increasing cores tend to scale poorly in CFD due to memory bottlenecking from the low number of memory channels, i.e. 2? From what I can see this tends to be anything less than 20 cores.

Parallel CFD can run into memory bandwidth limitations, I agree.
Don't understand the second part though???

4). From what I have found, most, if not all CPU's with around 16 cores (my CFD license limit), only have 2 memory channels.

You were probably browsing consumer-grade hardware exclusively. Mainstream CPUs from both Intel and AMD are limited to 2 memory channels.
Above that, you have HEDT or "Prosumer" CPUs with 4 memory channels. Intels X299 Platform or AMDs Threadripper.
And further up the stack are proper workstation and server CPUs with currently 8 memory channels. Intels Xeon Gold lineup or AMDs Epyc 7000 series CPUs.

5). Given the above points, I understand that a dual setup of 2 x 8 core CPU's is the best solution so 2 x 2 memory channels can be utilised.
(I believe cooling also becomes less of the issue?)

That's a common misconception, but it isn't that easy. Most CPUs can not be used in a dual-socket configuration. They lack the CPU to CPU interconnect, and of course there are no motherboards with two of those sockets. That's a feature reserved for server CPUs.

The Intel CPU's appear to have higher clock speeds and cache than similar 8 core AMD offerings, but I am agnostic about a particular brand.

Bare CPU clock speeds can not be used to compare two entirely different CPUs. It works when comparing e.g. an Intel I5 CPU to a different I5 CPU from the same generation. Outside of this narrow scope, The comparison doesn't work.
As for L3 cache: one of the defining features of AMDs current offerings is huge L3 caches. Don't know where you got contradicting information.

Any recommendations on CPU model? i9 with high cache and high clock speed?

Long story short: since you are probably paying quite a lot for the software licenses, and earn money with this PC, I see only two realistic options:
1) The price-sensitive choice: AMD Epyc 7443p (list price 1337$) . 24 cores of the latest generation with decent enough clock speeds, 8 memory channels. There is a 16-core variant of this CPU. But once you factor in the cost for all the other components, the difference in CPUs is a small price to pay for the ability to still use your workstation for lighter tasks while it is solving on 16 cores.
2) The all-in option: Epyc 74F3 (list price 2900$). Same as option 1, but with higher clock speeds and twice the L3 cache. Due to some weirdness in AMDs pricing, the 16-core version of this CPU actually costs more. So not worth it. Similar to option 1, once you tally up all the other components for a whole workstation, The price difference between the slower 7443p and this CPU isn't that high for the performance bump.

There are some interesting alternatives on the horizon. AMD might soon release Threadripper Pro CPUs based on their current Zen3 architecture. You might be one of the few people where this CPU actually is the best choice. But it hasn't been officially announced yet, and the performance increase compared to Epyc 74F3 will probably be pretty slim. And you might be waiting months for actual availability.

[Jimbo]

Originally Posted by flotus1

First things first: General recommendations for CFD hardware [WIP]


In general, you can use any amount of threads you want. You are not limited to powers of 2. Scaling tests are often done with increments power of 2, so that might be where this misconception is coming from. But that's mostly for convenience and because programmers really like powers of 2
I have written quick&dirty implementations in the past that worked best for powers of 2. But any general purpose software will work fine with other thread counts.


By and large, that's an irrelevant question in the early stages of picking a CPU. Even more so since you already know the maximum amount of cores you can use.
It becomes relevant for software that can't utilize a large number of cores by design, or is held back by a large serial portion, where one thread quickly limits total performance. Which is not the case here.


Parallel CFD can run into memory bandwidth limitations, I agree.
Don't understand the second part though???


You were probably browsing consumer-grade hardware exclusively. Mainstream CPUs from both Intel and AMD are limited to 2 memory channels.
Above that, you have HEDT or "Prosumer" CPUs with 4 memory channels. Intels X299 Platform or AMDs Threadripper.
And further up the stack are proper workstation and server CPUs with currently 8 memory channels. Intels Xeon Gold lineup or AMDs Epyc 7000 series CPUs.


That's a common misconception, but it isn't that easy. Most CPUs can not be used in a dual-socket configuration. They lack the CPU to CPU interconnect, and of course there are no motherboards with two of those sockets. That's a feature reserved for server CPUs.


Bare CPU clock speeds can not be used to compare two entirely different CPUs. It works when comparing e.g. an Intel I5 CPU to a different I5 CPU from the same generation. Outside of this narrow scope, The comparison doesn't work.
As for L3 cache: one of the defining features of AMDs current offerings is huge L3 caches. Don't know where you got contradicting information.


Long story short: since you are probably paying quite a lot for the software licenses, and earn money with this PC, I see only two realistic options:
1) The price-sensitive choice: AMD Epyc 7443p (list price 1337$) . 24 cores of the latest generation with decent enough clock speeds, 8 memory channels. There is a 16-core variant of this CPU. But once you factor in the cost for all the other components, the difference in CPUs is a small price to pay for the ability to still use your workstation for lighter tasks while it is solving on 16 cores.
2) The all-in option: Epyc 74F3 (list price 2900$). Same as option 1, but with higher clock speeds and twice the L3 cache. Due to some weirdness in AMDs pricing, the 16-core version of this CPU actually costs more. So not worth it. Similar to option 1, once you tally up all the other components for a whole workstation, The price difference between the slower 7443p and this CPU isn't that high for the performance bump.

There are some interesting alternatives on the horizon. AMD might soon release Threadripper Pro CPUs based on their current Zen3 architecture. You might be one of the few people where this CPU actually is the best choice. But it hasn't been officially announced yet, and the performance increase compared to Epyc 74F3 will probably be pretty slim. And you might be waiting months for actual availability.

Thanks Alex - awesome and quick feedback.

I think I understand everything you've brought up and filled the gaps in my knowledge which were leading to erroneous beliefs!

I'll digest further and revert with any further questions.

Thanks again.