[the_recruit]

Hey all, I'm trying to put together an entry-level workstation for FEA. Because of my small budget I'm looking at commercial CPUs. Trying to decide between Ryzen 5950X and the new Raptor Lake 13900k.

I hesitate on the 13900k because of the 8P+16E cores. From what I've read on other threads here about the Alder Lake chips the E cores are worthless for FEA, is this true? If so it seems I'd be better off a 16 core Ryzen. Unfortunately the 7950X seems to price me out of my budget with the expensive AM5 boards and DDR5 RAM . I'd like 128gb memory. So that leaves me with the 5950X.

Also, I've had difficulty finding FEM benchmarking for commercial CPUs but I did just come across this. The 13900k outperforms the 5950X considerably. Even the 12900k looks much better. I'm not familiar with Calculix or details of their test, but it seems we wouldn't expect this if the E cores weren't helpful, no?

Thoughts? Any other suggestions? I'm not very familiar with workstation CPUs, is there a better option available in my price range?

1. Software will be primarily Ansys Mechanical. Potentially Abaqus at some point in the future, if that makes a difference. Solidworks will also be used a bit for CAD, but not for running simulations.
2. Up to 2 HPC packs (so up to 36 cores). Abaqus licensing TBD.
3. Linear and Non-linear implicit simulations.
4. $2k to $2.5k
5. Engineer
6. More than likely build myself. I've looked at suppliers but not much in our paltry budget range and from what I do see available I may get more value building myself.
7. United States

[flotus1]

That site is generally a reliable source. But not for this particular detail. The benchmark used here is FEA in name only. The hierarchy of CPUs makes no sense for real-world FEA applications.

With that out of the way: FEA is one of those workloads where memory bandwidth matters a lot. Possibly to even greater extent than FV-CFD.
Which means the latest-gen consumer CPUs with the highest core count are never a good choice.

Before we start looking for alternatives, I feel like we need to discuss budget. We can surely find better ways to use the current budget, but it seems really low to me.
You are currently paying for an Ansys base license and 2 HPC packs. And possibly more license costs for Abaqus in the future. Which should be several times more each year than the one-time purchase of a computer to use the licenses. Is the budget negotiable?

[the_recruit]

We share a number of base mechanical seats across 100+ engineers who mostly do design. None of us are full time analysts so we use Ansys occasionally as needed on laptops not well built for FEA with at most 8 cores and 32GB of RAM. Which greatly limits the size of simulations we can reasonably run imo. I've proposed a more dedicated workstation that can be shared. Boss is willing to entertain me but wants to start small. If we find it useful, then we can spend more.

$2k was number discussed but I think I can push it to $2.5k. More than that, maybe I can convince him if it makes a big difference. But I'd like to figure out what the options are.

With that out of the way: FEA is one of those workloads where memory bandwidth matters a lot. Possibly to even greater extent than FV-CFD.
Which means the latest-gen consumer CPUs with the highest core count are never a good choice.

OK so memory bandwidth will be the bottleneck in performance?

[flotus1]

Since you can build the thing yourself, you might be able to make the following configuration work for 2500$.
I'll give prices I have in €. My understanding is that lots of computer parts are currently cheaper in the US. You will have to check if you can source the parts cheap enough.

CPU: Intel Xeon Gold 6326 (16 cores, 8 channels) | 1025€
Socket 4189 motherboard, e.g. Supermicro X12SPL-F | 570€
8x16GB DDR4-3200 reg ECC | 640€
Case | 80€
PSU | 80€
CPU cooler | 50-100€
1TB SSD | 100€
More storage???
Graphics card???

That's 2600€. If you need a graphics card, something "cheap" in the 250€ range should do.

[the_phew]

My FEA experience is limited to SolidWorks Simulation, but in that solver, most of the wall clock simulation time is consumed by single-threaded tasks like 'Establish stiffness matrix' or whatever. The actual simulation is solved in parallel and runs quite fast even on consumer CPUs like the 5950X. Ansys Mechanical may be better about performing simulation setup in parallel?

Any serious FEA requires at least 128GB of RAM, so DDR5 with a consumer CPU is out (since both Ryzen 7000 and Raptor/Alder Lake downclock the RAM to DDR4 speeds if you populate 2 sticks per memory channel, which is necessary for 128 GB since there are no consumer 64GB DDR5 DIMMs yet). But with Raptor Lake, at least you can buy DDR4 boards to save on memory costs with 128GB.

If Ansys Mechanical runs simulations fully parallel from setup through solve/post, the best bet within your budget is probably an older 32-core server CPU like the EPYC 7542. Those are available used for well under US$1000. If it spends a lot of time crunching on a single thread, then maybe a 13900K with 128 GB of DDR4 would give faster overall solve times, since those 8 'P-cores' are the fastest cores money can buy right now. The main caveat with the Lake CPUs is that Windows 10 has no idea what to do with the 'E-cores'; you need to use Windows 11 or a Linux distro that can schedule processes correctly on hybrid CPUs. Or else you will always be bottlenecked waiting on the slower 'E-cores'.

[the_recruit]

CPU: Intel Xeon Gold 6326 (16 cores, 8 channels) | 1025€
Socket 4189 motherboard, e.g. Supermicro X12SPL-F | 570€

I will recommend this build, but it looks like it's coming in over $3k for me unless I can find a better deal on the Xeon somewhere.

If I wanted to go a step down, would a 10920X make sense? Only 4 channels, but is less than half the cost of the Xeon. Is 4 channels DDR4 enough to feed 12 cores? I also see a Threadripper 2920X available for even less.

[the_recruit]

Originally Posted by the_phew

My FEA experience is limited to SolidWorks Simulation, but in that solver, most of the wall clock simulation time is consumed by single-threaded tasks like 'Establish stiffness matrix' or whatever. The actual simulation is solved in parallel and runs quite fast even on consumer CPUs like the 5950X. Ansys Mechanical may be better about performing simulation setup in parallel?

I'll be honest I'm not sure. This is also a concern I have, that sacrificing single-thread performance for more cores may be counterproductive at some point. Makes me wonder if this is the difference in the Calculix benchmarking done by IgorsLab...

Originally Posted by the_phew

If Ansys Mechanical runs simulations fully parallel from setup through solve/post, the best bet within your budget is probably an older 32-core server CPU like the EPYC 7542. Those are available used for well under US$1000.

I have to admit I am hesitant to buy used. Is there a reputable place you'd recommend to buy/sell used?

Originally Posted by the_phew

If it spends a lot of time crunching on a single thread, then maybe a 13900K with 128 GB of DDR4 would give faster overall solve times, since those 8 'P-cores' are the fastest cores money can buy right now. The main caveat with the Lake CPUs is that Windows 10 has no idea what to do with the 'E-cores'; you need to use Windows 11 or a Linux distro that can schedule processes correctly on hybrid CPUs. Or else you will always be bottlenecked waiting on the slower 'E-cores'.

Oh wow, is this true for Alder lake too? This may be a problem, I will have to discuss with IT whether Windows 11 is even an option.

[the_recruit]

From Intel's white paper on Ansys benchmarking:

For applications such as Ansys Fluent where memory bandwidth regions tend to dominate performance, the benchmark results demonstrated that the 12 memory channels per socket of the Intel Xeon Platinum 92xx processor family means more cores can be utilized to deliver nearly 2x the performance.Overall, tests confirm that all of the Intel Xeon Scalable Processor families deliver higher performance when faster memory DIMM modules are installed in the compute node.

For applications such as Ansys Mechanical where compute regions tend to dominate performance, the benchmarks show that processors with high processor base clock rates and high maximum turbo frequency performed best on the Ansys Mechanical benchmarks. As observed in this white paper the Intel 92xx family of Intel Xeon Scalable processors delivered the highest single node performance relative to the other Intel Xeon Scalable processors tested.

Haha, now I'm even more uncertain

[the_phew]

Originally Posted by the_recruit

Oh wow, is this true for Alder lake too? This may be a problem, I will have to discuss with IT whether Windows 11 is even an option.

Windows 10 doesn't yet have 'Thread Director', although it may get such an update one day. Some applications that have been optimized for Intel processors with hybrid architectures will still perform fine in Windows 10; Ansys Mechanical may or may not be among them. This is what Intel has to say on the matter:
https://www.intel.com/content/www/us...rocessors.html

I've avoided the Alder/Raptor Lake processors for CFD/FEA because I don't want to deal with this potential headache (but mainly because of the memory downclocking issues with 128 GB of DDR5 on all current consumer processors that use DDR5).

I don't have a recommendation for buying Rome-generation hardware (I might have four 7542's to sell as soon as I can get my hands on Genoa-X CPUs ), but Ebay's buyer protection is strong if you follow the procedures correctly. I too would be concerned that single-threaded performance ends up being very important for ANY workstation, which is why the Threadripper Pro 5000 CPUs are my general-purpose workstation CPU pick nowadays. They are just horribly overpriced, but deals are starting to appear.

[flotus1]

I mean... the easy solution would be disabling the E-cores in bios.
Another reason why the I9-13900k is not good value here. The I7-13700k has the same amount of P-cores. And I doubt that scaling would be good enough with any of these CPUs to outclass the I5-13600k with its 6 P-cores.

[the_phew]

Workstation users were spoiled for years with Intel's 'Extreme Edition' HEDT processors, which had the perfect blend of decent core counts (for their day), high clocks (and overclockability), 4 memory channels, 256 GB memory support, and plenty of options under $1k. AMD briefly entered the fray with the non-pro Threadripper, which did almost everything better than Intel's HEDT offerings for modestly higher cost.

Then both Intel and AMD abandoned the HEDT segment, realizing that they could fleece workstation users with $3k+ 'Workstation' versions of server CPUs instead. So anyone trying to build a CFD/FEA workstation for under US$3k now has to choose between various brutal compromises (older-gen server chips, non-ideal latest-gen desktop chips, neutered workstation CPUs like the 5955wx, etc.) Generally, it's best to just stretch your budget to include actual server or workstation CPUs rather than find out after the fact that your workflow is crippled by the limitations of a desktop CPU.

[the_recruit]

I've read rumors that Intel may be rebooting the X Series. But should I wait around for a rumor that may never materialize...probably not.

[the_phew]

Intel is definitely launching 'Fishhawk Falls' workstation CPUs at some point after Sapphire Rapids server CPUs hit the market, but that looks to be an extension of their 'Xeon-W' line, which to date have just been also-rans compared to Threadripper Pro (while being even more expensive). But the Xeon W-34xx CPUs should actually leapfrog the Threadripper Pro 5000 CPUs for a few months at least this time (the current Xeon W-33xx isn't even any faster than Threadripper Pro 3000).

Of course there are also persistent rumors that AMD will resurrect the non-Pro Threadripper line, but I don't see that happening unless Intel decides to compete in HEDT again first. The problem is that both AMD and Intel have been too successful at getting creative/scientific users to upgrade to much more expensive 'Workstation' CPUs that are overkill in many areas for them.

[Cinek_Poland]

Originally Posted by flotus1

...The benchmark used here is FEA in name only.

Could you explain that ? This benchmark was using FEA code "calculix" right ?

[flotus1]

I qualified that statement in the sentence right after the the one you picked for a quote.
It's an FEA solver alright. But from the numbers reported on the site linked in the initial post, two issues stand out.
1) Internal consistency, lack thereof. The hierarchy of CPUs, and the margins between their results, make no sense. No matter how the solver behaves.
2) More or less a consequence of 1: the results and the conclusions one would draw from them do not translate to real-world FEA applications.
Do you have a shitty solver that is mostly limited by single-core performance? Fine, the I9-13900k is the fastest possible CPU. But the I5-13600k should not be far behind.
Do you have a well-behaved FEA solver that makes efficient use of parallel processes, and thus becomes limited by memory bandwidth? Great, but there is still no reason why the high core count CPUs, especially those with lots of weak E-cores, would lead by such a huge margin.
Does this benchmark manage to be compute-bound and make good use of "big-little" CPU architectures? Great stuff, but then it REALLY should not be used as an indicator for real-world FEA.

It reminds me of the OpenFOAM numbers curated on openbenchmarking.org, which has similar issues.

[otokemo]

Originally Posted by flotus1

Since you can build the thing yourself, you might be able to make the following configuration work for 2500$.
I'll give prices I have in €. My understanding is that lots of computer parts are currently cheaper in the US. You will have to check if you can source the parts cheap enough.

CPU: Intel Xeon Gold 6326 (16 cores, 8 channels) | 1025€
Socket 4189 motherboard, e.g. Supermicro X12SPL-F | 570€
8x16GB DDR4-3200 reg ECC | 640€
Case | 80€
PSU | 80€
CPU cooler | 50-100€
1TB SSD | 100€
More storage???
Graphics card???

That's 2600€. If you need a graphics card, something "cheap" in the 250€ range should do.

Is a 2x some older cheaper Xeon a go?
I saw some of 26xx series are sold for 70$