[aumit]

Dear forum members,

I am planning a workstation to run transient 3-D turbulent magnetohydrodynamic (MHD) flow simulations. As MHD flows are modelled with many more PDEs than conventional flows, I assume that the hardware demands are different as well. To those who do not know: MHD flows are modelled with Navier-Stokes, Maxwell and several material equations. More equations have to be added if more than one species is considered plus typically RANS equations. Often one ends up with more than 16+ PDEs.

In my specific case, I use a coupled FVM solver and solve the Maxwell and Navier-Stokes equations sequentially on the same mesh. The number of cores is not limited by our license. Is it in this case more suitable to use fast CPUs with a few cores or slower CPUs with more cores? Note that the geometry/mesh is extremely simple (almost cylindrical) and small compared to the typical ones described on the forum, but the flow is extremely complex and unsymmetrical!

My intended workstation is a Dell Precision Tower 7910 with two Xeon E5 v4 processors and 8x16 GB = 128 GB RAM (at least 64 GB recommend by a colleague). The CPU I had in mind is the E5-2687W v4 as it is both fast and also has a reasonable number of cores, but I would be very happy if I could choose a cheaper one, as this configuration is already stressing our budget heavily.

Thanks in advance!

[RobertB]

Almost all of these type of codes do better with more memory bandwidth. The latest Xeons have 6 channels (as opposed to four). The Gold ones starting with a 6 seem to be a fairly good starting point for CFD type applications.

https://www.anandtech.com/show/11544...-of-the-decade

Gives background and lists the Xeon SKUs. A 6126/6136 would seem to be similar to a 2687.

You didn't say how long you expected your runs to be, that can obviously affect the matter.

[aumit]

Hi Robert,

thanks for your suggestions. Dell now updated their punchout with the Dell Precision 7920 workstation, where I can select these new processors. As I understand, I would benefit from higher memory bandwidth and also faster RAM. Regarding the RAM, one should now go for 192 GB instead of 128 GB or am I wrong?

The downside is that a configuration with the 6136 and 192 GB is about 2000 Euro more expensive than my previous configuration with the 2687 V4, which was already too expensive.

My transient simulations will run for a simulation time of approx. 10 ms.

[RobertB]

You could go for the 6126 and save ~$1500 for probably a loss of 10% in speed.

For memory you are right but you could also go to 96GB if that is good enough. It comes down to need against number of slots and likelihood of needing the additional memory space in the future.

The run time question was wall clock time. If the runs are short then the speed is less important if they are each a week then twice as quick has major benefits.

SpecrateFP is generally a good indicator of overall performance on CFD, I would use that to do the cost benefit analyses of various options

http://www.spec.org/cpu2006/results/rfp2006.html

[aumit]

Thanks again for your tips. We are now in the process of buying the workstation. We have decided to choose one of these newer Xeon CPUs, but in the beginning we will buy only one CPU and add the second one later if we need it.

Unfortunately, the retailer we have to buy from is not so helpful and some of their suggestions contradict what seems to be common knowledge here in the forum.

Which CPU would you recommend: 6136, 6134 or 6128? I am a bit doubtful how one would actually benefit from many cores with less clock frequency considering that we have to solve the Maxwell and Navier-Stokes equations sequentially.

Also, they suggest to buy two SSD drives which I think is unnecessary. Furthermore, the cheapest GPU they recommend is the Nvidia Quadro P5000, which I again think is overkill.

[flotus1]

Unfortunately, this is common practice.
Your retailer earns more money by selling you a 2000$ Quadro P5000 than with a cheaper graphics card. I wonder which of their other "suggestions contradict what seems to be common knowledge". Care to elaborate?
To save some money you could have a look at AMDs new "Epyc" CPUs. Thanks to their 8 memory channels and relatively high performance per clock, they can outperform much more expensive Intel CPUs in memory bound workloads. A node with 2x AMD Epyc 7351 is a quite cost-effective number-cruncher.

[aumit]

Apart from several other configurations with even more expensive GPUs, they also proposed a RAM configuration with modules not matching the number of memory channels.

These Epyc CPUs sound interesting, but they are not available in the configurator from my retailer. I have spent so much time related to this workstation that I have to buy it as soon as possible to not loose any more time. What is your view on the CPUs I have mentioned considering my type of simulation?

Regarding GPU, I am thinking about a Quadro P1000 which is a lot cheaper than the P4000 and I think also sufficient.

[flotus1]

Hardly anybody got fired for choosing Intel over AMD The CPUs you picked will get the job done.
Your choice of graphics card depends on the programs you use and the model sizes. A Quadro P1000 has a decent price/performance ratio among the professional lineup. A GTX 1060 is cheaper, has more memory and more raw performance. It depends on the driver optimizations for the software you use if this is an option.
Which graphics card does your current workstation have and do you have reason to suspect it is bottlenecking your workflow?

[aumit]

Actually, I worked only on my (quite expensive) laptop on simple simulations until now, so I hope for tremendous improvements with this workstation. I think I will just go with the P1000, although after asking the retailer about the proposed configurations they said that anything below the P5000 is not recommended. And of course they also admitted that I was right about the memory configuration.

Do you know whether moving meshes require much better GPUs compared to static meshes?

[flotus1]

You won't be using your graphics card do do any computations, right? Because if you do, even a Quadro P5000 would very likely be a bad choice.
Displaying the solution or animating a moving mesh simulation is usually not bottlenecked by any decent graphics card. Post-processing is usually I/O-bound or limited by the CPU performance. This should include the P1000. Which software will you use for post-processing?
It would be interesting to see what reason your retailer has to back up his claim that anything below a P5000 is not recommended. I doubt that he even has enough information about your requirements to draw this line. The fact that he learned about the hexa-channel memory interface of Skylake-SP from you -his customer- should say it all.

I would say go for the cheaper graphics card. If you should ever feel that it is bottlenecking your workflow, you can still buy a better card. Probably overall cheaper than getting the P5000 right away from your retailer.

[aumit]

I agree that the recommendations of this retailer cannot be trusted. Essentially they did not provide any justification for the GPUs, but some of the arguments were just pathetic.

No, I am not planning to do GPU computations, just post-processing in Paraview.

[flotus1]

For Paraview you can definitely use a cheap GTX instead. I have been doing that for quite a while now without any issues. Our model sizes are typically 50 million, up to 250 million cells so far.
Graphics card for Paraview

[aumit]

Thanks again for the suggestions. I checked the punchout again, but this GTX card is not available. On the other hand, the price difference of the Quadro P1000 compared to other cheaper cards available in the punchout (some NVS and Radeon) is negligible, so I think I'll stick to the P1000. At least, I am confident now that the GPU will not be the bottleneck.

[aumit]

As early Christmas present, our new workstation arrived today. We now went for a setup with 2 Intel Xeon Gold 6128 CPUs, 12x8=96 GB RAM, 512 GB SSD, 2 TB second drive and Nvidia Quadro P1000 GPU. Thanks a lot for your help! Without it, we would have most likely gone for the old Xeon CPUs. I will comment on the performance when we have started using it.