2xEPYC Setup, Fluent, 10M Poly Cells

Jeggi · March 17, 2020, 10:48

Hi all,

We are about to buy a workstation for our CFD simulations. The software is FLUENT/OpenFOAM and we expect to have cases with around 10 million polyhedral cells. This does fit well with the Fluent licensing with 1 solver license and 2 HPC packs, giving a total of 36 cores.

The current specs are:
Cabinet: Inter-Tech X-908 https://www.inter-tech.de/en/product...ng-tower/x-908
Power Supply: https://www.corsair.com/eu/en/Catego.../CP-9020139-EU
Motherboard: https://www.supermicro.com/en/produc...oard/H11DSi-NT
2xCPU’s: https://www.amd.com/en/products/cpu/amd-epyc-7302
CPU Coolers: https://noctua.at/en/nh-u12s-tr4-sp3
16xMemory: https://www.scan.co.uk/products/4gb-...ered-12v-rdimm
GFX: https://www.msi.com/Graphics-card/Ge...ENTUS-XS-6G-OC
SSD: https://www.samsung.com/us/computing...b-mz-v7p512bw/
HDD: https://www.newegg.com/seagate-barra...82E16822179129
Fans: https://noctua.at/en/nf-a12x25-pwm
OS: Windows 10 Pro

Does this setup seem reasonable?

flotus1 · March 17, 2020, 14:11

The good news is: a system with these components will probably work. But of course I have some nitpicks...

Case: make sure it can fit SSI-EEB motherboards. It looks like it is physically large enough, but you might have to do some modifications to mount the board. Also, that front panel might look nice, but is completely blocks any air flow.
If you want to avoid both problems: Phanteks Enthoo pro. There is even a variant with a TG side panel, in case you want to look at those glorious green PCBs all day

Shifting budget: some of the components are on the pricey side, without offering a real benefit. You can save some money here, and spend it elsewhere.
PSU: 215€ and 1000W are overkill. Cheaper alternatives like Seasonic Focus Plus Platinum 750W would do the trick.
SSD: Unless you use a PCIe slot adapter, that m.2 SSD will only get two lanes anyway. The most expensive I would buy is Samsungs 970 Evo line. Exceeding its life span while using it as a system disk would have to be done on purpose.
HDD: At least in the link you provided, 4TB cost over 200$. There are plenty of non-SMR HDDs out there for much less €/TB.
Fans: Those new Noctua fans are great and all, but even I am reluctant to spend 30€ on one of their fans. If you use the case I suggested, 3-4 of their NF-A14 fans would be a great addition.
CPU coolers: bigger is better: NH-U14s. They cool better, and the fans are less annoying.

All the money saved should go towards memory. Epyc Rome supports up to DDR4-3200. The lowest I would go, if there is really no other way to meet budget restrictions, is DDR4-2933. As always: get dual-rank if possible. You will have to pay ~6-7€/GB for that.

Jeggi · March 19, 2020, 08:22

Hi Flotus1,

Thanks for your thorough reply.
We have asked our provider to do the adjustments that you recommend.

I don't really know why we were recommended 2666Mhz RAM, considering that the simulation time decreases linearly with RAM speed. Hopefully they are able to deliver 3200Mhz DDR4 RAM.

I'll let you know how it goes.

March 17, 2020, 10:48	2xEPYC Setup, Fluent, 10M Poly Cells	#1
Jeggi Member Jógvan Join Date: Feb 2014 Posts: 31 Rep Power: 12	Hi all, We are about to buy a workstation for our CFD simulations. The software is FLUENT/OpenFOAM and we expect to have cases with around 10 million polyhedral cells. This does fit well with the Fluent licensing with 1 solver license and 2 HPC packs, giving a total of 36 cores. The current specs are: Cabinet: Inter-Tech X-908 https://www.inter-tech.de/en/product...ng-tower/x-908 Power Supply: https://www.corsair.com/eu/en/Catego.../CP-9020139-EU Motherboard: https://www.supermicro.com/en/produc...oard/H11DSi-NT 2xCPU’s: https://www.amd.com/en/products/cpu/amd-epyc-7302 CPU Coolers: https://noctua.at/en/nh-u12s-tr4-sp3 16xMemory: https://www.scan.co.uk/products/4gb-...ered-12v-rdimm GFX: https://www.msi.com/Graphics-card/Ge...ENTUS-XS-6G-OC SSD: https://www.samsung.com/us/computing...b-mz-v7p512bw/ HDD: https://www.newegg.com/seagate-barra...82E16822179129 Fans: https://noctua.at/en/nf-a12x25-pwm OS: Windows 10 Pro Does this setup seem reasonable?

March 17, 2020, 14:11		#2
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,400 Rep Power: 47	The good news is: a system with these components will probably work. But of course I have some nitpicks... Case: make sure it can fit SSI-EEB motherboards. It looks like it is physically large enough, but you might have to do some modifications to mount the board. Also, that front panel might look nice, but is completely blocks any air flow. If you want to avoid both problems: Phanteks Enthoo pro. There is even a variant with a TG side panel, in case you want to look at those glorious green PCBs all day Shifting budget: some of the components are on the pricey side, without offering a real benefit. You can save some money here, and spend it elsewhere. PSU: 215€ and 1000W are overkill. Cheaper alternatives like Seasonic Focus Plus Platinum 750W would do the trick. SSD: Unless you use a PCIe slot adapter, that m.2 SSD will only get two lanes anyway. The most expensive I would buy is Samsungs 970 Evo line. Exceeding its life span while using it as a system disk would have to be done on purpose. HDD: At least in the link you provided, 4TB cost over 200$. There are plenty of non-SMR HDDs out there for much less €/TB. Fans: Those new Noctua fans are great and all, but even I am reluctant to spend 30€ on one of their fans. If you use the case I suggested, 3-4 of their NF-A14 fans would be a great addition. CPU coolers: bigger is better: NH-U14s. They cool better, and the fans are less annoying. All the money saved should go towards memory. Epyc Rome supports up to DDR4-3200. The lowest I would go, if there is really no other way to meet budget restrictions, is DDR4-2933. As always: get dual-rank if possible. You will have to pay ~6-7€/GB for that. Last edited by flotus1; March 18, 2020 at 05:18.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Foam::error::PrintStack	almir	OpenFOAM Running, Solving & CFD	92	May 21, 2024 07:56
[ANSYS Meshing] polyhedral mesh which is generated directly in Workbench Poly Meshing for Fluent	teguhtf	ANSYS Meshing & Geometry	0	January 24, 2019 22:04
Fluent UDF wrong number of cells in parallel - correct in serial	dralexpe	Fluent UDF and Scheme Programming	7	May 17, 2018 08:26
Running UDF with Supercomputer	roi247	FLUENT	4	October 15, 2015 13:41
[ICEM] Cells have only wall faces error in Fluent	Sparsh6	ANSYS Meshing & Geometry	0	March 18, 2014 05:31

March 19, 2020, 08:22		#3
Jeggi Member Jógvan Join Date: Feb 2014 Posts: 31 Rep Power: 12	Hi Flotus1, Thanks for your thorough reply. We have asked our provider to do the adjustments that you recommend. I don't really know why we were recommended 2666Mhz RAM, considering that the simulation time decreases linearly with RAM speed. Hopefully they are able to deliver 3200Mhz DDR4 RAM. I'll let you know how it goes.