[Peter_600]

2 x AMD EPYC 7351, 16 x 8GB DDR4, Supermicro h11dsi-nt
OpenSuse Leap 15.1, Kernel 4.12.14-lp151.28.20-default
OpenFOAM-7

 # cores   Wall time (s):
------------------------
1 686.41
2 420.24
4 171.41
6 115.23
8 90.14
12 66.58
16 54.64
20 48.03
24 43.04
28 42.56
32 35.76

[flotus1]

Originally Posted by ctd

2X EPYC 7302, 16x16GB 2Rx8 DDR4-3200 ECC, OpenFOAM v5, Ubuntu 18.04.3
[...]

Forgot to ask: did you run this in NUMA or UMA mode? AMD calls it NPS4 if I am not mistaken.
https://www.anandtech.com/show/14694...epyc-2nd-gen/6

[ctd]

flotus1,
It was run in the default, "one NUMA domain per socket". I haven't had the opportunity yet to experiment with the options in:
https://developer.amd.com/wp-content...56745_0.75.pdf

I can try running the NPS4 setting if you're interested, but I may need some guidance on how to set it. I didn't initially see it in the bios, but could have missed it.

mh-cfd,
The Motherboard is a SuperMicro H11DSi version 2.0. It was purchased from https://www.interpromicro.com/ based on a tip from the thread below:
https://forums.servethehome.com/inde...yc-rome.25430/

[flotus1]

Due to an appalling lack of Epyc Rome equipment on my part, I can not help you with finding that bios option. But I would not be surprised if Supermicro just left it out. "Screw that noise, more options would just confuse our customers"

It is partly out of curiosity, but I also think it should give you some better performance with NUMA-aware software like OpenFOAM.

[jakethejake]

2 x AMD EPYC 7371, 16 x 16GB DDR4 Dual-Rank, Supermicro h11dsi
Windows 10 Pro Vers. 1903 Build 18362.418 - WSL Ubuntu 18.04 LTS
OpenFOAM-6 (precompiled package from openfoam.org)

# cores   Wall time (s):
------------------------
1 1254.01
2 447.25
4 212.51
6 139.17
8 101.92
12 88.24
16 88.04
20 83.5
24 74.72
28 70.44
32 87.87

I expected it to be worse than native Linux OS installed. But anyway its slower than I expected from what I see around here.
Any ideas? Or is it just windows 10?

[flotus1]

WSL = Windows subsystem for linux?

It might not be the best solution if you want near bare-metal performance. You might want to try a dockerized version of Openfoam, or a proper VM.

[Simbelmynë]

My experience is that the WSL is almost as fast as native Linux for this benchmark. Writing to disk often should be avoided though.


Here is some (old) information. WSL has seen improvements after this post.


https://www.phoronix.com/scan.php?pa...900x-wsl&num=1

[jakethejake]

Coming back with fresh results.. hardware build from above:
2 x AMD EPYC 7371, 16 x 16GB DDR4 Dual-Rank, Supermicro h11dsi


Software: both OpenFOAM-6 (precompiled package from openfoam.org)


1) Ubuntu 18.04.3 LTS (natively installed)

# cores   Wall time (s):
------------------------
1     838.71
2     395.21
4     193.89
6     120.27
8     88.43
12     64.74
16     48.43
20     44.14
24     38.32
28     37.14
32     32.7

2) Windows 10 Pro Vers. 1903 - Dockerized Ubuntu 18.04 (File I/O outside container), Docker has full access to the hardware

# cores   Wall time (s):
------------------------
1     1512.35
2     815.88
4     379.47
6     292.36
8     255.1
12     195.8
16     183.8
20     182.69
24     173.91
28     170.37
32     173.75

Results looking like somewhere else in this thread. Docker under windows seems to perform poorly and worse than WSL (at least with my configuration of a quite simple openfoam container).

[SLC]

Originally Posted by ctd

flotus1,
It was run in the default, "one NUMA domain per socket". I haven't had the opportunity yet to experiment with the options in:
https://developer.amd.com/wp-content...56745_0.75.pdf

I can try running the NPS4 setting if you're interested, but I may need some guidance on how to set it. I didn't initially see it in the bios, but could have missed it.

mh-cfd,
The Motherboard is a SuperMicro H11DSi version 2.0. It was purchased from https://www.interpromicro.com/ based on a tip from the thread below:
https://forums.servethehome.com/inde...yc-rome.25430/

Are you able to run Fluent benchmarks?

[HBH_aero]

Hi!

Is the below the only EPYC Rome benchmark available here?

I am looking to get a new Linux workstation, currently considering the Epyc Rome CPUs.

By the way - looks really promising from the below results!

Originally Posted by ctd

2X EPYC 7302, 16x16GB 2Rx8 DDR4-3200 ECC, OpenFOAM v5, Ubuntu 18.04.3

# cores   Wall time (s):
------------------------
1 723.64
2 328.11
4 164.21
8 81.4
12 55.2
16 41.1
20 37.53
24 34.27
28 29.99
32 26.89

[flotus1]

Yes, these are the only Epyc Rome results we have so far.
Yet I don't think you can go wrong with them. Especially for a general purpose workstation, they are a huge improvement over 1st gen due to the less complicated NUMA topology.

[SLC]

I’m in contact with a compute service provider who is offering to perhaps benchmark some fluent cases for me on a dual 7302 setup. Will post back if I get it done.

[HBH_aero]

I agree flotus1 - maybe the doubled L3 cache has a say too?
Wondering if they manage to increase the clock freq in the future.

[spwater]

Here is my result. Newlt configured workstation with Threadripper 3960x, 3.8 GHz 24C, 64 G memory (4 channel)

# cores Wall time (s):
------------------------
1 550.49
2 299.15
4 161.65
6 120.55
8 101.56
12 99.13
16 93.74
20 93.71
24 93.65

[spwater]

Hi I am wondering why my 3960x is worse so much than the EPYC. Their design should be close and 3960x do have 24 native cores.
Anything I may be missing?

Originally Posted by HBH_aero

Hi!

Is the below the only EPYC Rome benchmark available here?

I am looking to get a new Linux workstation, currently considering the Epyc Rome CPUs.

By the way - looks really promising from the below results!

[erik87]

Originally Posted by spwater

Hi I am wondering why my 3960x is worse so much than the EPYC. Their design should be close and 3960x do have 24 native cores.
Anything I may be missing?

Epyc has 8 memory channels for using only one cpu. The older and newer Threadripper run on 4 memory channels, despite for the newest TRx Platform, that can run on 8 channels.


So there is a lack on memory channels cosing worser results than on Epyc.

[flotus1]

Hence why Epyc 2nd gen is the better choice compared to TR 3000, at least with CFD/FEA as the main applications.
TR is better suited for the elusive "content creator" type.

Your TR 3960 results don't seem too far off. A TR 1950X on the first page of this thread is more than 50% slower running all cores. You could probably still optimize memory latency and timings. Which memory are you using, at which frequency and latencies?
And you could probably use NUMA mode for a slight performance increase in OpenFOAM.

[spwater]

You can check the memory in the attachment. It is some ordinary DDR4 memory for daily usage, they are running under XMP with 3200Mhz.

May I know the bandwidth and latency of a decent EPYC system will be like?

Thank you!

I try to run three 8 cores simulation simultaneously and all become slow. So the bottleneck of memory limit the paralel performance of Threadripper CPU, at least for OpenFOAM.

Also, even though the motherboard has 8 dimms for memory, the maximum channels supported by threadripper 3nd is 4, so there wont be any improvement if I top up them.

Originally Posted by flotus1

Hence why Epyc 2nd gen is the better choice compared to TR 3000, at least with CFD/FEA as the main applications.
TR is better suited for the elusive "content creator" type.

Your TR 3960 results don't seem too far off. A TR 1950X on the first page of this thread is more than 50% slower running all cores. You could probably still optimize memory latency and timings. Which memory are you using, at which frequency and latencies?
And you could probably use NUMA mode for a slight performance increase in OpenFOAM.

[flotus1]

May I know the bandwidth and latency of a decent EPYC system will be like?

A single Epyc Rome CPU has almost twice the memory bandwidth, and slightly higher latency. Epyc 7402P is the closest match for TR 3960X.

[wkernkamp]

Geekbench 5 Compute scores (Linux)

Processor 7402P 3960X

Single-Core Score 1035 1338
Multi-Core Score 22720 25199


Shui Pei, you see that in the Geekbench test, the threadripper does better than the comparable Epyc. However, that test is a little light on the memeory access speed factor, which dominates the CFD results on this thread.