CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

Epyc vs Xeon Skylake SP

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes

Reply
 
LinkBack Thread Tools Display Modes
Old   August 10, 2017, 13:13
Default
  #21
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
Right now there are two retailers in Germany that have 2 different dual-socket Epyc boards listed. They are not in stock, but the date of delivery is 11.08.2017.
Additionally, a single Epyc processor (AMD EPYC 7351, 16 cores) is available. Compatible coolers are already announced, e.g. from Noctua.
Skylake-SP is a different story. The CPUs have been widely available for quite a while now, but no dual-socket boards are in sight yet. Compatible coolers
It should be a matter of days until you can build your own Epyc system from retail parts.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   August 11, 2017, 05:12
Default
  #22
New Member
 
Join Date: Feb 2015
Posts: 4
Rep Power: 4
naveedgt is on a distinguished road
Based on the following benchmarks for OpenFoam, although limited on the information of the setup, it appears Threadripper in this case is quite inferior to to the Intel competition.

http://pclab.pl/art75073-19.html

Not sure how applicable this is to the Xeon/Epyc conversation, but I assume its quite relevant.
naveedgt is offline   Reply With Quote

Old   August 11, 2017, 05:23
Default
  #23
New Member
 
Join Date: May 2013
Posts: 20
Rep Power: 6
hpvd is on a distinguished road
pretty interesting!

what might be the reasons for this? Inter Die Latency?

Or simply a non optimized setup/config...
hpvd is offline   Reply With Quote

Old   August 11, 2017, 09:13
Default
  #24
Senior Member
 
Onno
Join Date: Jan 2012
Location: Germany
Posts: 120
Rep Power: 9
Kaskade is on a distinguished road
According to Google Translate the author calls OpenFOAM a FEM-tool. So chances are, while he got interFoam running (which can be daunting at first), he may lack some knowledge of simulation.

As long as we don't know the exact nature of the test case, I would take this benchmark with a large grain of salt.
Kaskade is offline   Reply With Quote

Old   August 12, 2017, 22:24
Default
  #25
New Member
 
Muteb Awaji
Join Date: Apr 2016
Posts: 2
Rep Power: 0
Goh77 is on a distinguished road
Tom's hardware benchmarks

http://www.tomshardware.com/reviews/...u,5167-12.html
Goh77 is offline   Reply With Quote

Old   August 13, 2017, 12:18
Default
  #26
Senior Member
 
Robert
Join Date: Jun 2010
Posts: 111
Rep Power: 10
RobertB is on a distinguished road
@naveedgt

The 4 memory channel TR will suffer against the 6 memory channel Intels.

Epyc has 8 memory channels and thus will tend to win - having more cores probably helps too.

Someone else posted

www.youtube.com/watch?v=gdYYRRDJDUc

but the Intels here only have four memory channels and thus would be expected to perform worse.

In general it would seem both manufacturers have put out a series of CPUs that will perform significantly better on CFD applications in this generation.

Now if only our understanding progressed as fast.....
RobertB is offline   Reply With Quote

Old   August 13, 2017, 12:52
Default
  #27
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
Quote:
The 4 memory channel TR will suffer against the 6 memory channel Intels
The TR counterpart from Intel (X299 platform, leaving out Kaby-Lake X) also has a 4-channel memory controller.
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   August 14, 2017, 04:14
Default
  #28
New Member
 
Join Date: May 2013
Posts: 20
Rep Power: 6
hpvd is on a distinguished road
Quote:
Originally Posted by flotus1 View Post
The TR counterpart from Intel (X299 platform, leaving out Kaby-Lake X) also has a 4-channel memory controller.
just an addition: only Xeon SP have 6-Channel
hpvd is offline   Reply With Quote

Old   August 14, 2017, 07:59
Default
  #29
New Member
 
W.T
Join Date: Oct 2012
Posts: 26
Rep Power: 7
dybuk is on a distinguished road
Quote:
Originally Posted by Kaskade View Post
According to Google Translate the author calls OpenFOAM a FEM-tool. So chances are, while he got interFoam running (which can be daunting at first), he may lack some knowledge of simulation.

As long as we don't know the exact nature of the test case, I would take this benchmark with a large grain of salt.
They probably use damBreak tutorial from OpenFoam (based on they description of the case). If cell count from tutorial was not changed(from original 2268 cells) whole analysis fits in L3, maybe even in L2 cache on AMD and Intel systems. If so, pclab benchmark is far away from typical simulation.
I will ask pclab for more information.
dybuk is offline   Reply With Quote

Old   September 1, 2017, 15:59
Default
  #30
New Member
 
Chad
Join Date: Jan 2017
Posts: 8
Rep Power: 3
chad is on a distinguished road
Out of curiosity, has anyone got their hands on the new AMD CPUs?

I will hopefully be ordering the 7281 and a Supermicro board in the coming weeks but was wondering if anyone had experience with them yet.
chad is offline   Reply With Quote

Old   September 5, 2017, 09:13
Default
  #31
Senior Member
 
Onno
Join Date: Jan 2012
Location: Germany
Posts: 120
Rep Power: 9
Kaskade is on a distinguished road
Ansys released a White Paper in which they directly compare the per-core-performance of a Xeon Gold 6148 to the previous two generations for a simulation on 32 cores in total: http://www.ansys.com/Resource-Librar...ssor?tli=en-us

Skylake-SP is faster of course, but despite 50% more memory channels and faster memory it is "only" between 15% and 36% faster.

So does that mean that the memory bandwidth stops being the bottleneck and the simulations become CPU limited? Meaning that you would profit from a higher clock rate? Or that there is still room for a boost due to the implementation of AVX-512 support in Fluent?

(There is a second white paper about the benefits of OPA instead of EDR: http://www.ansys.com/Resource-Librar...s-fluent-intel)
Kaskade is offline   Reply With Quote

Old   September 5, 2017, 10:33
Default
  #32
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
I guess this document is based on the same information, but readily available https://www.intel.com/content/dam/ww...uent-brief.pdf
I would interpret the results in a different way. 35% more performance per core is pretty substantial.
Remember: when talking about "memory bottlenecks" this does not mean that performance is entirely limited by memory performance. You will get a disproportional performance increase from using higher core count CPUs. And since these high core count CPUs are disproportionally more expensive they are usually not worth it. Especially when on an expensive "per-core" licensing model for for commercial CFD packages.
Maybe look at it this way: in order to get a similar improvement in performance per core on a 32-core license with Broadwell-EP, you would have to replace a single dual-socket workstation (2x16cores) with two nodes (2x8 cores each). That would be quite expensive and inconvenient.
To conclude: I don't think CFD will stop being memory sensitive. The gap between memory and raw CPU performance will continue to widen. We will see "new" technologies to compensate this in the future, e.g. another level of cache or HBM in the CPU package...
__________________
Please do not send me CFD-related questions via PM
flotus1 is offline   Reply With Quote

Old   September 6, 2017, 03:23
Default
  #33
Senior Member
 
Onno
Join Date: Jan 2012
Location: Germany
Posts: 120
Rep Power: 9
Kaskade is on a distinguished road
It's "up to" 35%. And the price pro core does not really increase with core number if you correct for the influence of clock frequency.

Dell called it 'saturation' in their benchmarks. In the sense of diminishing returns. In the end it's about deciding how much performance one needs and can afford.

Last edited by Kaskade; September 6, 2017 at 05:16.
Kaskade is offline   Reply With Quote

Old   September 11, 2017, 13:05
Default
  #34
New Member
 
Join Date: May 2013
Posts: 20
Rep Power: 6
hpvd is on a distinguished road
just some more benchmarks of cheap 2S Epycs in comparison to some others (more expensive Epycs, XEON SPs...)
https://www.servethehome.com/dual-am...nsive-2p-epyc/

I'm not quite sure yet if one can take any of these numbers to compare CFD-performance
hpvd is offline   Reply With Quote

Old   November 17, 2017, 13:30
Default
  #35
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
I finally got to do a quick&dirty performance comparison for one of our our in-house CFD codes.
Linux kernel versions were not the same and testing had to be done really quick without diving too much into details, so the numbers are just a rough estimate.
The code is an OpenMP parallel Lattice Boltzmann solver, so scaling on large numbers of cores is not ideal.

Intel System: 2x Xeon E5-2650v4 (2x12 cores), 8x16GB DDR4-2400 dual-rank, SMT off
time per iteration on 22 cores: ~63s

AMD System: 2x Epyc 7301 (2x16 cores), 16x16GB DDR4-2133 dual-rank, SMT off
time per iteration on 16 cores of one CPU: ~68s
time per iteration on 32 cores, both CPUs: ~43s

Especially when keeping in mind that the AMD system will translate higher memory speeds (up to DDR4-2666 supported) into better performance, these results are quite impressive.
At least promising enough for me to make the decision for my next workstation: -> dual AMD Epyc 7301.
Edit: There will be a more in-depth comparison once my new Epyc workstation is deployed.
tombo likes this.
__________________
Please do not send me CFD-related questions via PM

Last edited by flotus1; November 20, 2017 at 12:54.
flotus1 is offline   Reply With Quote

Old   November 20, 2017, 13:30
Default
  #36
Member
 
Knut Erik T. Giljarhus
Join Date: Mar 2009
Location: Norway
Posts: 33
Rep Power: 13
eric is on a distinguished road
Thanks for sharing, flotus! Will be interesting to see the in-depth benchmarks. Did you end up going for the build you posted in this thread? I am looking at something similar, but looks like no one here has the Epyc processors in stock yet.
eric is offline   Reply With Quote

Old   November 20, 2017, 14:34
Default
  #37
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
It will be similar to this setup.
The main difference in terms of performance is memory: I currently only have enough DDR4-2133 memory to fill all the slots and no plans to upgrade with the current memory prices. First thing I will try is running them at DDR4-2400 which would match AMDs (Supermicros) official specifications for dual-rank RDIMM.
Extrapolating benchmark results I have seen for Ryzen CPUs this should even yield slightly better performance than DDR4-2666 single rank.

I ended up ordering the CPUs from Alternate, one of the largest hardware retailers here in Germany. The CPUs are available but not in stock, they order them from their supplier once a customer places an order. Hence the 9 days of delivery. Fingers crossed, the status went to "not available" again today
One comparison that Intel clearly wins: availability of both CPUs and motherboards.

Edit: allegedly the CPUs should be ready to ship on Friday.
__________________
Please do not send me CFD-related questions via PM

Last edited by flotus1; November 22, 2017 at 12:37.
flotus1 is offline   Reply With Quote

Old   November 23, 2017, 18:56
Default
  #38
Member
 
Join Date: Oct 2011
Posts: 77
Rep Power: 8
naffrancois is on a distinguished road
Many thanks flotus for sharing your benchmarks, I am really interested to see how it goes
naffrancois is offline   Reply With Quote

Old   November 24, 2017, 04:58
Default
  #39
Senior Member
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 1,691
Rep Power: 26
flotus1 will become famous soon enoughflotus1 will become famous soon enough
So am I. CPUs should arrive today, motherboard perhaps next week. Who knows
Shopping for AMD parts is quite an adventure.

Edit: CPUs arrived last thursday. Motherboard came today. I got a different version than I ordered because of supply problems, but I stopped being picky.
naffrancois likes this.
__________________
Please do not send me CFD-related questions via PM

Last edited by flotus1; November 28, 2017 at 10:47.
flotus1 is offline   Reply With Quote

Old   November 29, 2017, 02:49
Default
  #40
New Member
 
Thomas Ewing
Join Date: Jul 2015
Location: Melbourne
Posts: 19
Rep Power: 4
tombo is on a distinguished road
What motherboard did they end up bringing you?

You may have seen this: https://www.anandtech.com/show/12084...ur-analysis-/4

An interesting comparison considering the source, but I'd like to see the comparison between the CPUs you chose or perhaps the 24c, they probably won't do much worse that the 7601 of memory bandwidth is limiting.
tombo is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
AMD EPYC or Intel Skylake-EP xuegy Hardware 18 August 18, 2017 13:11
AMD EPYC or Intel Skylake-EP xuegy OpenFOAM Running, Solving & CFD 0 June 27, 2017 08:00


All times are GMT -4. The time now is 10:37.