Operating System for AMD Epyc Workstation
 

Hi guys,


I have an AMD EPYC Workstation (2x 7371) and I am considering to run Windows 10 on it. But first I have some questions left over.



I found some other threads in this forum which stated that running W10 together with AMD EPYC Gen1 is causing some troubles, which has something to do with the management of the high amount of the memory channels the EPYCs have. How serious are these troubles especially in terms of running CFD Apppliactions in parallel (e.g. ANSYS Fluent)? --> Is W10 along with Dual EPYC build viable at all?



I read that "Windows 10 Pro for Workstation" would be needed (--> which means purchasing a special license), because other W10 Versions don't support EPYC processors https://docs.microsoft.com/en-us/windows-hardware/design/minimum/windows-processor-requirements


Does anyone has some experience using this Version of W10 along with a dual AMD EPYC Configuration? Is it viable or does it fix the already known troubles with W10-driven EPYC-Systems?


Cheers,
jakethejake

You can technically run that system with a Windows 10 Pro license, which now allows for 2 CPUs. I did that briefly to check something unrelated.
I can't tell you if there still is a performance penalty with more recent builds of Windows 10, or if the scheduler has caught up with Linux.
Is there any software that specifically requires you to use Windows?

Well, kind of. And let's say Im not gonna be the only one to use the workstation. That's why we are considering to use something "everyone is familar with" like Windows 10.


The Workstation will run most of the time ANSYS Meshing, Fluent and OpenFOAM (I know with W10 this means VMs or WSL then).



Latter is not the only reason why I think my favourite solution is our second consideration to install CentOS 7.X. because with a updated kernel I expect it to yield better performance out of the EPYC system than the W10-driven solution does.



Nevertheless I am still collecting arguments for that decision, which is not completely in my hand...


So you are saying that this special version "... for Workstation" is not needed to be purchased? Does it just offer support of even more CPUs / RAM ? And with the "now" you are referencing to some recent updates of W10 Pro with made that 2 CPU usage possible?

It must have been quite a while ago, but if I remember correctly, Windows 10 Pro did not always support dual-CPU. I could be wrong about that part though, but Microsoft definitely changes the "allowed" hardware over time. Win 10 pro for workstations allows 4 CPUs.
Anyway, Microsoft is currently using a "one kernel" strategy for their windows operating systems. So there should not be any noticeable difference between "Pro" and "Workstation" when it comes to the type of hardware supported. Aside from the artificial restrictions through different licenses.
It might still be worth giving it a try. If you are in an academic setting, a workstation license should not be too expensive? Maybe the IT department already has one floating around?

Why would you want to run a Windows system with your dual Epyc?


If you only use it for Fluent calculations then you are better off just installing a non-gui version of CentOS.


If you use it as a workstation then I would opt for CentoOS (regrettably only version 7.x at the moment since ANSYS is terrible at Linux software compatibility).

@flotus1 Ok thanks for the infos on W10. Maybe we will first try a normal W10 Pro Version because unfortunately there is no license of the "Workstation Version" floating around :(. But anyway, before that I have to discuss with my colleagues which OS we are going for.



Well, as I mentioned above. I'm not the only one who is going to use this WS. I have to discuss with colleagues. At the moment W10 is preferable for them because "everyone is familiar with it" and from softwares perspective we know that our desired tools run without any doubts on W10. You stated it by yourself - e.g. ANSYS is terrible at Linux software compatibility. From hardwares perspective we are not sure yet and thats why Im trying to collect some infos.
But let me say it like this: I still have doubts that W10 will run without any troubles on that EPYC system and therefore I have to argue against my colleagues or lets say argue in favour of another solution like CentOS.


Regarding Linux I have more experience in Ubuntu than in RHEL/CentOS. So for own interests I already set up a VM with CentOS 7 on my private desktop for testing things like kernel update etc. I started with CentOS 7.6 image and yum update brought me to version 7.7.1908. I don't have access to ANSYS on my private machines. Does it also run on 7.7? (In ANSYS supported platforms document I remember it says up to CentOS 7.6)
Why "regrettably only version 7.x"?

I think that if your colleagues can use the super button and search for installed software then they will manage CentOS with Gnome just fine. Also, ANSYS is working just fine on CentOS 7.x.




CentOS 7.7 works. It is "regrettably only version 7.x" in the sense that 8.x does not work right now. CentOS 8 shows the same type of errors that we can see on other Linux distributions such as Ubuntu/Debian. If they fix it for CentOS 8 then I imagine that it will also work on some other newer distros.


Btw, it you are adamant about Ubuntu based systems then you can always use Ubuntu 16.04 or Linux Mint 18.3 for a couple of years (end of life 2021) since they work for current releases of ANSYS.


Disclaimer: we are actually running Ubuntu 18.04 on some of our EPYC machines due to hardware support issues with CentOS. Fluent and design modeler works on those, but not meshing. This is not a problem for us since the EPYC machines are only used for solving problems that are setup on other machines.

Does ANSYS Meshing work for Ubuntu 16.04 or Linux Mint 18.3 ? What about CFX ? Any ANSYS Software which won't work on there?




Wait, CentOS has issues with dual EPYCs? Which kind of issues? Also along with most recent kernels? (When I updated the kernel on my CentOS VM I had kernel 5.something)

I have only tested Fluent, DesignModeler and Meshing. Here is my simple guide for linux Mint 18.3. Works with the most recent version of ANSYS as well.



https://www.cfd-online.com/Forums/fl...8-3-howto.html


The problem was with the combination of motherboard (H11DSi) and GPU (GTX 1660 I believe). No problems with the EPYCs. You should not need to update the kernel in CentOS, at least not for EPYC support.



https://community.amd.com/thread/226108

Well, interesting. We luckily also have a H11DSi and a GTX 1660 Ti. Sounds great..:rolleyes:
Sounds like CentOS is not viable for us then...:(
Or can this be solved by simply changing the GPU?



Ok in another thread flotus stated that at least kernel 4.12 should be used. But seems like I misunderstood this and its more a thing of performance rather than hardware support.
https://www.cfd-online.com/Forums/ha...-openfoam.html

Linux Kernel versions: our cluster with Epyc 7401 CPUs runs CentOS 7 with its default 3.x Kernel. Can't remember which version exactly. And it runs just fine. The CPUs are recognized correctly, and performance with everything I tested so far seems to be on par with different distributions and newer Kernel versions. I guess CentOS added compatibility for newer hardware into their old kernels?

FWIW, I recently replaced the Nvidia GPU in my personal workstation with a cheap RX570 8GB. Was tired of crossing my fingers every time I typed zypper up. :rolleyes:
And so far, I think the AMD driver experience is just superior to Nvidia.

Perhaps (I have ordered RX580 with our new EPYC machines, since it is reasonably old and should work well with the open source drivers).


Anyways, I did not put any effort into trying to solve our problem. I am sure it can be done with some searching.



I am not so sure about that. The CentOS 3.10 kernel cannot be compared to a vanilla 3.10 kernel. It has evolved well past the initial release. As you noted though, you can always go with a newer kernel to test this.


Anyways, if your colleagues are unwilling to learn Linux, then the choice is easy, go with Windows. My testing with the WSL (Ubuntu 18.04) in Windows shows almost no performance penalty when running OpenFOAM, and you also get access to Spaceclaim in the Windows version of ANSYS.

Hi jakethejake,

Did you install Windows 10 Pro on your system? If so, does Windows work properly?

Yes, at least it works (W10 Pro). Did not test the "..for Workstations"-Edition yet, but it seems that isnt necessary. We did the installation with the MediaCreationTool1903 (-> W10 May Update). Regarding CFD-performance we are still evaluating how much it's lacks in compare to a native Linux distribution installed. As an interim result I can say that it is worse than I expected.
https://www.cfd-online.com/Forums/ha...rdware-12.html

Well, I always thought it is the other way around. E.g. Simbelmynë, in the thread above you stated that you would always prefer NVIDIA GPUs when it comes to Linux distr. This might changed over the time!?

@jakethejake


You are correct. My impression of Nvidia has changed. Many years ago Nvidia was better than Radeon (less bad is perhaps a more fitting description). Today I think it is the other way around for most cases. The Open-Source AMD drivers are often better than the proprietary AMD drivers and they are less likely to break on a kernel update. I only purchase AMD cards (older versions to ensure that drivers are up to the task) with our recent workstations.


With that being said. We have several machines at work running Nvidia + CentOS 7 without any problems. Sure it takes a few minutes to install the drivers compared to Ubuntu where you just click a button, but when done it works. At home I have an Ubuntu box with a gtx 1080 for both work and playing games (many games work flawlessly under lutris or steam). With other releases (mostly rolling) of Linux I have had tons of problems with the Nvidia drivers during updates.