CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

CPU+GPU workstation for chess

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   November 1, 2022, 17:14
Default CPU+GPU workstation for chess
  #1
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
Hi guys, can you guys help me with my build:

CPU: 2x AMD Epyc 7B12 (64 core, 2.25Ghz, 240 TDP)
Mobo: SUPERMICRO MBD-H12DSi-N6-O
Case: Fractal Design Meshify 2 XL
nvMe: SAMSUNG 980 PRO SSD 2TB PCIe NVMe Gen 4,MZ-V8P2T0B


1) ??? Coolers: (air preferred) 2x Noctua NH-U14S TR4-SP3 ???
Is the orientation correct?
TDP is not very clear will it handle it?
what else would be good?

2) ??? PSU: (will run a RTX 4090, or 2x RTX 3080's if it fits)
ATX 3.0?
1600w?
silver? titanium?
which company?

(computer will run 24/7, with both processors and videocard (s))

3) VRM fan : for air cooling do i need a fan?

thanks
tturgut@aol.com
tturgut is offline   Reply With Quote

Old   November 1, 2022, 17:57
Default
  #2
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
1) ??? Coolers: (air preferred) 2x Noctua NH-U14S TR4-SP3 ???
Is the orientation correct?
TDP is not very clear will it handle it?
what else would be good?
Noctua NH-U14S TR4-SP3 is pretty much the best air cooler here. No need to look for alternatives, they are perfectly capable of handling any CPU for this socket.

Quote:
2) ??? PSU: (will run a RTX 4090, or 2x RTX 3080's if it fits)
ATX 3.0?
1600w?
silver? titanium?
which company?
ATX standards can be ignored. As long as you don't buy something that came out more than 10 years ago.
80Plus ratings only describe the efficiency of the PSU. Personally, I don't go lower than "Platinum" these days. With your PC running 24/7 with these components, an efficient PSU is probably a good investment.
As an example, Seasonic Prime PX-1300 1300W should work. Or any other quality PSU with similar or higher wattage.
Maybe look in the 1500W range if you really want to run dual RTX 3090.

Quote:
3) VRM fan : for air cooling do i need a fan?
I think that's already discussed in detail in this thread.
With the components you intend to run on air cooling, there is no way around filling ALL possible fan mounts in this case and running these fans at fairly high rpm. That should create enough airflow to keep the CPU VMRs in check.
Chris2337 likes this.
flotus1 is offline   Reply With Quote

Old   November 1, 2022, 18:06
Default
  #3
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
Thanks!

The orientation of the noctuas look wrong.
Am I mistaken?
tturgut is offline   Reply With Quote

Old   November 1, 2022, 18:57
Default
  #4
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Wrong in which sense?
On the H12DSI, their airflow is from bottom to top. That's fine in a workstation case that has exhaust fans at the top.
Server cases would be a different story.
flotus1 is offline   Reply With Quote

Old   November 1, 2022, 19:36
Default
  #5
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
But the bottom will be saturated with the videocard. may not have enough space for good airflow.

So its not a problem to have a bottom to top (instead of front to back) air flow?
tturgut is offline   Reply With Quote

Old   November 2, 2022, 04:09
Default
  #6
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
That's more a general problem with air cooling. Exactly the reason why you need lots of case fans: creating enough air flow to keep the temperature inside the case low.

With a front-to-back orientation, the rear CPU gets the pre-heated air from the front CPU. Not necessarily a problem, all servers work this way. You just need higher rpm on the CPU coolers to offset that.
But...
front-to-back: air exhausts mostly through a single 140mm fan in the back
bottom-to-top: air exhausts mostly through the top, where you can have 2-3 140mm fans

With something in the order of 1000W of heat dumped into the case, you need a lot of airflow. Hence my recommendation to fill all fan mounts in this case:
- 2x140mm front intake
- 2x140mm bottom intake
- 1x140mm back exhaust
- 2-3x140mm top exhaust (the one towards the front of the case might not help too much)
flotus1 is offline   Reply With Quote

Old   November 2, 2022, 05:07
Default
  #7
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
:-)
thanks.
1) Do you see any problems with my case?
Fractal Design Meshify 2 XL

2) 7V12 (2.40 Ghz, 280 TDP) ?
7B12 (2.25 Ghz, 240 TDP) ?

3) I think the 2 back 140 mm fans at the bottom will be blocked bu the GPU.
I was wondering if I should try to find a closed loop RTX 4090?

like this one: https://www.newegg.com/msi-geforce-r...82E16814137759

4) PSU:
https://www.newegg.com/corsair-hx150...quicklink=true

5) The other option is to put 2x3080's on air:
https://www.amazon.com/dp/B09QKL8XG5...v_ov_lig_dp_it

Not sure if it will fit/ cool enough to run 24/7 (I will use this for chess analysis)

thanks again!
tturgut is offline   Reply With Quote

Old   November 2, 2022, 05:14
Default
  #8
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
https://www.supermicro.com/en/produc...ard/H12DSi-NT6

this should be ok for 2x280 TDP CPU's correct? ( your messages above were saying 240 for H11DSI.
tturgut is offline   Reply With Quote

Old   November 2, 2022, 06:36
Default
  #9
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Nah, the Meshify 2 XL is a good case.
The power supply you found will do the job.

Don't overthink stuff like "air flow paths" or "blocking airflow". That's something you can't avoid in a DIY computer, and it's not a big deal as long as you have enough powerful case fans.

The H12DSI officially supports CPUs with 280W TDP.
Whether the OEM-specific CPUs you have in mind will run on it, you should double-check before buying. It will probably work, but I just don't know for certain.

Whether you want air or water cooling comes down to how silent you need this thing. If you don't mind having 8x140mm fans running at 1200-1500 rpm, air cooling will work just fine. Water cooling will allow you to get quieter cooling. It's up to your preference.

Single vs. dual GPU...
I don't know much about the software you intend to run.
I'd imagine you will end up configuring the CPUs in NPS=1 mode, leaving you with 2 NUMA nodes total. In which case it might be beneficial to have one GPU for each NUMA node? If multi-GPU is even a thing? No idea, you would have to ask some experts for the chess program you intend to use.
Blower-style GPUs would also help, especially if you need more than 1. They are loud, but they don't dump the heat into the case. Instead, they exhaust it right out the back. There are RTX 3080s with this type of cooler, but no 4090s that I am aware of.
flotus1 is offline   Reply With Quote

Old   November 3, 2022, 08:17
Default
  #10
Member
 
Matt
Join Date: May 2011
Posts: 43
Rep Power: 14
the_phew is on a distinguished road
Quote:
Originally Posted by flotus1 View Post
Motherboard
Supermicros H11DSi boards officially support CPUs with up to 240W TDP. You might be able to run higher TDP CPUs, but only with special care towards cooling the CPU VRMs. I really would not recommend it.
Even if you remain within the specs, the VRMs will be difficult to cool in a workstation case.
We run a CFD cluster using those motherboards w/ EPYC 7542s in workstation cases (albeit in a cold server room), with water-cooled CPUs. My experiences:
-Water cooling was overkill; even under 100% load 24/7/365, CPU temps haven't even cracked 40C. At least the water cooling allows the CPUs to boost up near the max single-core boost on all 32 cores
-VRM temps haven't been an issue for us, despite the VRMs getting virtually no airflow. I think the highest I've seen is 70 C
-Memory temps have been our biggest cooling challenge; the DIMMs in the top slots creep above 80C

I only run LBM-VLES CFD on this cluster, so that may tax the memory more relative to the CPU/VRMs vs. Navier-Stokes solvers.

If I had to do it again, I'd go 4U rackmount just for better memory cooling, with better VRM cooling being icing on the cake.
the_phew is offline   Reply With Quote

Old   November 3, 2022, 15:20
Default
  #11
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
The problem with memory overheating also correlates with memory density, i.e. how much capacity each module has. 16GB DDR4 DIMMs are fairly easy to cool.
32GB and 64GB RDIMMs can overheat easily with the high amount of ICs crammed onto the PCB.
Water cooling for these CPUs is a double-edged sword. You don't need a lot of case airflow to keep the CPUs cool. Airflow which the memory modules would really benefit from. Plus the Noctua CPU coolers for SP3 tend to get a bit of cooling on the memory modules as well.
I agree that water cooling for the CPUs is not necessary. It just allows for quieter cooling.

Have you tried adding a bunch of case fans and and running them at max rpm? Should not be a noise problem in a server room. Which case are you using?
flotus1 is offline   Reply With Quote

Old   November 3, 2022, 15:24
Default GPU addition:
  #12
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
thanks guys!!

I ordered 2x 7v12's from ebay for the Supermicro motherboard.
I will try aircooloing with Noctu N14 SP3.
PSU Corsair 1500w Hxi

GPU question:
Now, I need either 1 RTX 4090 or 2x RTX 3080's. (almost same total cuda cores)

AiO RTX 4090 is on black market and ridiculously expensive.
Turbo RTX 3080's may work, but will be too loud.

Would 2x 3080's with 3 fans each be OK? how thick are they, will they fit?
(blower style would fit, but too loud i guess)

what kind of strong GPU system can i run 24/7 without running into overheating problem? (not watercooled)
tturgut is offline   Reply With Quote

Old   November 3, 2022, 15:28
Default
  #13
Member
 
Matt
Join Date: May 2011
Posts: 43
Rep Power: 14
the_phew is on a distinguished road
Quote:
Originally Posted by flotus1 View Post
Have you tried adding a bunch of case fans and and running them at max rpm? Should not be a noise problem in a server room. Which case are you using?
It's a proprietary case (with awful airflow for memory cooling, IMHO) from Titan Computers:


The reservoir blocks the top intake fan, so the top DIMM banks get nothing (especially the aft, top bank). I could zip-tie some case fans somewhere to impingement-cool the DIMMs, but I haven't seen any evidence of memory throttling or whatever. These are 64 GB 3200 MHz DIMMs.
the_phew is offline   Reply With Quote

Old   November 3, 2022, 15:32
Default
  #14
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
Fractal Design Meshify 2 XL

https://www.gamersnexus.net/hwreview...ge-case-review

i like the case.. (though I am not an expert in this)
tturgut is offline   Reply With Quote

Old   November 3, 2022, 15:41
Default
  #15
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
i am building this as a chess computer . I play correspondence chess as a hobby. :-)

it needs to run both alpha beta programs (CPU) and alpha zero programs (GPU)
Wont play any video games on it. :-(

It will need to run 24/7. so good cooling is very important.

I built a 2x E5 2699v3 system (overclocked at 3.25 Ghz) a couple years ago, and used Noctua's for the CPU's with very good result.

I wonder if i can put 2x 3080's and get away with it without causing the temps to go high...
tturgut is offline   Reply With Quote

Old   November 4, 2022, 06:45
Default
  #16
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by the_phew View Post
It's a proprietary case (with awful airflow for memory cooling, IMHO) from Titan Computers:
The reservoir blocks the top intake fan, so the top DIMM banks get nothing (especially the aft, top bank). I could zip-tie some case fans somewhere to impingement-cool the DIMMs, but I haven't seen any evidence of memory throttling or whatever. These are 64 GB 3200 MHz DIMMs.
Oh wow, that's rough. Seems like they managed to engineer a solution where no amount of fan speed would really help with memory cooling.

Quote:
Originally Posted by tturgut View Post
thanks guys!!
GPU question:
Now, I need either 1 RTX 4090 or 2x RTX 3080's. (almost same total cuda cores)

AiO RTX 4090 is on black market and ridiculously expensive.
Turbo RTX 3080's may work, but will be too loud.

Would 2x 3080's with 3 fans each be OK? how thick are they, will they fit?
(blower style would fit, but too loud i guess)

what kind of strong GPU system can i run 24/7 without running into overheating problem? (not watercooled)
Just to manage your expectations a bit: the system you are building won't be silent. 1000W can be air-cooled, but not without some fan noise.
Now everyone has a different threshold for what they consider a "silent enough" PC. Mine is pretty low, maybe yours is higher and this will be perfectly acceptable for you.

Looking at the block diagram in the motherboard manual, you probably want to use the top-most PCIe slot for the first GPU, and the second from the bottom for a second GPU. This way, each one is connected to a different CPU.
That leaves 4 slots for the first GPU. So a 3080 with the most common triple-slot axial fan cooler still has some room to breathe, which is good.
You just need to make sure there is enough clearance towards the bottom of the case to fit the second GPU.
If there isn't enough space to leave one slot empty between each GPU, blower-style coolers are the way to go. The radial fans skew towards higher static pressure, which is crucial to overcome the flow restriction. And the benefit of exhausting the heat right away instead of dumping it into the case still remains.

Long story short: there is no way get quiet air cooling for a multi-GPU setup that also has two CPUs with 280W TDP. Whether it is quiet enough for you depends on personal preference.

Edit: I moved this discussion to a separate thread

Last edited by flotus1; November 4, 2022 at 08:43.
flotus1 is offline   Reply With Quote

Old   November 8, 2022, 20:11
Default
  #17
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
can you guys also help me:
I bought this motherboard and also 2 EPYC 7002 processors.
i am planning using this computer for chess analysis.
I have a Fractal meshify XL2 case.
https://www.fractal-design.com/produ...empered-glass/
It will run a RTX 4090 for chess analysis also. (not for gaming)

But I would like to use this not as a server if possible.

1) Where do the front I/O's go exactly (not in the manual)
https://www.supermicro.com/manuals/m...0/MNL-2363.pdf

2) Are there any USB 2.0, USB3.0 or USB 3.1 connections directly on the motherboard (not on the rear I/O)
I found 1 only: JUSB1

3) Sound (HD Audio)? is there a connection?
4) i can deactivate the VGA I guess

thanks!
tturgut is offline   Reply With Quote

Old   November 9, 2022, 03:17
Default
  #18
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
The H12DSi is a server motherboard. So the answer to most questions about front I/O (and rear I/O) is unfortunately: no
It's got one USB 3.0 header for the front (JUSB1), and that's pretty much it aside from power/reset/etc.
It has no onboard audio, hence no front panel audio connectors either.
If you need any of those things, you have to add it via PCIe adapter cards.

Not quite sure what you mean with "deactivate the VGA"
In the bios, you can choose which graphics is used as a priority: offboard (a GPU in the PCIe slot) or onboard (the one connected to the VGA port on the rear). And I'm pretty sure there is a jumper somewhere that changes the default setting here.
What do you want to achieve?

By the way, I moved your post here.
You don't have to rely on finding similar threads to ask your questions. That's usually considered bad etiquette.
If you have a new question, just open a new thread. Or in this case, since the question still revolves around the same computer, add the question to your own thread.
flotus1 is offline   Reply With Quote

Old   November 9, 2022, 13:04
Default
  #19
New Member
 
Tansel
Join Date: Nov 2022
Posts: 21
Rep Power: 3
tturgut is on a distinguished road
TPM module?
is this necessary for Windows 11 pro?

https://www.supermicro.com/en/produc...oard/H12DSi-N6

there are 3 versions of it.. do you have idea which one?

thanks
tturgut is offline   Reply With Quote

Old   November 9, 2022, 13:19
Default
  #20
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,398
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Sorry, no idea about that.
flotus1 is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
General recommendations for CFD hardware [WIP] flotus1 Hardware 19 February 29, 2024 12:48
GPU acceleration in Ansys Fluent flotus1 Hardware 63 May 12, 2023 02:48
CPU for Flow3d mik_urb Hardware 4 December 4, 2022 22:06
Superlinear speedup in OpenFOAM 13 msrinath80 OpenFOAM Running, Solving & CFD 18 March 3, 2015 05:36
Star cd es-ice solver error ernarasimman STAR-CD 2 September 12, 2014 00:01


All times are GMT -4. The time now is 05:36.