[thandor]

Hi everyone,

I'm looking to build a simple home workstation to work on some side projects with open source solvers and continuing to work on unfinished codes. Some use cases could be to characterize aircraft polars (with simple 20-30M meshes) and run optimization processes (airfoils, etc). I'll also use lower fidelity codes like panel methods.

I've set up a budget of 1k€ since this is just for side projects, I only have a laptop outside of work. I'm very open to second hand stuff.

1. Which software do you intend to use? Open source solvers (OF, SU2) and my own codes. I'll also use low-fidelity panel methods.
2. Are you limited by license constraints? I.e. does your software license only allow you to run on N threads? Not at all.
3. What type of simulations do you want to run? And what's the maximum cell count? Mainly external aerodynamics, at most 20-30M cells. For the most part I'd be interested in running several simulations at a time, with a lower cell count, for optimization.
4. If there is a budget, how high is it? Around 1k€ preferably
5. What kind of setting are you in? Hobbyist? Student? Academic research? Engineer? I'm an engineer, but this would be for playing around and learning a bit.
6. Where can you source your new computer? Buying a complete package from a large OEM? Assemble it yourself from parts? Are used parts an option? Anything is an option, including aliexpress/ebay.
7. Which part of the world are you from? It's cool if you don't want to tell, but since prices and availability vary depending on the region, this can sometimes be relevant. Particularly if it's not North America or Europe. Europe.
8. Anything else that people should know to help you better?

From what I've seen from lurking around this forum, a good budget option could be a LGA2011-3 dual socket option like:

• CPU: 2x Xeon E5 2697A v4
• RAM: 8x 16GB DDR4 2400 ECC sticks
• Mobo: something from aliexpress or preferably from a reputable brand on ebay
• Coolers: 2x 2011-3 compatible coolers (something like a Deepcool Gammaxx)

Plus the usual case, storage, PSU... This would be around 700-800€ with current second hand prices from what I've seen.

If you could give me some opinions/recommendations on this, I would greatly appreciate it

Thank you very much

[thandor]

I've come across a deal on ebay for a pre-owned dual socket Dell LGA2011-3 workstation with a single, lower tier CPU than the E5 2697A v4s. However it already includes a case, coolers, PSU, some basic storage and motherboard. If I'm to replace the CPU for 2x 2697A v4s and add 128GB of memory, it would go for a total of around 470€, which is I think unbeatable and I think I'll go for that option

[flotus1]

That indeed sounds pretty good in terms of price/performance.
Maybe worth checking beforehand that this OEM workstation supports the 2697A v4. It is technically a refresh, and comes with high TDP.

[wkernkamp]

Originally Posted by flotus1

That indeed sounds pretty good in terms of price/performance.
Maybe worth checking beforehand that this OEM workstation supports the 2697A v4. It is technically a refresh, and comes with high TDP.

It is handy that the workstation comes with a working cpu. This means that you can upgrade the bios using the old cpu before installing the new ones. You can go in the bios version history and get an idea if at some stage the bios was upgraded to allow v4 high core count cpus. (Still install latest bios.) I had to do this on a GigaByte MD80-F34 server to make it do v4.


You can check the Dell website for the BIOS versions. They should all be there including revision history. On Dell you press F12 for boot selection. Then you will see a selection of boot devices, but also "Upgrade BIOS". This will let you select a file (example) R7610A17.exe. Select the file and it will do the install automatically. No need to make bootable usb.

[thandor]

Thank you very much guys. I checked the Dell website like you suggested and there is indeed a BIOS update that enables compatibility with all E5 1xxx and 2xxx v4 Xeons, so I think I should be good.

Also apparently the PSU is 685W, so IMO it should be enough for the two CPUs and the basic included GPU.

[wkernkamp]

Yes, I think you should have enough power. If I remember correctly, two 2697Av4, at full power cause the system to use somewhere between 360 and 400 W. So there is enough power to also feed a GPU.

[naveedgt]

Any opinions on the hp ml150 gen9? I think it's limiting to 120W CPUs so the 2697a V4 is out. But seems to be a reasonable system to get dual cpus and 16 ram slots.

[wkernkamp]

You could get dual E5-2683v4. They are platform approved by HP. For CFD, the performance is almost as good as the E5-2697Av4. (I have dual systems of both).

[CFDfan]

One thing to watch out if the Dual CPU motherboard would have octal memory bandwidth, meaning that the MB should support quad memory bandwidth for each CPU. Not all dual CPU motherboards have octal memory bandwidth. I found this the hard way and had to buy another dual CPU MB from Aliexpress.

[naveedgt]

Originally Posted by CFDfan

One thing to watch out if the Dual CPU motherboard would have octal memory bandwidth, meaning that the MB should support quad memory bandwidth for each CPU. Not all dual CPU motherboards have octal memory bandwidth. I found this the hard way and had to buy another dual CPU MB from Aliexpress.

Which motherboard did you end up with?

[CFDfan]

the MB links don't exist anymore

[moving-boundaries]

I am in a similar situation as the thread author and wanted to get any opinions on Xeon Gold 6100 CPUs (6 channels, DDR4-2666). It seems like you currently can get, for example, 2x Xeon Gold 6140 + 12x16GB DDR4-2666 for just $720 https://pcserverandparts.com/build-y...-2-processors/, which theoretically should give about 240 GiB/s memory bandwith. And the benchmarks I managed to find on this forum do indicate of a very decent performance of these CPUs

Originally Posted by naffrancois

Hello, OpenFoam v1806 linux binaries:


2x Intel Xeon Gold 6148 +12x8Go 2666MHz single rank
# cores Wall time (s):
------------------------
10 105.82
20 64.77
40 49.28

Originally Posted by Hrushi

2 x Intel Xeon Gold 6136, 12 * 16 GB DDR4 2666MHz, Ubuntu 16.04 LTS,
Linux OF-7820-Tower 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

OpenFOAM-5.x

I have commented following lines in controlDict.

#include streamlines
#include wallBoundedStreamlines

Here are the results.

# cores   Wall time (s):
------------------------
1             874.54
2             463.34
4             205.23
6             137.95
8             106.04
12             74.63
16             61.09
20             53.26
24             49.17

Hrushikesh

Originally Posted by fedez91

2*Intel Xeon Gold 6140
96gb (12*8gb) 2666 MHz

#cores   Wall time (s):
------------------------
1            981.72
2            488.92
4            217.97
6            146.36
8            113.38
12           85.34
16           68.23
20           60.27
24           55.94
28           52.5
32           50.76
36           49.87

Improvement seems to become insignificant after 24 cores... any idea why this may happen?

Are there any other configurations under $2k that can actually beat this? Or maybe similar many core performance, but better single core?

[wkernkamp]

Originally Posted by moving-boundaries

I am in a similar situation as the thread author and wanted to get any opinions on Xeon Gold 6100 CPUs (6 channels, DDR4-2666). It seems like you currently can get, for example, 2x Xeon Gold 6140 + 12x16GB DDR4-2666 for just $720
.....
Are there any other configurations under $2k that can actually beat this? Or maybe similar many core performance, but better single core?

You could get a quad cpu workstation like this one:
https://www.ebay.nl/itm/115698853691...Bk9SR4iNzdb8Yg

I have a DL560 Gen8 (previous generation) that does the benchmark in 40 seconds. So the gen 9 should do at least as well, and the price is half your budget.

[CFDfan]

Originally Posted by wkernkamp

You could get a quad cpu workstation like this one:
https://www.ebay.nl/itm/115698853691...Bk9SR4iNzdb8Yg

I have a DL560 Gen8 (previous generation) that does the benchmark in 40 seconds. So the gen 9 should do at least as well, and the price is half your budget.

Very impressive performance for that price. I haven't worked with this kind of servers for CFD and have a couple of questions:
1. Can it run on Windows?
2. Can a GPU be added (low profile video card for example), i.e. are there free PCIe slots? If yes are these PCIe 3.0 or PCIe 4.0.
3. Is it very noisy?
4. What is the total memory bandwidth (8ch or 16ch).

Thank you.

[gnwt4a]

Originally Posted by wkernkamp

You could get a quad cpu workstation like this one:
https://www.ebay.nl/itm/115698853691...Bk9SR4iNzdb8Yg

I have a DL560 Gen8 (previous generation) that does the benchmark in 40 seconds. So the gen 9 should do at least as well, and the price is half your budget.

if it could fit 4 x mi60s at full width, it would have been a stunning dns machine. i would buy it - and possible a second one.