HPC server (AMD) to 40 million cells - How cores do I need? Where I can buy it?
 

Dear, we are looking for a server to run AMD approx. 40 million cells. Companies that import these types of equipment nucles ?, How many need to run and climb the 40 million cells ?.
If you know a company that sells these equipment do not hesitate to reply to this post.

regards

Greetings Ferdinand,

Can you please be a more specific on what you're looking for? Because AFAIK, 40 million cells can require anything between 20 and 120 GB of RAM, depending on the specific simulation modelling strategies that are being used.
In addition to that, it really depends on how fast you needs the results for each simulation.

Therefore, if I was to answer based only on what you've indicated, the simplest answer would be this: as long as you buy a machine with 64GB of RAM, it doesn't matter what CPU you're using. Although choosing the wrong CPU or number of CPUs can be the difference between taking 1 month or 20 minutes to run a complete simulation.

Best regards,
Bruno

Bruno Thanks for your answer.
We seek a server where you can run these 40 million cells within a reasonable time (eg 30 minutes) with OpenFOAM.
The mesh used is a hybrid mesh. At first we used an Intel Xeon processor but according to our budget (15 mil dollars) we decided to use AMD processors. AMD would save money but would have more cores. Is the memory ran it could be something of 128GB (32x16 CG)? Is it better to have an Intel Xeon or AMD processor ?.
We are in South America and we want to import the equipment. As described hope you can help me.

Mujchas thanks

I can't advise you where you should buy your machine(s), because that could be considered a form of advertising, which isn't allowed here on the forum.

Nonetheless, I can try and advise you on what CPU models to look for. If you have a look at the AMD website: http://www.amd.com/en-us/products/se...eron/6000/6300 - you'll find a list of all processors in the range you're looking for, including a button on online shops that AMD endorses, since they provide a "Buy Now" button there.
I would say that the Opteron 6380 model is the one more in line with what you're looking for, therefore a single tower with:

• 4x Opteron 6380 processors
• One motherboard that can handle the 4 CPUs
• 128GB of RAM (make sure to ask for 4 modules of RAM for each CPU/socket ... 16 RAM modules, with 8GB per module, or 32 modules with 4GB per module)
• One or two hard-drives

would result in roughly something around 14000 USD, according to one of the shops I've visited from the AMD website.


The other possibility would be to buy more than one machine and build a cluster with InfiniBand interconnect between all of the machines. Using the Opteron 6328 or 6308 CPUs would make more sense for such a cluster, since it could improve the performance in accessing memory.



Nonetheless, if possible, I would strongly suggest that you find a shop in your country that can sell you this hardware, including support for any issues that might occur or what specific machine you should buy. I say this because if anything happens to your machine, you will have to send it back to whom you bought it from for maintenance.


Good luck! Best regards,
Bruno

Do you have any CFD-related benchmark at hand for this CPU?
Just guessing here, but based on what I read so far when using all 16 Cores of the Opteron 6380 calculation speed might be limited by the 4 memory channels.
I would expect an 8-core opteron 6328 to be slightly faster because of the higher clock speed (3.2 GHz vs. 2.5 GHz) even for the single-machine/4-cpu setup.

Do we have some benchmarks with Opteron CPUs against Haswell Xeon CPUs?
Those Opterons are pretty old (introduced in 2011 I think), are they still competitive just because they offer similar memory bandwidth?

Greetings Alex,

Unfortunately I don't have any benchmarks. The closest I know of is this one: http://www.padtinc.com/blog/the-focu...-vs-intel-xeon

If we take into account that each Opteron 6380 costs roughly ~1100 USD, has 16 cores and has a... mmm, I see what you mean. A single Xeon E5-1650 CPU has a bit higher benchmark index at www.cpubenchmark.net than a dual socket Opteron 6380. And a single Xeon E5-1650v3 CPU costs roughly half of an Opteron 6380, and is even ~20% faster than a Xeon E5-1650 (v1).

I guess it would possibly make some sense to buy a server with 4x Opteron 6380 in second hand, instead of buying a new one, for 2000-5000 USD, if Ferdinand still wants to go AMD.

Best regards,
Bruno

Thank very much guys,

Actually we still think in purchasing a servirdor with AMD processor. We do not want to buy a second equipment because money put him university. Anyway thank you very much for your support.

regards

I see no reason to do so, but hey, it is your university's money :rolleyes:
Two dual-socket machines with modern Intel Xeon CPUs would offer more performance per $.
You dont even have to use Infiniband interconnects for two nodes, 10Gigabit-ethernet would be enough.

Thanks Flotus1.
And what would be the characteristics of the two dual-socket machines with modern Intel Xeon CPUs offer more performance for $?. And if so where I can get these machines?

regards

That would be any Xeon E5-2xxx v3 CPU that supports DDR4 2133 MHz.
The fastest choice would be the 10-core E5-2687W v3, but 4 of those might not fit into your budget.
A cheaper alternative would be the 10core E5-2660 v3. The retail price is around 1500$, so 6000$ for four of them.
Add 8 DDR4 2133 DIMMs (registered ECC) per machine. With 8GB per module (128 GB total) that will cost you another 1600$.
Lets add another 2500$ for the mainboards, cases, power-supplies, HDDs, and other small parts.
And another 1500$ for a decent graphics card in one of the machines.

I am sure you will find lots of companies in your region that will be more than happy to put the parts together and sell the system to you for the rest of your budget.

I did some research on quad socket systems and I stumble upon this Openfoam test between a quad Opteron 6344 and quad Xeon 4624L:

https://www.pugetsystems.com/labs/ar...d-Opteron-587/

The conclusion is that core for core at the same speed, the Xeon is 2.4 times faster than the Opteron. I would think it would actually be a bit less than that because the Xeon has more cache per core than the Opteron (2.5MB Xeon vs 1.3MB Opteron). So, this could be half the answer. The other half would be to find an Openfoam speedup test between Ivy Bridge and Haswell.

I suggest AMD Opteron 6308 will be best processor for Computation.
Because L3 carries 4Mb per core and L2 carries 1Mb per core.

Feel free to add benchmarks to back up your hypothesis.

CFD Bechnmarking
 

Dear Friends,

I have done a Computational Fluid Dynamics problem solving benchmark on 3 different computers. The problem solved while benchmarking is a turbomachinery problem. I hope this data is useful to people looking for CFD bechnmark in particular. I am sorry, I do not have HPC access to use all cores. Below are the details tabulated. Thank you.