Suggestions for AMD powered mini Servers for HPC

manojkg.me · August 16, 2015, 13:59

Hi all,
We do CFD simulation based on particle methods,(SPH, MLPG kind of). We use custom codes to develop and run the simulation and compare it with commercial solvers like ANSYS Fluent and CFX. We are a small research group and currently own 3 Dell T1700 workstations, all powered by Intel processors. (2*I7 8c and 1*Xenon 8c) with 32GB memory each.
We would like to upgrade performance of the simulation. We are looking at AMD powered mini servers.
Regarding simulations:

For commercial solvers: Cells anywhere between 3 to 16 million.(ANSYS and CFX)
For in house solvers: 3 to 12 million particles.(Have openmp capability).

Can somebody please suggest me a base configuration to start with?

Micael · September 7, 2015, 09:14

First suggestion is to go for Intel, not AMD. I see you have i7 8-core, that should be 5960X. From that point, one can only ends up disappointed with AMD (for CFD utilisation I mean)

How many HPC licence do you have for ANSYS?

manojkg.me · September 7, 2015, 09:42

Dear Micael,
Thank you for the reply. I work in a university. I'm not sure of the number of licences we have at out HPC facility.
I work in a lab and the HPC facility is common for all users, my professor want to get a high performance server to address the computational demands of our team. We are basically involved in developing custom programs for CFD simulations specific to the study of gravity waves using particle methods(like SPH). ANSYS is used to compare and benchmark these results.
This is a rapidly growing team of researchers and collaborators. So we are looking at long term upgradation option rather than short term benefits in cost.

I hope this answers the question. If the reply is confusing, please let me know.

Regards
Gangadharan, Manoj Kumar

Micael · September 7, 2015, 09:53

Dear Gangadharan,
First thing would be to estimate how many core you need for the cluster. You can probably do that using your workstations.

As an example, if you figure out that you want the cluster to run large cases 10 times faster than the workstation, then you would aim in the 100-core range.

manojkg.me · September 7, 2015, 10:33

Dear Micael,
Thank you. We were actually looking at machines, that are called mini servers. My professor have been told by someone about this machine and I was asked to come up with a feasible solution. Speed is an important factor. I will discuss with research group and update you regarding the same. Since we code based on openMPI, I think more number of cores can do wonders.

Will you be able to provide me an estimate for a 100 core machine?

Regards
Gangadharan, Manoj Kumar

Micael · September 9, 2015, 15:10

My opinion would be to consider a 8 node cluster using the processor Xeon E5-2643 v3. (2 x 6-core per node). Interconnect would be FDR InfiniBand (Mellanox SX6005 switch and MCX353A-FCBT adapter as example). 64 GB of ECC DDR4 2133MHZ per node (8 x 8GB module).

Such a system has 96 core and should be around 55 000 USD.

manojkg.me · September 10, 2015, 01:52

Dear Micael
Thank you. I will put forth your suggestion as one of the options. I don't really know how much the price is going to be in India. I will update you regarding the same. Mean while, I will start my base research with this model and combination. Thank you.

Regards
Manoj

August 16, 2015, 13:59	Suggestions for AMD powered mini Servers for HPC	#1
manojkg.me New Member Manoj Kumar Gangadharan Join Date: Apr 2015 Location: Delaware, USA Posts: 8 Rep Power: 11	Hi all, We do CFD simulation based on particle methods,(SPH, MLPG kind of). We use custom codes to develop and run the simulation and compare it with commercial solvers like ANSYS Fluent and CFX. We are a small research group and currently own 3 Dell T1700 workstations, all powered by Intel processors. (2I7 8c and 1Xenon 8c) with 32GB memory each. We would like to upgrade performance of the simulation. We are looking at AMD powered mini servers. Regarding simulations: For commercial solvers: Cells anywhere between 3 to 16 million.(ANSYS and CFX) For in house solvers: 3 to 12 million particles.(Have openmp capability). Can somebody please suggest me a base configuration to start with?

September 7, 2015, 09:14		#2
Micael Senior Member Micael Join Date: Mar 2009 Location: Canada Posts: 156 Rep Power: 18	First suggestion is to go for Intel, not AMD. I see you have i7 8-core, that should be 5960X. From that point, one can only ends up disappointed with AMD (for CFD utilisation I mean) How many HPC licence do you have for ANSYS?

September 7, 2015, 09:42		#3
manojkg.me New Member Manoj Kumar Gangadharan Join Date: Apr 2015 Location: Delaware, USA Posts: 8 Rep Power: 11	Dear Micael, Thank you for the reply. I work in a university. I'm not sure of the number of licences we have at out HPC facility. I work in a lab and the HPC facility is common for all users, my professor want to get a high performance server to address the computational demands of our team. We are basically involved in developing custom programs for CFD simulations specific to the study of gravity waves using particle methods(like SPH). ANSYS is used to compare and benchmark these results. This is a rapidly growing team of researchers and collaborators. So we are looking at long term upgradation option rather than short term benefits in cost. I hope this answers the question. If the reply is confusing, please let me know. Regards Gangadharan, Manoj Kumar

September 7, 2015, 09:53		#4
Micael Senior Member Micael Join Date: Mar 2009 Location: Canada Posts: 156 Rep Power: 18	Dear Gangadharan, First thing would be to estimate how many core you need for the cluster. You can probably do that using your workstations. As an example, if you figure out that you want the cluster to run large cases 10 times faster than the workstation, then you would aim in the 100-core range.

September 7, 2015, 10:33		#5
manojkg.me New Member Manoj Kumar Gangadharan Join Date: Apr 2015 Location: Delaware, USA Posts: 8 Rep Power: 11	Dear Micael, Thank you. We were actually looking at machines, that are called mini servers. My professor have been told by someone about this machine and I was asked to come up with a feasible solution. Speed is an important factor. I will discuss with research group and update you regarding the same. Since we code based on openMPI, I think more number of cores can do wonders. Will you be able to provide me an estimate for a 100 core machine? Regards Gangadharan, Manoj Kumar

September 9, 2015, 15:10		#6
Micael Senior Member Micael Join Date: Mar 2009 Location: Canada Posts: 156 Rep Power: 18	My opinion would be to consider a 8 node cluster using the processor Xeon E5-2643 v3. (2 x 6-core per node). Interconnect would be FDR InfiniBand (Mellanox SX6005 switch and MCX353A-FCBT adapter as example). 64 GB of ECC DDR4 2133MHZ per node (8 x 8GB module). Such a system has 96 core and should be around 55 000 USD.

September 10, 2015, 01:52		#7
manojkg.me New Member Manoj Kumar Gangadharan Join Date: Apr 2015 Location: Delaware, USA Posts: 8 Rep Power: 11	Dear Micael Thank you. I will put forth your suggestion as one of the options. I don't really know how much the price is going to be in India. I will update you regarding the same. Mean while, I will start my base research with this model and combination. Thank you. Regards Manoj