HP DL580 servers: How to get the most out of my 160 vCPUs!

mcneilyo · October 24, 2016, 02:34

Hi,
After many months of research and about 18 months blowing money on EC2 instances, I took a dive and bought two refurbished HP DL580 servers (for an excellent price I might add).

I've been running many FORTRAN executable in parallel (MODFLOW) to undertake Monte Carlo uncertainty analysis. I'm seeing some very slow runtimes - much much slower than anticipated.

Here's my setup:
2x HP DL580 servers, with the following in each:
2x Hypervisor Windows 10 pro virtual machines
4x Intel® Xeon® Processor E7-4870 (30M Cache, 2.40 GHz, 6.40 GT/s Intel® QPI)
256GB RAM (32x 8GB PC3-10600R)
P410i raid controller
8x 480GB SSD (stripe setup)

Running a single model takes about 1 hour, which I assume = 2.8GHz with boosting. When run 1 model per physical core, I'm seeing speeds of about 4 hours. And then 1 model per virtual core about 6 hour run times.

Where's my bottle-necking coming from? These days, shouldn't I be getting 2.4GHz from each vCPU? Or is it somehow split? Either way, I should be seeing faster run times with 1 model per physical core.

I've tried setting affinity per model, and it did little to speed up the runs. It can't be harddrive speeds, as I'm writing data at about 1 MB/s per model. Each model is consuming 0.5GB RAM.

Any advice greatly appreciated!

mcneilyo · October 24, 2016, 02:43

I should add that I saw pretty similar speeds on the EC2 virtual machines. e.g. r3.8xlarge

To save you a google, in total I have 4 x virtual machines. Each machine has 20 physical cores allocated to it (40 vcpus).

October 24, 2016, 02:34	HP DL580 servers: How to get the most out of my 160 vCPUs!	#1
mcneilyo New Member Neil Join Date: Oct 2016 Posts: 2 Rep Power: 0	Hi, After many months of research and about 18 months blowing money on EC2 instances, I took a dive and bought two refurbished HP DL580 servers (for an excellent price I might add). I've been running many FORTRAN executable in parallel (MODFLOW) to undertake Monte Carlo uncertainty analysis. I'm seeing some very slow runtimes - much much slower than anticipated. Here's my setup: 2x HP DL580 servers, with the following in each: 2x Hypervisor Windows 10 pro virtual machines 4x Intel® Xeon® Processor E7-4870 (30M Cache, 2.40 GHz, 6.40 GT/s Intel® QPI) 256GB RAM (32x 8GB PC3-10600R) P410i raid controller 8x 480GB SSD (stripe setup) Running a single model takes about 1 hour, which I assume = 2.8GHz with boosting. When run 1 model per physical core, I'm seeing speeds of about 4 hours. And then 1 model per virtual core about 6 hour run times. Where's my bottle-necking coming from? These days, shouldn't I be getting 2.4GHz from each vCPU? Or is it somehow split? Either way, I should be seeing faster run times with 1 model per physical core. I've tried setting affinity per model, and it did little to speed up the runs. It can't be harddrive speeds, as I'm writing data at about 1 MB/s per model. Each model is consuming 0.5GB RAM. Any advice greatly appreciated! Last edited by mcneilyo; October 24, 2016 at 03:14. Reason: wording

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
New Servers Launched for CFD Online Website	pete	Site News & Announcements	0	June 12, 2015 06:09
Setup processing servers	chattphotos	STAR-CCM+	0	October 10, 2013 20:33
[Gmsh] gmsh 2.6.0 conversion to OpenFoam 160	rosswin	OpenFOAM Meshing & Mesh Conversion	0	March 5, 2013 07:34
Making a Cluster with servers at different locations?	Mobz	OpenFOAM Programming & Development	1	February 27, 2013 17:16
Multiblocks	Rider	OpenFOAM	16	April 12, 2012 03:00

October 24, 2016, 02:43		#2
mcneilyo New Member Neil Join Date: Oct 2016 Posts: 2 Rep Power: 0	I should add that I saw pretty similar speeds on the EC2 virtual machines. e.g. r3.8xlarge To save you a google, in total I have 4 x virtual machines. Each machine has 20 physical cores allocated to it (40 vcpus).