Computer suggestion with the most processors possible
 

I plan to run STAR-CCM+ on a 4-CPU 32-core workstation. It seems Dell and HP only have 2-CPU 24-core. Your suggestions will be appreciated. Thanks!

Intel 4-core machines are very expensive, and require a special (very expensive) version of the Xeon (e7) processor. Either go for two networked 16-core machines, or if you really insist on so many cores in one box, 4-socket AMD. The 4-socket AMD systems use the same processors as the 2-socket ones, and have 4 memory channels per socket, just like the Xeon e5 systems. In CFD the memory speed counts for a great deal (this has been discussed to death on this forum), most often more than the raw CPU performance.

Thanks for your answer. I think I should go with an AMD machine. Do you have any suggestions regarding brand name vs non-brand name?

I am looking at Puget Systems which has competitive pricing compared to Dell and HP. I am not sure about the quality of product and reliability of their service.

p.s., Somebody told me that certain AMD processor models had heat issues before. Do you have any idea about this?

There's always a bit of a risk with a non name brand, but I've had good success with white box systems before. AFAIK, the AMD heat issues were about 12 years ago! What is true these days is that they seem to be less interested (or less able to?) in trying to compete head-on with Intel for outright performance.

Thanks for your information. I have narrowed down to the following two systems with similar price.

Intel Computer:
Motherboard Supermicro X9QRI-F+
CPU 4 x Intel Xeon E5-4610 V2 2.3GHz Eight Core 16MB 95W
Ram Kingston 64GB DDR3-1600 REG ECC (16x4GB)
Video Card NVIDIA GeForce GTX 780 Ti 3GB

AMD Computer:
Motherboard Supermicro H8QGi-F
CPU 4 x AMD Opteron (G34) 6380 16-Core 2.5GHz 115W
Ram Kingston 64GB DDR3-1600 REG ECC (16x4GB)
Video Card NVIDIA GeForce GTX 780 Ti 3GB

Since Star-CCM+ has flat HPC license, I will be able to use all available cores. I understand the performance gain v.s. number of cores is not linear. However, the second system has 64 cores, compared to 32 cores on the Intel system. Which one will be faster?

Greetings to all!

@tjtx: Thanks for providing the specs of each machine, it makes it a lot easier to do some math on this.

From a GHz only point of view and taking into account the lithography each processor uses, here's what I get:

• E5-4610 V2 is done with a lithography of 22nm and runs at 2.3 base clock, 2.7 with turbo. This usually equates to the average of 2.5 GHz when using all cores, therefore it's 8 cores at 2.5GHz = 20 GHz of cumulative pure power.
• AMD Opteron 6380 is done with a lithography of 32 nm runs at 2.5 GHz, but a bit of searching indicates that it can run at 2.8 GHz with all cores. This equates to 16 cores at 2.8 GHz = 44.8 GHz of cumulative power.

Now factoring in the lithography of each CPU and using 32nm as the reference:

• E5-4610 V2: 32/22 = 1.45(45) scale up factor, which means that 20GHz times 1.45(45) equates to 29.09(09) GHz.
• AMD Opteron 6380: 32/32 = 1, i.e. it's still 44.8 GHz.

Therefore, theoretically, the machine using the AMD Opteron 6380 CPUs is roughly 4/3 more powerful than the Intel E5-4610 V2 CPUs.


But then there is a problem that is hard to account for:

• In the AMD CPUs, there are 16 cores per socket accessing the same 4 banks of RAM at the same time.
• In the Intel CPUs, there are only 8 cores per socket accessing the same 4 banks of RAM at the same time.

And since both are using 1600MHz RAM modules, it means that the Intel processors more likely have a more orderly access to the RAM than the AMD processors. And as CapSizer indicated, memory accesses are extremely important for CFD, therefore there is a risk that in a worst case scenario, having twice the core count might reduce to half the performance of the AMD processors, which would (in an equivalent way) bring them down from 44.8 to 22.4 GHz of cumulative power.

On the heating point of view, both solutions will likely have very good cooling solutions (it's a professional workstation after all). Nonetheless, these AMD processors are indexed as using 115W each and the Intel ones 95W each... so, yeah, the AMD ones are a bit hotter ;)


Although I am a fan of AMD processors (and personally I've never owned a machine with an Intel CPU), I have to say that the Intel solution is the one that seems more likely to perform better, since it has a smaller core count for each RAM group and has a higher (when scaled) CPU frequency per core, which might pay off is situations where a simulation case might not be as parallelizable.
On the other hand, if the machine will also run other kinds of software that take better advantage of a higher core count (and less memory accesses), then the AMD solution seems definitely the way to go.


I would suggest that if you plan on buying the machine(s) as an HPC solution, you might want to ask your vendor if they can perform a benchmark on each set-up, to ensure which one is the best purchase.

Best regards,
Bruno

Yes, got to agree with Bruno there. At the same time, you should be aware that a "fat" workstation like this is seldom these days regarded as the most cost effective way of getting a lot of computing power. A small cluster may be a better option, especially given the type of licence that you have available. But then you will need fast networking as well. The 4-socket machine will be less hassle.

We tried Star-CCM on:

HP BL660c Gen8
4x Intel(R) Xeon(R) CPU E5-4620 0 @ 2.20GHz, 8/8 cores; 16 threads
8x 8 GB DDR3 memory 1333 Mhz
4x 10 GB Ethernet
2x 146 GB 15 Krpm

With windows as operating system (I know, I also prefer Linux).

It did not perform at all!! It was at least 5 times slower than a cluster setup we were using (same number of cores).

Most probable cause was that this machine is not designed as HPC machine and therefore the CPU were not efficient in accessing the memory. We didn't do a lot of testing and the people from HP eventually said it was not meant for HPC, althought they recommended it at first.

I think a 4x AMD will work better because the communication between the CPU's is different.

Sorry I do not have more information.
Just be sure to do some benchmarking before you buy!

Wow, that's a lot of valuable information. Thanks so much for all the answers. I now have learned what to look at before purchasing a computing workhorse. I will request the seller do some benchmark testing.

One more question, how much does hard drive speed affect CFD performance? For example SSD vs 7200rpm.

It won't help CFD solver performance at all, as it solves in core (RAM) and doesn't access the hard drive. It would make loading results files and opening meshes or other hard drive operations much faster though.

If you are doing transient simulations and writing a lot of time steps it may help as well, but usually we only write a very small fraction of the actual iterations.