Hardware for Cfx/openfoam calculations of 30-50 million cells
 

Hi all,

I was hoping for some pointers in purchasing a workstation to use for CFD calculations with CFX and/or OpenFoam. I am just about to start a PhD in wake modelling for wind turbines and will run calculations with actuator disc/line models with around 30-50 million cells or so. The budget for the workstation is the region of 2500-3500$ I have discussed a little with some colleagues but would very much appreciate some advice from more experienced CFD users.

The current model I am considering is:
1 x Intel S5000 Psl Motherboard
2 x Intel Xeon E5420 2.5Gz /12MB / 1333Mhz (total 8 cores)
2 x Kingston DDR2/667Mhz 2x4GB (total 16GB RAM)
1 x Leadtek Quadro FX570 256MB

Please come with comments and suggestions. Anyone know if it would be worth the difference going up the Intel nehalem processor? Is the memory sufficient for 8 cores?

Many thanks
Rkeck

Need more memory
 

You're not going to be doing 50 million cells with 16 GB of RAM. Depending on which solver, which type of grid and turbulence model you are using, 16 GB will maybe give you around 20 million cells.

I think you will see a huge gain if you go with a Nehalem or a newer Opteron. Even on dual core Xeons with the old architecture, the calculation is bottlenecked by the memory bandwidth. I only run 2 threads per quad core chip, because it is actually faster than using all 4!

And as the other reply said, 16gb will not work for 50 million cells. Most of my cases are around 40-60 million and we use 64gb machines.

Thanks for the answers guys!

Good point about the memory and absolutely right, it's about 2-2.5 million cells per 2GB... I had 32GB in the fist spec but cut it down to save money (...and did not fully consider the effects). I might need to rethink the computer spec to stay within budget and go for a workstation for smaller simulations with the capability to mesh big problem in ICEM. For this purpose a Nehalem processor of 4 cores and 12GB of RAM should be enough. This would also mean that I could go for a cheaper motherboard.

Anyone has a comment about the graphic card (1 x Leadtek Quadro FX570 256MB)? Is it needed to go for such a high spec card for the simulation? What would happen if I went with a cheaper "gamer" card with higher memory?

Rkeck

Games cards seem to work quite well
 

My experience of "Games" cards has been quite good. I've had workstations with workstation cards, and I also have workstations with games cards, and I haven't had any trouble with the cheaper ones. Right now my best card is an NVidia 9600GT, and there's been no difficulties with it. To be honest, I also have a compact box with a quad-core AMD Phenom CPU, running on a motherboard with an AMD 780G chipset, and the integrated graphics processor on that is actually good enough to cope with a lot of the Icem and post-processing work. Not all of it, of course, but it works suprisingly well.


As for your computing requirement, look a the possibility of putting together a dual-socket machine with low-end Opteron quad-core CPU's. You can get reasonably affordable motherboards for these that can take 16 memory modules. You would be trading off performance for capacity, but it sounds like you might have to do that.

Why not buy two core i7 920 Computer and make a cluster?
You could have with same money 24 gb of ram, and 8 core.
I don't know how big is the difference to have 8 cores on the same motherboard or on two different MB.

Onda

Small cluster for big memory
 

There's a lot to be said for that suggestion. Multi-socket hardware is easier to manage, but if you really have serious memory demands, a small commodity cluster is hard to beat in terms of value for money. Like many others on this forum, I've worked a cheap ad-hoc cluster like this very hard, and it is surprisingly reliable.

Hi think that there is a big difference in cluster performance between different software.
It depend on the amount of data should go trough net during the analysis.
Software that keep this amount of data low, work very well with cluster and can be used with a 1Gbit net. Other software that have a bigger use of the net, need a faster net connection (fiber optical) and the performance on a cluster are much lower to respect on performance on a multi socket workstation.
The possibility to buy two (or more) pc and make a cluster instead to buy a multi socket workstation depend on the amount of data that should go trough the net.
I heard (but really not sure about that) that CFX work pretty well with a 1Gbit net because the data pass trough the net only at begin and at end of solution step.
I don't know how Openfoam manage a cluster and the amount of data needed to be moved in the net for the analysis.

Small clusters and CFD codes
 

Yes, I've had a similar experience to that with CFX. Coupled solver, does a lot of work per iteration, relatively little traffic per iteration, so parallelises quite well over even a relatively slow network. By contrast, if you use the segregated solver in Fluent, there is a lot more network traffic, and the type of interconnect becomes much more important. Having said that, if you only have two or three nodes for running big models, interconnect is not your biggest problem.