Particle methods: optimum hardware

anon_a · January 10, 2012, 10:44

Hello everybody

In the next months I am going to be running some very heavy particle based simulations.
Could an expert please advise concerning the best selection of hardware for this task?
Thank you for your time.

Notes: The Direct Simulation Monte Carlo (DSMC) method will be used.
I was thinking of using 8-core machines to run each realization and
only perform ensemble averaging by exchanging the macroscopic data
between machines to avoid great communication costs (and also since
the computational effort of particle methods can not be so easily
distributed in a balanced manner).

anon_a · January 13, 2012, 07:47

You probably mean I am being pessimistic about the parallel performance.
It's true that I can't speak about ALL particle methods.
Clearly, if the particle trajectories do not depend on one another, the parallelization will be very efficient. But this is not the case for DSMC. The speedup is not so spectacular here, see e.g.

http://www.accessengineeringlibrary....9905_ar006.pdf
http://citeseerx.ist.psu.edu/mwg-int...?id=4Uob5MvOr7
http://mashhad.academia.edu/EHSANroo...pm.2011.08.036
http://ieeexplore.ieee.org/xpl/freea...&userType=inst

The method I was thinking of using can by no means be called "tips" or "guide for optimum performance". I am just trying this out. Advice is welcome, but I am mostly concerned on hardware performance.

andyj · January 19, 2012, 01:07

You most likely will want a newer server motherboard or newly released high quality desktop motherboard with lots of Ram slots (full size ATX). More ram , more particles.

Many folks are getting good results in Monte Carlo situations with
8 core processors.
About $269 (starting price) for a Magny Cours server processor.
I would look closely at the
AMD Bulldozer 8150 or 8120 for about $200. Might be better deal than a Magny Cours, also a desktop motherboard would be cheaper too.

The 12 core processors are getting the best results,
but the processors cost $1000 each!

Obviously, your software has to installed for parallel use.

Intel Xeons might be faster, but also cost a lot more. Maybe an i7, also cost more.

You might consider a Supermicro or Tyan dual CPU motherboard.
You could start off with one processor and add another if you need it or have the money.

Or go with an 8 core and a high quality newly released motherboard. More cpu cache is better.
I would simply build my own computer, there are only 7 parts.

DMSC (free?) windows programs, resources and info:
http://gab.com.au/

Excellent article on Hardware for DMSC:
Parallel Implementation of the Direct Simulation Monte Carlo
Method For Shared Memory Architecture
Authors- Gao and Thomas E. Schwartzentruber†
Univ of Minn.

256kb of cpu cache allows 4096 particles in this application:

An Analysis of Direct Simulation Monte Carlo
And Its Application to Simulating Supersonic Shockwaves
New Mexico Supercomputing Challenge
Final Report
April 2008

Also:
A Parallel DSMC Implementation Including Adaptive
Mesh Renement And Cut-Cell Algorithms
Da Gao and Thomas E. Schwartzentrubery
Department of Aerospace Engineering and Mechanics, University of Minnesota,

I would assume you are going to use Linux and OpenFoam.

If the hardware you wind up with wont do the job, then go to the Cloud at Amazon or SGI and use their hardware cluster.
I am no expert, but since no one else addressed your question, I thought I might add a few thoughts.

anon_a · January 19, 2012, 12:51

Thanks a lot for the hardware update, that was quite useful!
My budget is around 1000 euros/PC, so I guess I will stick with something not moderately expensive. I will be using a customized version of the (already parallel) OF code.

Before I reach some conclusion, I have two more questions:
- Would you expect that memory access speed could be the bottleneck in particle simulations as it is considered in other CFD applications? (http://www.cfd-online.com/Forums/har...number-gb.html)
- Any suggestions for a site where I can check prices for the equipment you mentioned? If possible, something with a custom PC building applet and preferably in Germany.

andyj · January 19, 2012, 13:14

For Hardware in Germany:
http://www1.hardwareversand.de/home.jsp
Look at PC Components..cpu ect..

andyj · January 19, 2012, 13:50

In German
http://www.youtube.com/watch?v=tyikRYvDFck
Youtube:
PC zusammenbauen für Anfänger 2011 - Intel Teil 1
in English, On Youtube
Newegg TV: How To Build a Computer - Part 1 - Choosing Your Components
For 12 core server cpus:
Magny Cours
http://www.idealo.de/

anon_a · January 19, 2012, 15:02

Thanks again for the help!

andyj · January 19, 2012, 22:54

You might be able to find the newer version of the Opteron (Interlagos) at a reasonable price. Maybe 350 eu. Would be close to the budget limit though, as you need ram, motherboard ect..

January 10, 2012, 10:44	Particle methods: optimum hardware	#1
anon_a Disabled Join Date: Mar 2011 Posts: 174 Rep Power: 15	Hello everybody In the next months I am going to be running some very heavy particle based simulations. Could an expert please advise concerning the best selection of hardware for this task? Thank you for your time. Notes: The Direct Simulation Monte Carlo (DSMC) method will be used. I was thinking of using 8-core machines to run each realization and only perform ensemble averaging by exchanging the macroscopic data between machines to avoid great communication costs (and also since the computational effort of particle methods can not be so easily distributed in a balanced manner).

January 19, 2012, 01:07		#3
andyj Member Andy Jones Join Date: Sep 2010 Location: Columbus, Ohio Posts: 78 Rep Power: 15	You most likely will want a newer server motherboard or newly released high quality desktop motherboard with lots of Ram slots (full size ATX). More ram , more particles. Many folks are getting good results in Monte Carlo situations with 8 core processors. About $269 (starting price) for a Magny Cours server processor. I would look closely at the AMD Bulldozer 8150 or 8120 for about $200. Might be better deal than a Magny Cours, also a desktop motherboard would be cheaper too. The 12 core processors are getting the best results, but the processors cost $1000 each! Obviously, your software has to installed for parallel use. Intel Xeons might be faster, but also cost a lot more. Maybe an i7, also cost more. You might consider a Supermicro or Tyan dual CPU motherboard. You could start off with one processor and add another if you need it or have the money. Or go with an 8 core and a high quality newly released motherboard. More cpu cache is better. I would simply build my own computer, there are only 7 parts. DMSC (free?) windows programs, resources and info: http://gab.com.au/ Excellent article on Hardware for DMSC: Parallel Implementation of the Direct Simulation Monte Carlo Method For Shared Memory Architecture Authors- Gao and Thomas E. Schwartzentruber† Univ of Minn. 256kb of cpu cache allows 4096 particles in this application: An Analysis of Direct Simulation Monte Carlo And Its Application to Simulating Supersonic Shockwaves New Mexico Supercomputing Challenge Final Report April 2008 Also: A Parallel DSMC Implementation Including Adaptive Mesh Renement And Cut-Cell Algorithms Da Gao and Thomas E. Schwartzentrubery Department of Aerospace Engineering and Mechanics, University of Minnesota, I would assume you are going to use Linux and OpenFoam. If the hardware you wind up with wont do the job, then go to the Cloud at Amazon or SGI and use their hardware cluster. I am no expert, but since no one else addressed your question, I thought I might add a few thoughts. Last edited by andyj; January 19, 2012 at 01:43. Reason: long links were not working

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Blood Damage Modelling via Particle Tracking in a Centrifugal Heart Pump	scatman	CFX	7	January 8, 2018 00:59
Problems involved heat transfer inside a particle and its thermal degradation	zhangr	STAR-CCM+	2	August 25, 2011 06:59
Particle Tracking for ion	Jun	CFX	2	August 31, 2010 08:19
DPM UDF particle position using the macro P_POS(p)[i]	dm2747	FLUENT	0	April 17, 2009 01:29
the separated particle temperature in cyclone	mkuswadi	FLUENT	0	December 17, 2003 08:25

January 13, 2012, 07:47		#2
anon_a Disabled Join Date: Mar 2011 Posts: 174 Rep Power: 15	You probably mean I am being pessimistic about the parallel performance. It's true that I can't speak about ALL particle methods. Clearly, if the particle trajectories do not depend on one another, the parallelization will be very efficient. But this is not the case for DSMC. The speedup is not so spectacular here, see e.g. http://www.accessengineeringlibrary....9905_ar006.pdf http://citeseerx.ist.psu.edu/mwg-int...?id=4Uob5MvOr7 http://mashhad.academia.edu/EHSANroo...pm.2011.08.036 http://ieeexplore.ieee.org/xpl/freea...&userType=inst The method I was thinking of using can by no means be called "tips" or "guide for optimum performance". I am just trying this out. Advice is welcome, but I am mostly concerned on hardware performance.

January 19, 2012, 12:51		#4
anon_a Disabled Join Date: Mar 2011 Posts: 174 Rep Power: 15	Thanks a lot for the hardware update, that was quite useful! My budget is around 1000 euros/PC, so I guess I will stick with something not moderately expensive. I will be using a customized version of the (already parallel) OF code. Before I reach some conclusion, I have two more questions: - Would you expect that memory access speed could be the bottleneck in particle simulations as it is considered in other CFD applications? (http://www.cfd-online.com/Forums/har...number-gb.html) - Any suggestions for a site where I can check prices for the equipment you mentioned? If possible, something with a custom PC building applet and preferably in Germany.

January 19, 2012, 13:14		#5
andyj Member Andy Jones Join Date: Sep 2010 Location: Columbus, Ohio Posts: 78 Rep Power: 15	For Hardware in Germany: http://www1.hardwareversand.de/home.jsp Look at PC Components..cpu ect..

January 19, 2012, 13:50		#6
andyj Member Andy Jones Join Date: Sep 2010 Location: Columbus, Ohio Posts: 78 Rep Power: 15	In German http://www.youtube.com/watch?v=tyikRYvDFck Youtube: PC zusammenbauen für Anfänger 2011 - Intel Teil 1 in English, On Youtube Newegg TV: How To Build a Computer - Part 1 - Choosing Your Components For 12 core server cpus: Magny Cours http://www.idealo.de/

January 19, 2012, 15:02		#7
anon_a Disabled Join Date: Mar 2011 Posts: 174 Rep Power: 15	Thanks again for the help!

January 19, 2012, 22:54		#8
andyj Member Andy Jones Join Date: Sep 2010 Location: Columbus, Ohio Posts: 78 Rep Power: 15	You might be able to find the newer version of the Opteron (Interlagos) at a reasonable price. Maybe 350 eu. Would be close to the budget limit though, as you need ram, motherboard ect..