CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

MPIRun How many processors

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By gschaider

Reply
 
LinkBack Thread Tools Display Modes
Old   October 28, 2008, 10:28
Default Hi.. I've just recently got ac
  #1
New Member
 
Mark Herskind
Join Date: Mar 2009
Posts: 8
Rep Power: 7
markh83 is on a distinguished road
Hi.. I've just recently got access to a quite big cluster.. Containing a total of about 1440 cores.. But I've noticed, that more cores do not always increase the calculation speed.. As an example.. I have a grid, containing 106250 cells.. I've tried three different runs, one on 100 cores, one on 20 cores and one on an other workstation with just 1 core.. The estimated calculation speeds is:
1 core: 163 hours
20 cores: 10 hours
100 cores: 18 hours


I've also noticed, that the seems to be an acceleration in simulation speed from the start, eg if you make an estimate just when the calculations have begun, it will be higher than an estimate after an hour.
The simulations is to simulate 500 seconds, and the the simulations has so far reached (They were not started at the same time):
1 core: 450 secs simulated
20 cores: 19 secs simulated
100 cores: 23 secs simulated

So my conclussion is, that since the 20 cores seems to be faster than both 100 cores and 1 core, there must be an optimum number of cores to use..

Is there any literature available on this subject, or does any of you guys have a rule of thumb to use, as to how many cores should be utilized?
markh83 is offline   Reply With Quote

Old   October 28, 2008, 12:11
Default Hi Mark! There seem to be s
  #2
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,783
Rep Power: 37
gschaider will become famous soon enoughgschaider will become famous soon enough
Hi Mark!

There seem to be several things at work here. Amongst these http://en.wikipedia.org/wiki/Amdahl%27s_law
If you say that the parallel part of the computation is approximately proportional to the volume of the domain of one CPU and the serial part proportional to the processor boundaries then it is easy to see that the ratio of parallel/serial gets even more unfavourable for higher processor numbers which would explain why the computation times are getting worse (instead of approaching a fixed value like Ahmdal's law would predict)

The optimum in you case is definitly N<20 but it depends on your problem. Depending on whom you ask the rule of thumb is that you should have at least 50k (10k) cells per processor to get reasonable speedups

Bernhard
rajibroy likes this.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   October 28, 2008, 13:56
Default Are they all single-cores, dua
  #3
Senior Member
 
Anonymous
Join Date: Mar 2009
Posts: 110
Rep Power: 7
madad2005 is on a distinguished road
Are they all single-cores, dual-cores, quad-cores and what make are they? This could be due to memory bandwidth limitation. Actually, you have a grid with 100k cells. Using 100 cores means each core gets 1000 cells. I'm sure I read that Openfoam shows no benefits from parallel processing when the cell count per node is under 10k. I have a quad-core at home and I found that, for a 100k mesh running on 2 cores was actually slightly faster than running on all 4.
madad2005 is offline   Reply With Quote

Old   October 29, 2008, 04:08
Default Thank you guys for the quick a
  #4
New Member
 
Mark Herskind
Join Date: Mar 2009
Posts: 8
Rep Power: 7
markh83 is on a distinguished road
Thank you guys for the quick answers..

The simulation on 100 cores is still running, and has simulated 204 seconds 18,5hours.. The simulation finished (500s) on 20 cores in 16,85hours..

I'm an intern in the company that bought the cluster, so I really don't know that much about it.. All i know, is that is has approx. 1440 cores, and that each node has 4 cores.. So approx. 360 nodes..

As I read it, I should take 10k-50k cells/core (This must depend on flow complexity), and then try to start a couple of simulations using eg 10k cells/core, 30k cells/core and 50k cells/core, and then do an estimate after a while, and then kill the two slowest..
markh83 is offline   Reply With Quote

Old   October 29, 2008, 09:37
Default Sounds good, Mark. Let us know
  #5
Senior Member
 
Anonymous
Join Date: Mar 2009
Posts: 110
Rep Power: 7
madad2005 is on a distinguished road
Sounds good, Mark. Let us know how you get on.
madad2005 is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with mpirun duderino OpenFOAM 17 February 5, 2010 13:00
Make mpirun use 4 out of 8 possible cpus christian_kunkelmann OpenFOAM Running, Solving & CFD 8 September 15, 2008 16:58
MPIrun... AVL FIRE Peter Main CFD Forum 1 September 4, 2008 04:41
Intel Fortran debugger with mpirun agg Main CFD Forum 2 March 21, 2007 18:43
Lamboot and mpirun r2d2 OpenFOAM Running, Solving & CFD 2 January 10, 2006 11:31


All times are GMT -4. The time now is 04:13.