CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

OpenMPI performance

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   November 6, 2007, 22:09
Default I evaluated parallel performan
  #1
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
I evaluated parallel performance of simpleFoam using 1.24 Mill model on intel 2-processor machine (2.8 GHz, 3.5GB Memory, Feisty-Fawn Ubuntu OS). Here are numbers:

...................... ExecutionTime ...... ClockTime
Single CPU:........... 28114 s ....... 28119 s
2-CPUs: .............. . 25712 s ...... 25765 s

It does not look very good. Is model too small to
see the real difference? I had used openMPI for parallel runs. I used the same simpleFoam executable for both simulations. Does a parallel executable need to be compiled differently for openMPI?

thanks

--
Rajneesh
vega is offline   Reply With Quote

Old   November 6, 2007, 23:28
Default it is a little strange that it
  #2
New Member
 
mou
Join Date: Mar 2009
Posts: 20
Rep Power: 8
mamaly60 is on a distinguished road
it is a little strange that it took a long time near 8 hour for one iteration. I test simpleFoam for 2,5 mill and it took 210 s. I think your memory is not enough for your problem and it use paging or swaping to do that. paging cause it take to long. you can test it. when you see "top" list, after a while your memory become near 100% and after that you can see ksawpd is added to the top list.
mamaly60 is offline   Reply With Quote

Old   November 6, 2007, 23:37
Default Mou, Time is for 300 iterat
  #3
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
Mou,

Time is for 300 iterations. Sorry, should have mentioned in the post.
vega is offline   Reply With Quote

Old   November 7, 2007, 06:39
Default I observed the same with OpenM
  #4
Senior Member
 
Thomas Jung
Join Date: Mar 2009
Posts: 100
Rep Power: 8
tehache is on a distinguished road
I observed the same with OpenMPI and OpenFOAM 1.4.1- almost no speedup, where before, with mpich and OpenFOAM1.4, I had linear speedup up to 4 processors.
I did not dare to post this because I thought I would have made a mistake - but if Rajneesh observes the same?

Still investigating, however...

Thomas
tehache is offline   Reply With Quote

Old   November 7, 2007, 21:53
Default Rajneesh: Q1) Is this a dua
  #5
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
Rajneesh:

Q1) Is this a dual-core or a dual-cpu or a hyperthreaded machine?

If this is a dual-core or a hyperthreaded offering, then I am not surprised in the least. These configurations at best give you 1.2 speedup for 2 processes. This is because the memory bandwidth (which is the biggest bottleneck for CFD) is shared between the cores.

Q2) When the parallel run is in progress did you check the online 'top' monitor which reports memory and CPU usage. Were both processes using 99% CPU all the time?
msrinath80 is offline   Reply With Quote

Old   November 7, 2007, 23:03
Default Srinath, I am quite positiv
  #6
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
Srinath,

I am quite positive that it is a dual-core (not dual CPU machine. (Its a HP pavilion media center PC, about 2 year old). So then these results are to be expected.

CPU % was fluctuating between 99-100 for both CPUs all the time. Memory usage was only ~1.2GB (out of 3.5GB total).

My concern was if I had to create parallel executable by compiling (wmake_ing) with some openMPI specific options. Looks like default executable is appropriate for both the serial and parallel runs.

--
Rajneesh
vega is offline   Reply With Quote

Old   November 7, 2007, 23:39
Default My concern was if I had to cre
  #7
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
My concern was if I had to create parallel executable by compiling (wmake_ing) with some openMPI specific options. Looks like default executable is appropriate for both the serial and parallel runs.

That's because abstraction in OpenFOAM is so cool. They have a single Pstream library that needs to be modified if one chooses to use a different MPI offering. I've noticed that one also needs to rebuild the parallel utilities (decomposePar, reconstructPar et al.) if they change MPI libraries. And of course all solvers are compiled in such a way that they work without any rebuilds for both serial and parallel modes.
msrinath80 is offline   Reply With Quote

Old   November 26, 2007, 01:15
Default I re-ran the same case on netw
  #8
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
I re-ran the same case on network of a Laptop and Desktop. Laptop has Centos Linux while the Desktop has Ubuntu.OpenMPI is used for parallel run. Here are numbers for 300 iterations.
.................................. ............ ExecutionTime ...... ClockTime
Single CPU (desktop):.................... 28114 s ........ 28119 s
2 CPU (desktop + Laptop) ............ 14049 s ........ 14052 s

Numbers are actually even better than linear speedup!

--
Rajneesh
vega is offline   Reply With Quote

Old   November 26, 2007, 05:08
Default What interconnect did you use?
  #9
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
What interconnect did you use?
msrinath80 is offline   Reply With Quote

Old   November 26, 2007, 05:09
Default Also, speedup makes sense only
  #10
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
Also, speedup makes sense only if the Desktop and Laptop feature the exact same processor.
msrinath80 is offline   Reply With Quote

Old   November 26, 2007, 13:32
Default I had Laptop and Desktop conne
  #11
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
I had Laptop and Desktop connected through a Belkin router. I can get exact specs but its ~5 year old.

Should'nt time for parallel run be atmost twice (for this setup of 2 CPUS) of the serial run time on the slowest of the two machines for linear speedup?
vega is offline   Reply With Quote

Old   November 26, 2007, 13:33
Default I should have written half ins
  #12
New Member
 
Rajneesh
Join Date: Mar 2009
Posts: 28
Rep Power: 8
vega is on a distinguished road
I should have written half instead of twice in the previous post.
vega is offline   Reply With Quote

Old   November 26, 2007, 14:05
Default Well, if the machines are not
  #13
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
Well, if the machines are not the same speed, you can of course do the speedup calculation using the lowest serial time. That would project the worst possible scenario and thus a more definite speedup estimate.
msrinath80 is offline   Reply With Quote

Old   November 27, 2007, 02:28
Default It would great if you could al
  #14
Member
 
a a saha
Join Date: Mar 2009
Posts: 67
Rep Power: 8
asaha is on a distinguished road
It would great if you could all post briefly about setting up parallel run with OpenMPI on network of computers. I have struggled so far, with no success.
asaha is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with openmpi mighelone OpenFOAM Running, Solving & CFD 42 July 18, 2012 14:26
SnappyHexMesh in parallel openmpi wikstrom OpenFOAM Bugs 18 November 26, 2008 06:55
Problems using local openmpi stephan OpenFOAM Installation 1 December 5, 2007 19:01
Cant run in parallel on two nodes using OpenMPI CHristofer Main CFD Forum 0 October 26, 2007 09:54
OpenFOAM 14 with OpenMPI 12 fhy OpenFOAM Installation 0 July 12, 2007 18:12


All times are GMT -4. The time now is 16:56.