CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (http://www.cfd-online.com/Forums/openfoam-solving/)
-   -   OpenMPI performance (http://www.cfd-online.com/Forums/openfoam-solving/59313-openmpi-performance.html)

vega November 6, 2007 22:09

I evaluated parallel performan
 
I evaluated parallel performance of simpleFoam using 1.24 Mill model on intel 2-processor machine (2.8 GHz, 3.5GB Memory, Feisty-Fawn Ubuntu OS). Here are numbers:

...................... ExecutionTime ...... ClockTime
Single CPU:........... 28114 s ....... 28119 s
2-CPUs: .............. . 25712 s ...... 25765 s

It does not look very good. Is model too small to
see the real difference? I had used openMPI for parallel runs. I used the same simpleFoam executable for both simulations. Does a parallel executable need to be compiled differently for openMPI?

thanks

--
Rajneesh

mamaly60 November 6, 2007 23:28

it is a little strange that it
 
it is a little strange that it took a long time near 8 hour for one iteration. I test simpleFoam for 2,5 mill and it took 210 s. I think your memory is not enough for your problem and it use paging or swaping to do that. paging cause it take to long. you can test it. when you see "top" list, after a while your memory become near 100% and after that you can see ksawpd is added to the top list.

vega November 6, 2007 23:37

Mou, Time is for 300 iterat
 
Mou,

Time is for 300 iterations. Sorry, should have mentioned in the post.

tehache November 7, 2007 06:39

I observed the same with OpenM
 
I observed the same with OpenMPI and OpenFOAM 1.4.1- almost no speedup, where before, with mpich and OpenFOAM1.4, I had linear speedup up to 4 processors.
I did not dare to post this because I thought I would have made a mistake - but if Rajneesh observes the same?

Still investigating, however...

Thomas

msrinath80 November 7, 2007 21:53

Rajneesh: Q1) Is this a dua
 
Rajneesh:

Q1) Is this a dual-core or a dual-cpu or a hyperthreaded machine?

If this is a dual-core or a hyperthreaded offering, then I am not surprised in the least. These configurations at best give you 1.2 speedup for 2 processes. This is because the memory bandwidth (which is the biggest bottleneck for CFD) is shared between the cores.

Q2) When the parallel run is in progress did you check the online 'top' monitor which reports memory and CPU usage. Were both processes using 99% CPU all the time?

vega November 7, 2007 23:03

Srinath, I am quite positiv
 
Srinath,

I am quite positive that it is a dual-core (not dual CPU machine. (Its a HP pavilion media center PC, about 2 year old). So then these results are to be expected.

CPU % was fluctuating between 99-100 for both CPUs all the time. Memory usage was only ~1.2GB (out of 3.5GB total).

My concern was if I had to create parallel executable by compiling (wmake_ing) with some openMPI specific options. Looks like default executable is appropriate for both the serial and parallel runs.

--
Rajneesh

msrinath80 November 7, 2007 23:39

My concern was if I had to cre
 
My concern was if I had to create parallel executable by compiling (wmake_ing) with some openMPI specific options. Looks like default executable is appropriate for both the serial and parallel runs.

That's because abstraction in OpenFOAM is so cool. They have a single Pstream library that needs to be modified if one chooses to use a different MPI offering. I've noticed that one also needs to rebuild the parallel utilities (decomposePar, reconstructPar et al.) if they change MPI libraries. And of course all solvers are compiled in such a way that they work without any rebuilds for both serial and parallel modes.

vega November 26, 2007 01:15

I re-ran the same case on netw
 
I re-ran the same case on network of a Laptop and Desktop. Laptop has Centos Linux while the Desktop has Ubuntu.OpenMPI is used for parallel run. Here are numbers for 300 iterations.
.................................. ............ ExecutionTime ...... ClockTime
Single CPU (desktop):.................... 28114 s ........ 28119 s
2 CPU (desktop + Laptop) ............ 14049 s ........ 14052 s

Numbers are actually even better than linear speedup!

--
Rajneesh

msrinath80 November 26, 2007 05:08

What interconnect did you use?
 
What interconnect did you use?

msrinath80 November 26, 2007 05:09

Also, speedup makes sense only
 
Also, speedup makes sense only if the Desktop and Laptop feature the exact same processor.

vega November 26, 2007 13:32

I had Laptop and Desktop conne
 
I had Laptop and Desktop connected through a Belkin router. I can get exact specs but its ~5 year old.

Should'nt time for parallel run be atmost twice (for this setup of 2 CPUS) of the serial run time on the slowest of the two machines for linear speedup?

vega November 26, 2007 13:33

I should have written half ins
 
I should have written half instead of twice in the previous post.

msrinath80 November 26, 2007 14:05

Well, if the machines are not
 
Well, if the machines are not the same speed, you can of course do the speedup calculation using the lowest serial time. That would project the worst possible scenario and thus a more definite speedup estimate.

asaha November 27, 2007 02:28

It would great if you could al
 
It would great if you could all post briefly about setting up parallel run with OpenMPI on network of computers. I have struggled so far, with no success.


All times are GMT -4. The time now is 10:52.