CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Installation (https://www.cfd-online.com/Forums/openfoam-installation/)
-   -   Parallel speedup Fluent Gigabit Ethernet Myrinet etc (https://www.cfd-online.com/Forums/openfoam-installation/57516-parallel-speedup-fluent-gigabit-ethernet-myrinet-etc.html)

tehache July 20, 2007 10:58

Hi all, I am running OpenFo
 
Hi all,

I am running OpenFoam on a cluster of 2Core/2Cpu AMD Opteron 275 (i.e. 4 cores per blade), which are connected by simple 1GB Ethernet.
Parallel efficiency is almost perfect if running all on 1 blade, but goes down drastically if I have processes on different blades.
Fluent, however, scales well even across blades. And yes, I have been experimenting with float transfer and scheduled transfer, made it only worse.
So I have 2 questions:

1.) Any idea what Fluent is making different?

2.) Does someboy have experience if Myrinet or Inifiband as inter-blade connection would change this?

Thanks for any comment!

regards,

Thomas

msrinath80 July 21, 2007 03:53

Have you tried MPI/GAMMA inste
 
Have you tried MPI/GAMMA[1] instead of LAM/OpenMPI?

[1] http://www.disi.unige.it/project/gamma/mpigamma/

mattijs July 23, 2007 15:57

We're getting very good behavi
 
We're getting very good behaviour for GAMMA or MPI/GAMMA (GAMMA with MPI compatibility layer). The latest version GAMMA is very stable.

- install GAMMA (not trivial)
- choose 'export WM_MPLIB=GAMMA' in your .OpenFOAM-xxx/bashrc and recompile Pstream:

http://www.cfd-online.com/cgi-bin/Op...1346#POST11346

tehache July 24, 2007 09:52

Thank you ! Sounds good. I
 
Thank you !

Sounds good.
I am just a little scared because the gamma people specify the required kernel exactly as 2.6.18.1
We are running 2.6.16.13 (smp), and I doubt I can convince my admin to change the running kernel on the cluster.

Any experience if it works with other kernel versions too?

Thanks again!

Thomas

mattijs July 24, 2007 16:38

It isn't too hard to patch a n
 
It isn't too hard to patch a not-too-different kernel. From what I remember the only change is in the network driver (e1000.c) where you have to disable the network interface so GAMMA has exclusive rights to it. 'diff'ing the GAMMA supplied version with your kernel's e1000.c should tell you what to change.

bastil September 7, 2009 06:51

Sorry for ading to this very old thread. However I have simular considerations. I use the same cpus but with a myrinet interconnect. I get nearly no speedup (less then 10%) from OpenFOAM when switching from 16 (4 blades x 2 cpus x 2 cores) to 32 (8 blades x 2 cpus x 2 cores) cores. This is totally different for FLUENT. This leads to the fact that FLUENT outperforms OpenFOAM on 32 cores by a factor of 2 whereas on 16 cpus speed is nearly the same. I have no idea what's going on there.


All times are GMT -4. The time now is 17:42.