CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

Parallel speedup Fluent Gigabit Ethernet Myrinet etc

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 20, 2007, 09:58
Default Hi all, I am running OpenFo
  #1
Senior Member
 
Thomas Jung
Join Date: Mar 2009
Posts: 100
Rep Power: 8
tehache is on a distinguished road
Hi all,

I am running OpenFoam on a cluster of 2Core/2Cpu AMD Opteron 275 (i.e. 4 cores per blade), which are connected by simple 1GB Ethernet.
Parallel efficiency is almost perfect if running all on 1 blade, but goes down drastically if I have processes on different blades.
Fluent, however, scales well even across blades. And yes, I have been experimenting with float transfer and scheduled transfer, made it only worse.
So I have 2 questions:

1.) Any idea what Fluent is making different?

2.) Does someboy have experience if Myrinet or Inifiband as inter-blade connection would change this?

Thanks for any comment!

regards,

Thomas
tehache is offline   Reply With Quote

Old   July 21, 2007, 02:53
Default Have you tried MPI/GAMMA inste
  #2
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 698
Rep Power: 12
msrinath80 is on a distinguished road
Have you tried MPI/GAMMA[1] instead of LAM/OpenMPI?

[1] http://www.disi.unige.it/project/gamma/mpigamma/
msrinath80 is offline   Reply With Quote

Old   July 23, 2007, 14:57
Default We're getting very good behavi
  #3
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
We're getting very good behaviour for GAMMA or MPI/GAMMA (GAMMA with MPI compatibility layer). The latest version GAMMA is very stable.

- install GAMMA (not trivial)
- choose 'export WM_MPLIB=GAMMA' in your .OpenFOAM-xxx/bashrc and recompile Pstream:

http://www.cfd-online.com/cgi-bin/Op...1346#POST11346
mattijs is offline   Reply With Quote

Old   July 24, 2007, 08:52
Default Thank you ! Sounds good. I
  #4
Senior Member
 
Thomas Jung
Join Date: Mar 2009
Posts: 100
Rep Power: 8
tehache is on a distinguished road
Thank you !

Sounds good.
I am just a little scared because the gamma people specify the required kernel exactly as 2.6.18.1
We are running 2.6.16.13 (smp), and I doubt I can convince my admin to change the running kernel on the cluster.

Any experience if it works with other kernel versions too?

Thanks again!

Thomas
tehache is offline   Reply With Quote

Old   July 24, 2007, 15:38
Default It isn't too hard to patch a n
  #5
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
It isn't too hard to patch a not-too-different kernel. From what I remember the only change is in the network driver (e1000.c) where you have to disable the network interface so GAMMA has exclusive rights to it. 'diff'ing the GAMMA supplied version with your kernel's e1000.c should tell you what to change.
mattijs is offline   Reply With Quote

Old   September 7, 2009, 05:51
Default
  #6
Senior Member
 
BastiL
Join Date: Mar 2009
Posts: 471
Rep Power: 11
bastil is on a distinguished road
Sorry for ading to this very old thread. However I have simular considerations. I use the same cpus but with a myrinet interconnect. I get nearly no speedup (less then 10%) from OpenFOAM when switching from 16 (4 blades x 2 cpus x 2 cores) to 32 (8 blades x 2 cpus x 2 cores) cores. This is totally different for FLUENT. This leads to the fact that FLUENT outperforms OpenFOAM on 32 cores by a factor of 2 whereas on 16 cpus speed is nearly the same. I have no idea what's going on there.
bastil is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
parallel Fluent on two PCs connected via ethernet! jack FLUENT 0 December 2, 2008 12:55
OpenFOAM 141 parallel results infiniband vs gigabit vs SMP msrinath80 OpenFOAM Running, Solving & CFD 10 November 30, 2007 19:11
fluent journal files - speedup your postprocessing warlock FLUENT 0 March 9, 2006 00:36
cluster - parallel speedup George Main CFD Forum 3 March 29, 2005 11:32
cluster - parallel speedup George FLUENT 0 March 25, 2005 06:54


All times are GMT -4. The time now is 17:17.