CFD Online Discussion Forums

CFD Online Discussion Forums (
-   OpenFOAM Running, Solving & CFD (
-   -   Interesting behavior when using LAM without ib rpi on an infiniband cluster OF 141 (

msrinath80 October 2, 2007 16:31

I am running a multphaseInterF
I am running a multphaseInterFoam case (approx 1.1 million hex cells) on an infiniband cluster. Thanks to Henry's help I have switched from quartic interfacecompression scheme to quadratic. This seems to keep the volume fractions stable. Anyways, when I tried to use the following procedure to start a parallel run with 8 processors, I get an error message similar to that reported in [1]:

lamboot -v $PBS_NODEFILE
mpirun -ssi rpi ib -np 8 multiphaseInterFoam . three_fluid_test_4 -parallel

I was just about to give up on getting the inifiniband to work, when I came across this webpage[2]. Following their procedure, I used the following commands to start the run:

lamboot -v -b -ssi boot rsh -ssi rsh_agent rsh $PBS_NODEFILE
mpirun -ssi rpi tcp -np 8 multiphaseInterFoam . three_fluid_test_4 -parallel

and the run started without problems. The lamboot initialization parameters suggest that I am not using the ib module? So I expected degraded performance. However, comparing the time after 7 time steps for a serial run with that of a parallel run, I get a speed up of 8.125818891 which is indeed impressive. I cannot explain why this seems to work. Any comments are appreciated.

PS: The cluster is a combination of 256 processors (i.e. 128 dual opteron boxes NOT dual-core) connected with infiniband.


All times are GMT -4. The time now is 15:49.