[ripperjack]

Hi guys,

I am trying to run openfoam in a cluster.
I use the following script to run icoFoam in parallel using 16 CPUs. I can get the simulation running, but it seems that the openfoam does not run in "real parallel", instead, the case is run by 16 times.

#! /bin/csh
#BSUB -W 5
#BSUB -n 16
#BSUB -R "em64t span[ptile=1]" 
source /XXXX/foam171.csh
blockMesh
reconstructPar
mpiexec_hydra icoFoam
reconstructPar
#BSUB -o /foamo.%J
#BUSB -e /foame.%J

I met this problem before. That is, if you run openfoam in a local parallel case, for example, using

mpirun -n 16 icoFoam

it will run the case 16 times instead of running in parallel with 16 CPUs. You should add -parallel option as

mpirun -n 16 icoFoam -parallel

which will run with 16 CPUs in parallel.
Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??
Many thanks!

[sail]

Originally Posted by ripperjack

Hi guys,

I am trying to run openfoam in a cluster.
I use the following script to run icoFoam in parallel using 16 CPUs. I can get the simulation running, but it seems that the openfoam does not run in "real parallel", instead, the case is run by 16 times.

#! /bin/csh
#BSUB -W 5
#BSUB -n 16
#BSUB -R "em64t span[ptile=1]" 
source /XXXX/foam171.csh
blockMesh
reconstructPar
mpiexec_hydra icoFoam
reconstructPar
#BSUB -o /foamo.%J
#BUSB -e /foame.%J

I met this problem before. That is, if you run openfoam in a local parallel case, for example, using

mpirun -n 16 icoFoam

it will run the case 16 times instead of running in parallel with 16 CPUs. You should add -parallel option as

mpirun -n 16 icoFoam -parallel

which will run with 16 CPUs in parallel.
Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??
Many thanks!

http://wiki.mpich.org/mpich/index.ph...rocess_Manager

have you tried the -n 16 switch?

mpiexec_hydra -n 16 icoFoam

I'm not sure it will work though

have you tried the man or help file?


mpiexec_hydra --help

[bigphil]

Hi,

Just my thoughts:

Originally Posted by ripperjack

Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??

As far as I know, the "-parallel" option is for the OpenFOAM application to let it know that it is running is parallel, and is not an mpirun option.

So have you tried:

mpiexec_hydra -n 16 icoFoam -parallel

Philip

[ripperjack]

Originally Posted by sail

http://wiki.mpich.org/mpich/index.ph...rocess_Manager

have you tried the -n 16 switch?

mpiexec_hydra -n 16 icoFoam

I'm not sure it will work though

have you tried the man or help file?


mpiexec_hydra --help

Hi Sail,

Thanks for your reply! I have try "mpiexec_hydra -n 16 icoFoam", I got the following error message:

which: invalid option -- n
Usage: which [options] [--] programname [...]
Options: --version, -[vV] Print version and exit successfully.
         --help,          Print this help and exit successfully.
         --skip-dot       Skip directories in PATH that start with a dot.
         --skip-tilde     Skip directories in PATH that start with a tilde.
         --show-dot       Don't expand a dot to current directory in output.
         --show-tilde     Output a tilde for HOME directory for non-root.
         --tty-only       Stop processing options on the right if not on tty.
         --all, -a        Print all matches in PATH, not just the first
         --read-alias, -i Read list of aliases from stdin.
         --skip-alias     Ignore option --read-alias; don't read stdin.
         --read-functions Read shell functions from stdin.
         --skip-functions Ignore option --read-functions; don't read stdin.
Not Found: (2 mpiexec_hydra)

It seems that n is not an default option for mpiexec_hydra. And n is given by the script "#BSUB -n 16".

[ripperjack]

Originally Posted by bigphil

Hi,

Just my thoughts:



As far as I know, the "-parallel" option is for the OpenFOAM application to let it know that it is running is parallel, and is not an mpirun option.

So have you tried:

mpiexec_hydra -n 16 icoFoam -parallel

Philip

Hi Philip,

I have tried it but it does not work. But you gave me a good suggestion, I am not trying to add -parallel option to icoFoam, instead of mpiexec. But still not know where should I add it.

[mayank.dce2k7]

Hi ripperjack,

Did you find any solution to your problem? I am facing the same problem.

Regards,
Mayank.