CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (http://www.cfd-online.com/Forums/openfoam-solving/)
-   -   Can not run OpenFOAM in parallel in clusters, help! (http://www.cfd-online.com/Forums/openfoam-solving/115996-can-not-run-openfoam-parallel-clusters-help.html)

ripperjack April 10, 2013 16:49

Can not run OpenFOAM in parallel in clusters, help!
 
Hi guys,

I am trying to run openfoam in a cluster.
I use the following script to run icoFoam in parallel using 16 CPUs. I can get the simulation running, but it seems that the openfoam does not run in "real parallel", instead, the case is run by 16 times.
Code:

#! /bin/csh
#BSUB -W 5
#BSUB -n 16
#BSUB -R "em64t span[ptile=1]"
source /XXXX/foam171.csh
blockMesh
reconstructPar
mpiexec_hydra icoFoam
reconstructPar
#BSUB -o /foamo.%J
#BUSB -e /foame.%J

I met this problem before. That is, if you run openfoam in a local parallel case, for example, using
Code:

mpirun -n 16 icoFoam
it will run the case 16 times instead of running in parallel with 16 CPUs. You should add -parallel option as
Code:

mpirun -n 16 icoFoam -parallel
which will run with 16 CPUs in parallel.
Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??
Many thanks!

sail April 11, 2013 13:54

Quote:

Originally Posted by ripperjack (Post 419722)
Hi guys,

I am trying to run openfoam in a cluster.
I use the following script to run icoFoam in parallel using 16 CPUs. I can get the simulation running, but it seems that the openfoam does not run in "real parallel", instead, the case is run by 16 times.
Code:

#! /bin/csh
#BSUB -W 5
#BSUB -n 16
#BSUB -R "em64t span[ptile=1]"
source /XXXX/foam171.csh
blockMesh
reconstructPar
mpiexec_hydra icoFoam
reconstructPar
#BSUB -o /foamo.%J
#BUSB -e /foame.%J

I met this problem before. That is, if you run openfoam in a local parallel case, for example, using
Code:

mpirun -n 16 icoFoam
it will run the case 16 times instead of running in parallel with 16 CPUs. You should add -parallel option as
Code:

mpirun -n 16 icoFoam -parallel
which will run with 16 CPUs in parallel.
Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??
Many thanks!

http://wiki.mpich.org/mpich/index.ph...rocess_Manager

have you tried the -n 16 switch?

mpiexec_hydra -n 16 icoFoam

I'm not sure it will work though

have you tried the man or help file?


mpiexec_hydra --help

bigphil April 14, 2013 15:23

Hi,

Just my thoughts:

Quote:

Originally Posted by ripperjack (Post 419722)
Now I am using the cluster, I can not use "mpirun" command, I must use "mpiexec_hydra" instead, so how can I add something like "-parallel" into mpiexec_hydra command, which will let openfoam perform correct parallel calculation??

As far as I know, the "-parallel" option is for the OpenFOAM application to let it know that it is running is parallel, and is not an mpirun option.

So have you tried:
Code:

mpiexec_hydra -n 16 icoFoam -parallel
Philip

ripperjack April 15, 2013 12:39

Quote:

Originally Posted by sail (Post 419958)
http://wiki.mpich.org/mpich/index.ph...rocess_Manager

have you tried the -n 16 switch?

mpiexec_hydra -n 16 icoFoam

I'm not sure it will work though

have you tried the man or help file?


mpiexec_hydra --help

Hi Sail,

Thanks for your reply! I have try "mpiexec_hydra -n 16 icoFoam", I got the following error message:
Code:

which: invalid option -- n
Usage: which [options] [--] programname [...]
Options: --version, -[vV] Print version and exit successfully.
        --help,          Print this help and exit successfully.
        --skip-dot      Skip directories in PATH that start with a dot.
        --skip-tilde    Skip directories in PATH that start with a tilde.
        --show-dot      Don't expand a dot to current directory in output.
        --show-tilde    Output a tilde for HOME directory for non-root.
        --tty-only      Stop processing options on the right if not on tty.
        --all, -a        Print all matches in PATH, not just the first
        --read-alias, -i Read list of aliases from stdin.
        --skip-alias    Ignore option --read-alias; don't read stdin.
        --read-functions Read shell functions from stdin.
        --skip-functions Ignore option --read-functions; don't read stdin.
Not Found: (2 mpiexec_hydra)

It seems that n is not an default option for mpiexec_hydra. And n is given by the script "#BSUB -n 16".

ripperjack April 15, 2013 12:42

Quote:

Originally Posted by bigphil (Post 420491)
Hi,

Just my thoughts:



As far as I know, the "-parallel" option is for the OpenFOAM application to let it know that it is running is parallel, and is not an mpirun option.

So have you tried:
Code:

mpiexec_hydra -n 16 icoFoam -parallel
Philip

Hi Philip,

I have tried it but it does not work. But you gave me a good suggestion, I am not trying to add -parallel option to icoFoam, instead of mpiexec. But still not know where should I add it.

mayank.dce2k7 May 6, 2014 15:25

Hi ripperjack,

Did you find any solution to your problem? I am facing the same problem.

Regards,
Mayank.


All times are GMT -4. The time now is 12:10.