OpenFOAM and MPI cluster using SSH
 

Hi, I am not being able to execute openfoam using mpi in a two node cluster. In one hand when I execute



[code]
mpirun -np 24 -hostfile hosts --use-hwthread-cpus -wdir `pwd` -x FOAM_ETC=/usr/lib/openfoam/openfoam2112/etc pimpleFoam
the second node won't find openfoam's binaries, when I set then using



[code]

mpirun -np 24 -hostfile hosts --use-hwthread-cpus -wdir `pwd` -x LD_LIBRARY_PATH=/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib/sys-openmpi -x PATH=/usr/lib/x86_64-linux-gnu/openmpi/bin:/home/matt/OpenFOAM/matt-v2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/site/2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/bin:/usr/lib/openfoam/openfoam2112/wmake:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -x FOAM_RUN=/home/openfoam/rotatingSphere -x HOME=/home/openfoam -x FOAM_SIGFPE=1 -x FOAM_ETC=/usr/lib/openfoam/openfoam2112/etc pimpleFoam


the second node won't find the casedir. I've already settled /etc/ssh/ssd_config and /etc/ssh/sshd_config with sendEnv and Accepenv but it had no difference. I also noticed when I type

[code]
ssh node2 'which pimpleFoam'


I receive nothing, but when i log "ssh node2', and execute which pimpleFoam I receive the right answer. Does anyone could, please, help me?

A lot of job schedulers (I assume your cluster has one) can be used to provide the proper setup on all nodes.
If you have to do it yourself, the bin/tools/foamExec should be able to help with chaining through. Haven't checked recently, but the etc/openfoam wrapper should probably also work, but not 100% certain.

I managed to solve it! Thanks a lot!

The problem was the non interactive access to the slave node. In some linux distributions the bash.bahsrc has a line that closes it in case of a non interactive access (such a remote ssh), which is



[code]

# If not running interactively, don't do anything
[ -z "$PS1" ] && return


The solution: commenting this line such as



#[ -z "$PS1" ] && return


solves it immediately. If anyone has this problem refer to the environment set on openfoam an check if the profiles are settled to non interactive access! Also the error will be very random, such as openfoam complaining it can't find the controlDict.

About job schedulers, I am not using it, I am setting up a docker container cluster hehe. In a old cluster I used to use PBS pro, I am trying to use docker swarm.