[katiuskas]

Dear foamers,

I am currently struggling to run a parallel simulation (with OpenFOAM v2012) on a cluster (Cent OS 7.9.2009). I rely on the openmpi versions already installed on the cluster.

Until now, I ran the simulations using a version called ‘mpi/openmpi-x86_64’. However, only ‘mpi/openmpi-4.1.2’ is allowed for multi-node simulations (which is something I need for nProcessors > 48).
When I changed to this other version, the simulation produced an empty log file (and I had to stop the job manually).

In the bashrc file, I have

WM_MPLIB=SYSTEMOPENMPI

The job submission script looks something like this:

module load mpi/openmpi-4.1.2
source $HOME/OpenFOAM/OpenFOAM-v2012/etc/bashrc

export PBS_O_WORKDIR=$(readlink -f $PBS_O_WORKDIR)
cd $PBS_O_WORKDIR

# Host name
cat $PBS_NODEFILE 
(
for host in `cat $PBS_NODEFILE | sort -u -V`
do
     count=`grep $host $PBS_NODEFILE | wc -l`
     echo $host cpu=$count
done
) > host_file
cat host_file

######## Start the solver ###############
# Source tutorial run functions
. $WM_PROJECT_DIR/bin/tools/RunFunctions

# Set solver name
solver="mpirun -np $PBS_NP --hostfile host_file pimpleFoam -parallel"

# Run the solver
runApplication $solver
mv log.mpirun log.pimpleFoam

I would appreciate any help. Thanks!