CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Installation

[OpenFOAM.com] OpenFOAM and MPI cluster using SSH

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 30, 2022, 19:40
Default OpenFOAM and MPI cluster using SSH
  #1
New Member
 
Mateus Grassano Lattari
Join Date: Apr 2018
Posts: 16
Rep Power: 6
MateusLatt is on a distinguished road
Hi, I am not being able to execute openfoam using mpi in a two node cluster. In one hand when I execute



[code]
mpirun -np 24 -hostfile hosts --use-hwthread-cpus -wdir `pwd` -x FOAM_ETC=/usr/lib/openfoam/openfoam2112/etc pimpleFoam
the second node won't find openfoam's binaries, when I set then using



[code]

mpirun -np 24 -hostfile hosts --use-hwthread-cpus -wdir `pwd` -x LD_LIBRARY_PATH=/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib/sys-openmpi -x PATH=/usr/lib/x86_64-linux-gnu/openmpi/bin:/home/matt/OpenFOAM/matt-v2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/site/2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/bin:/usr/lib/openfoam/openfoam2112/bin:/usr/lib/openfoam/openfoam2112/wmake:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin -x FOAM_RUN=/home/openfoam/rotatingSphere -x HOME=/home/openfoam -x FOAM_SIGFPE=1 -x FOAM_ETC=/usr/lib/openfoam/openfoam2112/etc pimpleFoam


the second node won't find the casedir. I've already settled /etc/ssh/ssd_config and /etc/ssh/sshd_config with sendEnv and Accepenv but it had no difference. I also noticed when I type

[code]
ssh node2 'which pimpleFoam'


I receive nothing, but when i log "ssh node2', and execute which pimpleFoam I receive the right answer. Does anyone could, please, help me?
MateusLatt is offline   Reply With Quote

Old   July 1, 2022, 12:01
Default
  #2
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: https://olesenm.github.io/
Posts: 1,503
Rep Power: 35
olesen will become famous soon enougholesen will become famous soon enough
A lot of job schedulers (I assume your cluster has one) can be used to provide the proper setup on all nodes.
If you have to do it yourself, the bin/tools/foamExec should be able to help with chaining through. Haven't checked recently, but the etc/openfoam wrapper should probably also work, but not 100% certain.
olesen is offline   Reply With Quote

Old   July 1, 2022, 12:12
Default
  #3
New Member
 
Mateus Grassano Lattari
Join Date: Apr 2018
Posts: 16
Rep Power: 6
MateusLatt is on a distinguished road
I managed to solve it! Thanks a lot!

The problem was the non interactive access to the slave node. In some linux distributions the bash.bahsrc has a line that closes it in case of a non interactive access (such a remote ssh), which is



[code]

# If not running interactively, don't do anything
[ -z "$PS1" ] && return


The solution: commenting this line such as



#[ -z "$PS1" ] && return


solves it immediately. If anyone has this problem refer to the environment set on openfoam an check if the profiles are settled to non interactive access! Also the error will be very random, such as openfoam complaining it can't find the controlDict.
MateusLatt is offline   Reply With Quote

Old   July 1, 2022, 12:13
Default
  #4
New Member
 
Mateus Grassano Lattari
Join Date: Apr 2018
Posts: 16
Rep Power: 6
MateusLatt is on a distinguished road
About job schedulers, I am not using it, I am setting up a docker container cluster hehe. In a old cluster I used to use PBS pro, I am trying to use docker swarm.
MateusLatt is offline   Reply With Quote

Reply

Tags
cluster computing, mpi, openfoam

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[OpenFOAM.com] Install OpenFOAM 3.0.1 on cluster without root access; mpi error Madeinspace OpenFOAM Installation 1 July 4, 2020 15:16
OpenFOAM Parallelization MPI Cluster Problem arslan.ali OpenFOAM Running, Solving & CFD 4 September 23, 2018 14:50
[Other] Using different MPI types and versions with OpenFOAM spaceprop OpenFOAM Installation 2 May 28, 2018 04:31
OpenFOAM v3.0+ ?? SBusch OpenFOAM 22 December 26, 2016 15:24
How to install the OpenFoam in the cluster. Please help me! flying OpenFOAM Installation 6 November 27, 2009 04:00


All times are GMT -4. The time now is 06:59.