CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

openfoam on 2 machines with passwordless ssh

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   April 22, 2014, 06:48
Default openfoam on 2 machines with passwordless ssh
  #1
New Member
 
Devayan Bir
Join Date: Jul 2013
Posts: 15
Rep Power: 4
devayabir is on a distinguished road
Hi All,
I have managed to get passwordless ssh started on the two computers through the following steps.
1. Setup NFS sever:

1.3 Add Directory: /home/user/OpenFOAM
1.4 Add Host: enter the host name (or IP). enter
rw,sync under options.
1.5 click Finish to start NFS server.
2. Setup NFS Client
2.1 start YaST. Select Network Service. Click "NFS Client".
2.2 Click "Add".
2.3 enter NFS Server name.
2.4 enter Remote File System: /home/user/OpenFOAM.
2.5 enter Mount Point: /home/user/OpenFOAM
3. Setup passwordless SSH
3.1 Client Side (This actually would be the NFS server, because the NFS server needs to log onto the clients)
3.1.1 Type "ssh-keygen -t dsa"
3.1.2 hit return twice when prompts for passphrase
3.1.3 type "cd ~/.ssh"
3.1.4 type "ls", you should get 3 files: id_dsa id_dsa.pub known_hosts
3.1.5 scp id_dsa.pub user@NFS_client<img src="images/smil...>ublic-key.tmp
3.2 Server side (this will be NFS client):
3.2.1 type "ls"
3.2.2 type "mkdir .ssh"
3.2.3 type "chmod 700 .ssh"
3.2.4 type "cd .ssh"
3.2.5 type "cat ~/public-key.tmp >> authorized_keys"
3.2.6 type "rm ~/public-key.tmp"
Note: from the NFS sever side, type "ssh -l userID NFS_client" to see if you can log in without a password.

After this I can connect to one computer to the other through ssh without the password.

I made the hostfile machine and tried running the mpirun command but it doesnt do anything.
It doesnt give me an error also


Please advice......
devayabir is offline   Reply With Quote

Old   April 22, 2014, 18:33
Default
  #2
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
I have just got this going today myself. 16 processes , 8 on master, 8 on slave
I did it this way:
Get a parallel processing tutorial case working on the master PC
Install openfoam on the slave,verify it works.
Get /etc/hosts files in sync on both nodes
Set up password less ssh from master PC to slave PC for the same user on both machines .
Set up the master PC to Nfs serve the directory structure above the tutorial case eg /home
Set up slave to mount the served structure e.g. As /mnt/master-home
Change the openmpi default host file on master to have an entry for master and slave.
Code:
Master cpu=8
Slave cpu=8
Now you need to change the decomposepar files and Allrun in the case to use 16 processes.
In the topmost decomposepar you need to set the the distributed flag and describe the 7 roots for each master process and 8 roots for each slave process The root is the path to a directory level just above the case directory
The processes on master will have a root that begins with at /home the processes on the slave begins with /mnt/master-home


It took a lot of time to work it out, but easy once you have it Sussed. I will give a more detailed account tomorrow.
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Old   April 23, 2014, 01:18
Default
  #3
New Member
 
Devayan Bir
Join Date: Jul 2013
Posts: 15
Rep Power: 4
devayabir is on a distinguished road
Quote:
Originally Posted by derekm View Post
In the topmost decomposepar you need to set the the distributed flag and describe the 7 roots for each master process and 8 roots for each slave process The root is the path to a directory level just above the case directory
The processes on master will have a root that begins with at /home the processes on the slave begins with /mnt/master-home
I did the rest.Could you give me an example of how to set the root files.
devayabir is offline   Reply With Quote

Old   April 23, 2014, 06:33
Default
  #4
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
Quote:
Originally Posted by devayabir View Post
I did the rest.Could you give me an example of how to set the root files.
This example is straight from a working system

The example has two nodes and 8 cores in each. The example ismultiRegionHeater

The master node is linux-kzcl
The slave is linux-uv4

Both boxes are opensuse 12.3 with OpenFOAM 2.3

The case is at
/home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16

/home on linux-kzcl is mounted on linux-uv4 as /mnt/linux-kzcl_home

on linux-kzcl : /usr/lib64/mpi/gcc/openmpi/et/openmpi-default-hostfile contains

Code:
linux-kzcl  cpu=8 
linux-uv34  cpu=8
the file on linux-kzcl at /home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/system/decomposeParDict contains at the end :
Code:
...
/// Is the case distributed
distributed     yes;
//// Per slave (so nProcs-1 entries) the directory above the case.
roots
(
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"//1
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/home/derekm/OpenFOAM/derekm-2.3.0/run"//7
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"  //8
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"
  "/mnt/linux-kzcl_home/derekm/OpenFOAM/derekm-2.3.0/run"  //15
}
/home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/system/decomposeParDict

home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/system/heater/decomposeParDict

home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/system/rightsolid/decomposeParDict

home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/system/topAir/decomposeParDict

...

all have

Code:
...
numberOfSubdomains  16;
...

home/derekm/OpenFOAM/derekm-2.3.0/run/multiRegionHeaterdmm16/Allrun
contains
Code:
...
# Run
runParallel `getApplication` 16
...
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Old   April 23, 2014, 08:08
Default
  #5
New Member
 
Devayan Bir
Join Date: Jul 2013
Posts: 15
Rep Power: 4
devayabir is on a distinguished road
Hi,

The only difference with our case is that we are trying to run on two different computers.
They are connected via a lan cable, and I can ping into the other machine and viceversa.
Also I have made a password-less ssh access. Following is my set up,

comp(1) tetrahedrix@tetrahedrix-desktop
comp(2) kj@Stokes

I have divided the case as given in the post give under the following link


My hostfile is
HTML Code:
 tetrahedrix@tetrahedrix-desktop cpu=4
kj@Stokes cpu=4

My decomposeParDict is

HTML Code:
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node1"
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node2"
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node3"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node4"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node5"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node6"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node7"
Whenever I'm running the command through mpirun --hostfile it does nothing, please advice.

Thanks
devayabir is offline   Reply With Quote

Old   April 23, 2014, 08:54
Default
  #6
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
have you got working in parallel but on a single node?
do that first
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Old   April 23, 2014, 09:04
Default
  #7
New Member
 
Devayan Bir
Join Date: Jul 2013
Posts: 15
Rep Power: 4
devayabir is on a distinguished road
yes on a single I have done parallel computing.That is pretty straight forward.
My issue being that this doesnt even give me an error.It does nothing.
If in the machine files I dont specify the username it asks me for password assuming the username of the first machine.(Usernames on both computers are different)
So I think there is an issue in the path for the processors in the second machine.

Please advice
devayabir is offline   Reply With Quote

Old   April 23, 2014, 10:06
Default
  #8
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
Quote:
Originally Posted by devayabir View Post
Hi,

The only difference with our case is that we are trying to run on two different computers.
They are connected via a lan cable, and I can ping into the other machine and viceversa.
Also I have made a password-less ssh access. Following is my set up,

comp(1) tetrahedrix@tetrahedrix-desktop
comp(2) kj@Stokes

I have divided the case as given in the post give under the following link


My hostfile is
HTML Code:
 tetrahedrix@tetrahedrix-desktop cpu=4
kj@Stokes cpu=4
My decomposeParDict is

HTML Code:
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node1"
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node2"
"/home/tetrahedrix/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node3"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node4"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node5"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node6"
"kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node7"
Whenever I'm running the command through mpirun --hostfile it does nothing, please advice.

Thanks
reread my example... the root is the path to directory above the case, so why all this node1 node2 for each subdomain?
that implies a separate case for each subdomain.

are tetrahedrix@tetrahedrix-desktop and kj@stokes the host names in /etc/hosts? they look like username@host


kj@Stokes:/home/kj/OpenFOAM/OpenFOAM-2.3.x/tutorials/incompressible/simpleFoam/node4 does not look like a path for the slave processes to access the master node case.

work out the path to the master node case directory from both the master and the slave. Test it by logging into both node and see if the path really works.

the root is the path MINUS the case directory name. for the first 4 it will just be the path on the master node
on the second 4 it will be the path as seen by the slave node.

you appear to be using a non default directory structure for OpenFOAM

give me the hostnames
the path on the master node to access the case directory on the masternode
the path on the slavenode to access the case directory on the masternode
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Old   April 25, 2014, 03:52
Default
  #9
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
Quote:
Originally Posted by devayabir View Post
yes on a single I have done parallel computing.That is pretty straight forward.
My issue being that this doesnt even give me an error.It does nothing.
If in the machine files I dont specify the username it asks me for password assuming the username of the first machine.(Usernames on both computers are different)
So I think there is an issue in the path for the processors in the second machine.

Please advice
use the same username on both machines
Try to get it working on the simplest possible configuration using an standard example using the standard Allrun. Then make changes.
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Old   August 3, 2014, 13:39
Default
  #10
New Member
 
Devayan Bir
Join Date: Jul 2013
Posts: 15
Rep Power: 4
devayabir is on a distinguished road
Hi,

This is quite late but thanks, for your assistance were able to run OF on two machines in parallel. We are connecting the two machines via a LAN cable. We wanted to know if this can be a limiter to speed up of the simulation. How should the two machines be connected to have maximum speed.
In our case we have a 30 Million mesh and the configuration of our machines is i5 processor quad core with 32 GB RAM. How many processors should be kept for slave and master to have a maximum speed up.

Waiting for your reply,

Thanks,
devayabir is offline   Reply With Quote

Old   August 4, 2014, 18:25
Default
  #11
Member
 
Derek Mitchell
Join Date: Mar 2014
Location: UK, Reading
Posts: 95
Rep Power: 4
derekm is on a distinguished road
Quote:
Originally Posted by devayabir View Post
Hi,

This is quite late but thanks, for your assistance were able to run OF on two machines in parallel. We are connecting the two machines via a LAN cable. We wanted to know if this can be a limiter to speed up of the simulation. How should the two machines be connected to have maximum speed.
In our case we have a 30 Million mesh and the configuration of our machines is i5 processor quad core with 32 GB RAM. How many processors should be kept for slave and master to have a maximum speed up.

Waiting for your reply,

Thanks,
The limiting number of cells per core often quoted on the forum as 100 000 so you would need about 300 cores to achieve the maximum scaling
my configuration is targeting 10 million cells with 88 cores. Above 20 cores is normally infiniband not ethernet according the sources on the forum.
__________________
A CHEERING BAND OF FRIENDLY ELVES CARRY THE CONQUERING ADVENTURER OFF INTO THE SUNSET
derekm is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Memory protection in OpenFOAM / combinig with FORTRAN botp OpenFOAM Programming & Development 1 February 6, 2013 05:26
OpenFOAM 1.6.x, 1.7.0 and 1.7.x are not fully prepared to work with gcc-4.5.x wyldckat OpenFOAM Bugs 18 October 21, 2010 05:51
OpenFOAM 1.5.x package - CentOS 5.3 x86_64 linnemann OpenFOAM Installation 7 July 30, 2009 03:14
OpenFOAM Install problem masb OpenFOAM 3 May 25, 2009 11:32
The OpenFOAM extensions project mbeaudoin OpenFOAM 16 October 9, 2007 09:33


All times are GMT -4. The time now is 00:54.