CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

Running Foam on multiple nodes (small cluster)

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   June 9, 2012, 08:43
Default Running Foam on multiple nodes (small cluster)
  #1
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 7
Hisham is on a distinguished road
Hello Foamers,

I need to run OpenFOAM on two PCs. So:
1. I work on one (Master) and have a password SSH access to the other (slave).
2. I have installed version 2.1.0 from OpenCFD on both.
3. I have changed the ./bashrc of the slave to have OpenFOAM sourced before exiting the script due to non-interactive session.
4. I have the damBreak case divided into 12 sub-domains using scotch algorithm of decomposePar on both (the same path)
5. The case runs serially and in-parallel on both PCs individually.
6. I have a machines file:
Code:
localhost
username@slaveIP cpu=4
7. Running
Code:
foamJob -p interFoam
yields this error:
Code:
$ foamJob -p -s interFoam
Parallel processing using SYSTEMOPENMPI with 12 processors
Executing: /usr/bin/mpirun -np 12 -hostfile machines /opt/openfoam210/bin/foamExec -prefix /opt interFoam -parallel | tee log
username@IP password: 
[1] /*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.1.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.1.0-0bc225064152
Exec   : interFoam -parallel
Date   : Jun 09 2012
Time   : 14:40:07
Host   : "numubuntu-System-Product-Name"
PID    : 4865

[1] 
[1] --> FOAM FATAL IO ERROR: 
[1] Expected a ')' or a '}' while reading List, found on line 0 an error
[1] 
[1] file: IOstream at line 0.
[1] 
[1]     From function Istream::readEndList(const char*)
[1]     in file db/IOstreams/IOstreams/Istream.C at line 159.
[2] 
[2] 
[2] [1] --> FOAM FATAL IO ERROR: 
[2] Expected a ')' or a '}' while reading List, found on line 0 an error
[2] 
[2] file: IOstream at line 0.
[2] 
[2]     From function Istream::readEndList(const char*)
[2]     in file db/IOstreams/IOstreams/Istream.C at line 159.
[2] 
FOAM parallel run exiting
[2] 

FOAM parallel run exiting
[1] --------------------------------------------------------------------------
MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD 
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

--------------------------------------------------------------------------
mpirun has exited due to process rank 2 with PID 2105 on
node SLAVE-IP exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
[3] 
[3] [4] 
[4] 
[4] --> FOAM FATAL IO ERROR: 
[4] Expected a ')' or a '}' while reading List, found on line 0 an error
[4] 
[4] file: IOstream at line 0.
[4] 
[4]     From function Istream::readEndList(const char*)
[4]     in file db/IOstreams/IOstreams/Istream.C at line 159.
[4] 
FOAM parallel run exiting
[4] 

[3] [numubuntu-System-Product-Name:04861] 2 more processes have sent help message help-mpi-api.txt / mpi-abort
[numubuntu-System-Product-Name:04861] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
I appreciate any help!

Best regards
Hisham El Safti
Hisham is offline   Reply With Quote

Old   June 10, 2012, 14:23
Default
  #2
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 7,345
Blog Entries: 32
Rep Power: 72
wyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the rough
Greetings Hisham,

Indeed there seems to be some strange detail that is escaping here...

OK, let's try to debug this in parts:
  1. Is the user name the same on both machines? If not, you might want to try using the multiple roots definition in "decomposeParDict". I wrote a blog post about it some time ago: Running OpenFOAM in parallel with different locations for each process
  2. Let's skip the need for password. Follow these instructions for a passwordless access for your own user between machines: http://homepages.inf.ed.ac.uk/imurra...dless_ssh.html
    It's only passwordless if you don't use a password for your key. If yours is a closed internal network and unlikely that an internal access attack is done, then you won't need a password for this private/public key pair.
  3. Specify the number of cpu's for the local machine as well, just in case.
I can't think of any other hypothesis for now.

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   June 10, 2012, 15:21
Default
  #3
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 7
Hisham is on a distinguished road
Hello Bruno

Thanks for your reply. I have the same user name with "sudo" capabilities on both machines. I have specified the cpu count as proposed. I still get the same error. I think there is something to do with version incompatibilities. However, the error does not say which bit has the ) instead of } ... Also how can I view the so called "help messages"?

Thanks
Hisham

Edit: I have open mpi 1.4.3 on both machines
Hisham is offline   Reply With Quote

Old   June 10, 2012, 15:59
Default
  #4
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 7,345
Blog Entries: 32
Rep Power: 72
wyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the rough
Hi Hisham,

OK, try running this command:
Code:
/usr/bin/mpirun -np 12 -hostfile machines /opt/openfoam210/bin/foamExec interFoam -parallel
I removed the "-prefix /opt" from the previous command.

As for the help messages... according to the error message, it says to use the MCA system to set "orte_base_help_aggregate" to 0... Should be something like this:
Code:
/usr/bin/mpirun -np 12 -mca orte_base_help_aggregate 0 -hostfile machines /opt/openfoam210/bin/foamExec interFoam -parallel
For a full help on mpirun: http://www.open-mpi.org/doc/v1.4/man1/mpirun.1.php

In the machines file, define things like this:
Code:
localhost slots=8
slaveIP slots=4
Source: http://www.open-mpi.org/faq/?categor...imple-spmd-run

edit:
Quote:
A nice summary of how-to: OpenFOAM on deux pc post #13
Source: Notes about running OpenFOAM in parallel

Best regards,
Bruno

Last edited by wyldckat; June 10, 2012 at 16:03. Reason: see "edit:"
wyldckat is offline   Reply With Quote

Old   June 11, 2012, 13:44
Default
  #5
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 7
Hisham is on a distinguished road
Thanks a lot Bruno. I had the chance to use a third PC today and the problem was with one of the PCs. I have changed the machines file as you suggested though.

Thanks again
Hisham is offline   Reply With Quote

Reply

Tags
cluster, nodes, parallel

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to set periodic boundary conditions Ganesh FLUENT 13 January 22, 2014 04:11
Using starToFoam clo OpenFOAM Other Meshers: ICEM, Star, Ansys, Pointwise, GridPro, Ansa, ... 33 September 26, 2012 04:04
OpenFOAM static build on Cray XT5 asaijo OpenFOAM Installation 9 April 6, 2011 12:21
Import problem ARC Open Source Meshers: Gmsh, Netgen, CGNS, ... 0 February 27, 2010 10:56
StarToFoam error Kart OpenFOAM Meshing & Mesh Conversion 1 February 4, 2010 04:38


All times are GMT -4. The time now is 15:01.