CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

SimpleFoam run in Parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   May 4, 2010, 01:28
Default SimpleFoam run in Parallel
  #1
New Member
 
Jay
Join Date: Feb 2010
Posts: 15
Rep Power: 7
jayrup is on a distinguished road
Hi Foamers

I have two machines and it's host names are 'cfd0' and 'cfd1'.
The cfd0 machine is main node machine and cfd1 is client node machine.
I have install OF on cfd0 and mpich is also install on cfd0. And cfd0 is mounted on cfd1 machine. Both machine are dual core machines.
Now this two machines are in cluster and I can run simple hello world program on this cluster properly using all 4 processor.
When I run simpleFoam case in parallel on one machine with decomposepardict file as :

numberOfSubdomains 4;

method simple;

simpleCoeffs
{
n ( 2 2 1 );
delta 0.001;
}

hierarchicalCoeffs
{
n ( 1 1 1 );
delta 0.001;
order xyz;
}

metisCoeffs
{
processorWeights ( 1 1 1 1 );
}

manualCoeffs
{
dataFile "";
}

distributed no;


roots ( );
/*------------------------------------------------------------------------*/

it works on only one machine.
To run simpleFoam on cluster I need to change

distributed yes;

roots 2(cfd0, cfd1 );

in decomposepardict file.
But it dosen't work.
Please help.
Thank you in advance.

Jay
jayrup is offline   Reply With Quote

Old   May 4, 2010, 09:52
Default
  #2
New Member
 
Yasar
Join Date: Dec 2009
Location: Stuttgart
Posts: 9
Rep Power: 7
HelloWorld is on a distinguished road
Hello Jay,

I'm not quite sure but maybe this helps:

set distributed no; and roots ();

create a file in your case-directory which is named machines or something like that. In the machines-file write down the IP-addresses of the machines you would like to compute on, as follows:
192.168.2.1 slots=2 max_slots=2
192.168.2.2 slots=2 max_slots=2 and so on

decompose your case and start simpleFoam using the mpirun-command, e.g. in your case something like:
mpirun -np 4 --hostfile machines /Path/To/OpenFOAM/bin/foamExec simpleFoam -parallel > logfile

I had VMWare running on the machines so mpirun couldn't find the IP-adresses of the machines for some reason so I added the following mca-option after the hostfile-option to the mpirun-command:
--mca btl_tcp_if_include eth0

worked fine for me with 4 PCs and 16 CPUs, so I hope it helps...

Greetings.
HelloWorld is offline   Reply With Quote

Old   May 5, 2010, 02:41
Default
  #3
New Member
 
Jay
Join Date: Feb 2010
Posts: 15
Rep Power: 7
jayrup is on a distinguished road
Hello Yasar
Thank you for reply.

As you said I create one file in case directory named as machines and added ip addresses of the machines as said.
and after decompose when I run the case :

$ mpirun -np 4 --mca btl_tcp_if_include eth0 machines /home/mpi/OpenFOAM/OpenFOAM-1.6/bin/foamExec simpleFoam -parallel
--------------------------------------------------------------------------
mpirun was unable to launch the specified application as it could not find an executable:

Executable: machines
Node: cfd0

while attempting to start process rank 0.
--------------------------------------------------------------------------
I got the above error. Please help if have any option.
Regards
Jay
jayrup is offline   Reply With Quote

Old   May 5, 2010, 06:50
Default
  #4
New Member
 
Yasar
Join Date: Dec 2009
Location: Stuttgart
Posts: 9
Rep Power: 7
HelloWorld is on a distinguished road
Hello Jay,

seems to me that you mixed something up in your command. Try the following and hopefully it'll do:

$ mpirun -np 4 --hostfile machines --mca btl_tcp_if_include eth0 /home/mpi/OpenFOAM/OpenFOAM-1.6/bin/foamExec simpleFoam -parallel

Regards
Yasar
HelloWorld is offline   Reply With Quote

Old   May 6, 2010, 00:13
Default
  #5
New Member
 
Jay
Join Date: Feb 2010
Posts: 15
Rep Power: 7
jayrup is on a distinguished road
Hello Yasar
Thanks for again reply.
It doesn't work again.
$ mpirun -np 4 --hostfile machines --mca btl_tcp_if_include eth0 /home/mpi/OpenFOAM/OpenFOAM-1.6/bin/foamExec simpleFoam -parallel
bash: orted: command not found
--------------------------------------------------------------------------
A daemon (pid 8078) died unexpectedly with status 127 while attempting
to launch so we are aborting.

There may be more information reported by the environment (see above).

This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
mpirun: clean termination accomplished

Again having some error.

Regards
Jay
jayrup is offline   Reply With Quote

Old   May 6, 2010, 07:15
Default
  #6
New Member
 
Yasar
Join Date: Dec 2009
Location: Stuttgart
Posts: 9
Rep Power: 7
HelloWorld is on a distinguished road
Hello Jay,

maybe you should try it without the mca-command:

$ mpirun -np 4 --hostfile machines /home/mpi/OpenFOAM/OpenFOAM-1.6/bin/foamExec simpleFoam -parallel

if this doesn't help I'm afraid I don't know further suggestions...

Regards
Yasar
HelloWorld is offline   Reply With Quote

Old   May 6, 2010, 20:39
Default
  #7
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,253
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hello Jay and Yasar,

Jay, might I suggest that you try using the parallelTest utility - information available on this post: OpenFOAM updates post #19
The parallelTest utility (it's part of OpenFOAM's test utilities) can aid you in sorting out the more basic MPI problems, like communication problems or missing environment settings or libraries not found, without running any particular solver functionalities.

Secondly, since it's MPICH (and by my experience with MPICH 1 and 2), you have to force it to send the environment variables to the targeted slaves. Check the manual on how to force it to send environment variables. Although, the script foamExec should do that for you, but only if you've setup properly the files settings.sh and bashrc in the folder OpenFOAM/OpenFOAM-1.6/etc!

As for:
Quote:
mpirun -np 4 --hostfile machines --mca btl_tcp_if_include eth0 /home/mpi/OpenFOAM/OpenFOAM-1.6/bin/foamExec simpleFoam -parallel
bash: orted: command not found
If I'm not mistaken, orted is also known as mpirun and mpiexec in some MPI libraries. Are you sure that the same MPICH library is installed in both machines? And is it installed as a system library/application? Or is it installed as part of OpenFOAM's Third Party applications in the folder ThirdParty-1.6?
Because if it's not installed as a system library/application, and like I've written above, you should edit the files settings.sh and bashrc in OpenFOAM/OpenFOAM-1.6/etc folder, to define properly the environment variables for using your MPICH version! But even if it's a system MPI, you should edit them just the same, so it won't set-up the default OpenMPI library as the working MPI library to be used!

Best regards,
Bruno
wyldckat is online now   Reply With Quote

Old   November 23, 2010, 05:40
Default question
  #8
New Member
 
jtalla
Join Date: Nov 2010
Posts: 1
Rep Power: 0
jtalla is on a distinguished road
Hello,
I really need your help, I am stuck with this issue for a while and I
can not figure it out.
I build a cluster using rocks Centos 5.3 and everything seems works
fine accept when I am trying to submit a job to the cluster (try to do
parallel simulation) I always get the message: mpiexec was unable to
launch the specified application as it could not find an executable:
Executable: abinip"

I am using mpich2 in my system.

I am not sure what to do?

I am new to the linux world so any help will be appreciated.
jtalla is offline   Reply With Quote

Old   November 25, 2010, 01:43
Default
  #9
New Member
 
Jay
Join Date: Feb 2010
Posts: 15
Rep Power: 7
jayrup is on a distinguished road
Hello jtalla
First of all I would like to thank Bruno Santos.
Jtalla if you have properly install mpich2 and if you can access other machine without password, then you can try the command which is there in Bruno's link
"Jay, might I suggest that you try using the parallelTest utility - information available on this post: OpenFOAM updates post #19
The parallelTest utility (it's part of OpenFOAM's test utilities) can aid you in sorting out the more basic MPI problems, like communication problems or missing environment settings or libraries not found, without running any particular solver functionalities. "

Because I have sorted out my problems.
Once again Thank you Bruno
Jay
jayrup is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unable to run OF in parallel on a multiple-node cluster quartzian OpenFOAM 3 November 24, 2009 14:37
Run in parallel: bug or what? novyno OpenFOAM Running, Solving & CFD 0 November 10, 2009 11:34
serial run fine, but parallel run diverged phsieh2005 OpenFOAM Running, Solving & CFD 2 October 6, 2009 08:33
Swap usage on parallel run nikhilesh OpenFOAM Running, Solving & CFD 0 April 30, 2009 09:50
Run in parallel a 2mesh case cosimobianchini OpenFOAM Running, Solving & CFD 2 January 11, 2007 07:33


All times are GMT -4. The time now is 20:00.