CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

Problem running with 3 CPU's in parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   August 1, 2012, 10:57
Default Problem running with 3 CPU's in parallel
  #1
New Member
 
Join Date: Aug 2012
Posts: 3
Rep Power: 5
saurabh3737 is on a distinguished road
Hello everyone,
I have setup clustring environment on ubuntu 12. on 3 machines.
1 server and 2 clients
my cluster setup works well with any 1 server and 1 client combination,
but i was unable to run on all 3 machines for buoyantSimpleFoam in parallel

i m using command
mpirun -hostfile machines -np 3 /nfs/openfoam211/bin/foamExec buoyantSimpleFoam -parallel

when i execute this command for 3 machines than i din get any response not even an error it just hangs,
i check process on other 2 clients machines using Top command, so i can see that server is trying to initiate some process from server IP but it goes off every while and than, and ultimately buyantSimpleFoam is not running,
but it works perfectly with and 2 machines with 1 server and 1 client with different number of cores whatever i assigned.
(i personally feel its interprocess communication problem between clients)

Please Help me, any solution for above, its really very Urgent, Thank u so much.
saurabh3737 is offline   Reply With Quote

Old   August 2, 2012, 15:19
Default
  #2
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,507
Blog Entries: 34
Rep Power: 86
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings saurabh3737 and welcome to the forum!

I moved your post to a new thread and the proper sub-forum, because the other thread where you posted was really old...

OK, the problem you're having is very likely due to issues with network configuration. There are two details that come to mind:
  • Check if the machines you're working with have the files "/etc/hosts" properly configured.
  • What MPI toolbox are you using? Open-MPI? MPICH2?
    I ask this because for Open-MPI you can do something similar to this:
    Quote:
    Originally Posted by pkr View Post
    When using MPI_reduce, the OpenMPI was trying to establish TCP through a different interface. The problem is solved if the following command is used:
    mpirun --mca btl_tcp_if_exclude lo,virbr0 -hostfile machines -np 2 /home/rphull/OpenFOAM/OpenFOAM-1.6/bin/foamExec interFoam -parallel

    The above command will restrict MPI to use certain networks (lo, vibro in this case).
  • Additionally, on how to test if MPI is working: post #4 of "openfoam 1.6 on debian etch", and/or post #19 of "OpenFOAM updates" - Note: As of OpenFOAM 2.0.0, the application "parallelTest" is now called "Test-parallel".
For more information, you can also check my blog post Notes about running OpenFOAM in parallel

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   August 9, 2012, 08:47
Default Thanks for your help but still not solved, any other ways
  #3
New Member
 
Join Date: Aug 2012
Posts: 3
Rep Power: 5
saurabh3737 is on a distinguished road
I tried with this command as below but still same problem persist,

mpirun --mca btl_tcp_if_exclude lo,virbr0 -hostfile machines -np 6 /nfs/openfoam211/bin/foamExec buoyantSimpleFoam -parallel

is there any need to have passwordless SSH between 2 client machines as well?
or any other solution

Also my Test-parallel is not working for 1 server and 2 clients and it hangs after giving execution command, but works perfectly fine when i assigned with 1 server and 1 client.

can you please help me out to identify and resolve the problem.

Last edited by saurabh3737; August 9, 2012 at 09:09. Reason: missing information
saurabh3737 is offline   Reply With Quote

Old   August 9, 2012, 10:19
Default
  #4
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,507
Blog Entries: 34
Rep Power: 86
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Quote:
Originally Posted by saurabh3737 View Post
is there any need to have passwordless SSH between 2 client machines as well?
Only if one of the client machines is the one acting as master.

Quote:
Originally Posted by saurabh3737 View Post

Also my Test-parallel is not working for 1 server and 2 clients and it hangs after giving execution command, but works perfectly fine when i assigned with 1 server and 1 client.

can you please help me out to identify and resolve the problem.
A few ideas/questions:
  1. Check the file "/etc/hosts" in all 3 machines. Do they all have knowledge of the names and IPs of the other machines?
  2. Are you able to ping all machines among themselves? Can the server ping each one of the two clients and is each client able to ping the other client and server?
wyldckat is offline   Reply With Quote

Old   August 16, 2012, 05:38
Default Yes i have checked all the host requirement
  #5
New Member
 
Join Date: Aug 2012
Posts: 3
Rep Power: 5
saurabh3737 is on a distinguished road
I have checked all the hosts requirement ping will works in all 3 machines to any machine among them. .etc/hosts is properly configured, is there any need to configure wmakeschedular of openfoam or have to set enviornment for schedule or somwhere else in openfoam to give no of machines?
Is it possible to have any direct communcation with you by phone, email or skype, will be grateful to u.
Thanks
saurabh3737 is offline   Reply With Quote

Old   August 16, 2012, 17:05
Default
  #6
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,507
Blog Entries: 34
Rep Power: 86
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Quote:
Originally Posted by saurabh3737 View Post
I have checked all the hosts requirement ping will works in all 3 machines to any machine among them. .etc/hosts is properly configured
Testing if you can ping is just one of the most basic tests. The next step is to determine how many network cards are there and if there is visibility among all of them. In other words, when you run on each machine:
Code:
/usr/sbin/ifconfig
Are all of the connections visible to the same internal network? Or is there only lo and eth0 on the client machines, as shown here:
Code:
eth0      Link encap:Ethernet  HWaddr AA:BB:CC:DD:EE:FF  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:41 Base address:0xc000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
Quote:
Originally Posted by saurabh3737 View Post
is there any need to configure wmakeschedular of openfoam or have to set enviornment for schedule or somwhere else in openfoam to give no of machines?
wmakescheduler related information only affects building OpenFOAM itself in parallel, it does not affect running cases. For cases, you should use foamJob and a machines file. Example:
Code:
foamJob -s -p Test-parallel
File "machines", placed inside the case folder:
Code:
server slots=4
slave1 slots=2
slave2 slots=2
Quote:
Originally Posted by saurabh3737 View Post
Is it possible to have any direct communcation with you by phone, email or skype, will be grateful to u.
Thanks
Via email in my off-duty hours, you can start the conversation by using the "Contact Info -> Send a message via email" from my profile page: http://www.cfd-online.com/Forums/members/wyldckat.html
For Skype or phone, it'll have to be on-duty and you'll have to be willing to pay for the support; you can either send me an email or contact the company I work for, namely blueCAPE: http://www.bluecape.com.pt
wyldckat is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
OF 2.0.1 parallel running problems moser_r OpenFOAM Running, Solving & CFD 7 March 18, 2014 09:49
problem in the CFX12.1 parallel computation BalanceChen ANSYS 2 July 7, 2011 10:26
Running mapFields with Parallel Source and Parallel Target RDanks OpenFOAM Mesh Utilities 2 June 6, 2011 07:56
Problem running a parallel fluent job on local machine via mpd highhopes FLUENT 0 March 3, 2011 06:07
Issue with running in parallel on multiple nodes daveatstyacht OpenFOAM 7 August 31, 2010 17:16


All times are GMT -4. The time now is 01:13.