CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

OpenFOAM Parallelization MPI Cluster Problem

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree2Likes
  • 1 Post By wyldckat
  • 1 Post By wyldckat

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   August 11, 2018, 02:27
Default OpenFOAM Parallelization MPI Cluster Problem
  #1
New Member
 
Arslan Ali
Join Date: Jul 2018
Posts: 2
Rep Power: 0
arslan.ali is on a distinguished road
Hello FOAMers,

Recently I have been working on parallelization of OpenFOAM on BeoWulf clusters. Basically I’m having a very minor problem in my cluster setup which I’m unable to debug.



I have a cluster of 2 laptops.

Ubuntu 18.04LTS (dual-boot with windows)

OpenFOAM-v1806 installed on home directory

i7 processor, quad core, 64-bit


These specifications are same on both laptops.



- LAN connection has been setup between both laptops using a switch and
cables.
- Both laptops ping(0.3ms-0.6ms), and password-less ssh is also working
perfectly.
- Both laptops have the same case folder in the home directory.
- And in both laptops we run decomposePar separately after editing its Dict
file correctly.
- /etc/hosts file has been also edited on both laptops.
- Master node has the ‘machines’ file in the home folder.


Now we run the mpirun command but it doesnt run. After typing the command:

‘mpirun -np 8 --hostfile ~/machines pisoFoam -parallel’

We press enter and the terminal freezes on the next line. (Nothing happens)


I have uploaded screenshots of all necessary information you might need.

Thanks in Advance.


https://imgur.com/a/mWe0ZSF
https://imgur.com/a/TIQBEkc
https://imgur.com/a/hXe4DLq
https://imgur.com/a/k8FlHjC
arslan.ali is offline   Reply With Quote

Old   August 22, 2018, 09:43
Default
  #2
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,974
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Greetings arslan.ali,


I don't know if you have solved this issue, but from what I can see, the problem is that OpenFOAM is installed in different folders.


I don't know how you are sourcing the OpenFOAM environments on each machine, but the idea is that you should let them be sourced when your "/home/*/.bashrc" file is called.


The other possibility is to use multiple launch calls, namely as explained here: https://www.open-mpi.org/faq/?category=running#mpmd-run
My guess is that in your case, you should make the launch command something like this:
Code:
mpirun --hostfile ~/machines  -np 4 $HOME/OpenFOAM/OpenFOAM-v1806/bin/foamExec pisoFoam -parallel : -np 4 /home/otherUserAtSecondMachine/OpenFOAM/OpenFOAM-v1806/bin/foamExec pisoFoam -parallel

But the simplest would be to either have the same username on both machines or to simply install OpenFOAM at "/opt".


Best regards,
Bruno
Aabadani likes this.
__________________
wyldckat is offline   Reply With Quote

Old   August 22, 2018, 12:42
Default
  #3
Member
 
Join Date: Nov 2014
Posts: 92
Rep Power: 11
hokhay is on a distinguished road
It can be the communication problem. Have you make sure the passwordless ssh is working on both sides. The node also needs to be able to ssh into the master
hokhay is offline   Reply With Quote

Old   September 13, 2018, 08:31
Default
  #4
New Member
 
Arslan Ali
Join Date: Jul 2018
Posts: 2
Rep Power: 0
arslan.ali is on a distinguished road
Thanks wyldckat and hokhay for your replies.

Soon after this post I solved this problem by changing my openFoam installation directory to /opt. Just as wyldckat mentioned. However, later we received an error on the mpi command stating that "point files in polymesh could not be found".. We solved that issue aswell by using NFS shared directory method.

At this point we were able to run OpenFoam simulations using multiple processors from multiple PCs.


However, right now the problem we're facing is that our simulation clocktimes dont make sense. Basically, using 1 PC and 8 cores we get a simulation clocktime of 271s on a vortex shedding case while on a cluster of 2 PCs, 16 cores we get a clocktime of 530s which doesnt make sense. Also we dont think that there are critical communication barriers between PCs since the ping between the PCs is 0.1-0.3ms.

Cluster specifications for both PCs are:
i7-4770
3.3GHz x 8.
8Gb RAM.
UBUNTU 14.04 LTS

Can you please guide us on what can be the issue. We're really hhoping to hear from you soon. Thanks for your time.
arslan.ali is offline   Reply With Quote

Old   September 23, 2018, 14:50
Default
  #5
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,974
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Quick answer: The slow down you are seeing may be because either:
  1. The number of cells in each subdomain should be at least 50000 cells. Below that, you will start seeing little benefit in performance; perhaps you only have 5000 cells in each subdomain (in each "processor" folder).
  2. The case might be monstrous, for example, 50 million cells total, where it takes a ton of time to transfer information between machines.
  3. Or you forgot to add the argument "-parallel" to the command that calls the solver, as I indicated in a previous post.
  4. Or you are saving to disk too frequently.
There is a thread that tries to explain the following: How to give enough info to get help
Aabadani likes this.
wyldckat is offline   Reply With Quote

Reply

Tags
beowulf, cluster, mpi, openfoam, parallelization

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to contribute to the community of OpenFOAM users and to the OpenFOAM technology wyldckat OpenFOAM 17 November 10, 2017 16:54
[Other] Openfoam for windows 16.02 [CFD support] -problem with paraview ditmeyer OpenFOAM Installation 3 May 15, 2017 13:04
Problem with parallelization on cluster GiuMan OpenFOAM Running, Solving & CFD 12 August 14, 2015 06:11
How to setup a simple OpenFOAM cluster? TommiPLaiho OpenFOAM Installation 3 October 27, 2013 16:15
Sgimpi pere OpenFOAM 27 September 24, 2011 08:57


All times are GMT -4. The time now is 11:47.