CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

parallel running

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   March 30, 2010, 08:32
Default parallel running
  #1
Member
 
Join Date: Mar 2010
Posts: 30
Rep Power: 7
bunni is on a distinguished road
Hi,

Sorry if this is not the right place. I'm having a very strange problem with running openfoam with parallel processing. I've run the job with 1 processor, no problem. Then I run the job with 2 processors, it goes faster, the data's in the right place, no problem. Three processors, no problem. 4 processors - problem. I get no communication back from the program after about 4 or 5 minutes of run time. Nothing is updated to the logfile I redirect to, nothing is written to the processor? directories. But the jobs are still running. But, if I change the output write time (writeInterval in controlDict) to be a lesser value, I get no problem with 4 procs. So, somehow writeInterval changes whether or not the program hangs? any ideas as to what's going on?
bunni is offline   Reply With Quote

Old   March 30, 2010, 08:53
Default
  #2
Member
 
Antoine Devesa
Join Date: Mar 2010
Posts: 36
Rep Power: 7
A.Devesa is on a distinguished road
could you please post your controlDict and decomposeDict?
A.Devesa is offline   Reply With Quote

Old   March 30, 2010, 10:44
Default
  #3
Member
 
Join Date: Mar 2010
Posts: 30
Rep Power: 7
bunni is on a distinguished road
Sure,
(I've also included these as attachments with a txt at the end, due to the attachment manager).

It works fine for 2,3,4 processors for writeInterval =1e-6, and endTime 2e-5

but when the writeInterval is 1e-5 and endTime 1e-4, it seems to hang at (by looking at logfile) between t=7e-6 and 8e-6 - for 4 procs only. 2 or 3 work fine for 1e-5 and endTime 1e-4.

For 4 procs, I do a decomposition equally in x,y: (2,2,1). Again this works fine for writeInterval is 1e-6 and endTime 2e-5.

Cheers



controlDict :

FoamFile
{
version 2.0;
format ascii;
class dictionary;
location "system";
object controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application interFoam;

startFrom latestTime;

startTime 0;

stopAt endTime;

endTime 1e-04;

deltaT 1e-08;

writeControl adjustableRunTime;

writeInterval 1e-05;

purgeWrite 0;

writeFormat ascii;
writePrecision 8;

writeCompression uncompressed;

timeFormat general;

timePrecision 6;

runTimeModifiable yes;

adjustTimeStep yes;

maxCo 0.5;

maxDeltaT 1e-08;

/******/

decomposeParDict:

FoamFile
{
version 2.0;
format ascii;
class dictionary;
location "system";
object decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 3;

method simple;

simpleCoeffs
{
n ( 3 1 1 );
delta 0.001;
}

hierarchicalCoeffs
{
n ( 1 1 1 );
delta 0.001;
order xyz;
}

metisCoeffs
{
processorWeights ( 1 1 1 1 );
}

manualCoeffs
{
dataFile "";
}

distributed no;

roots ( );
Attached Files
File Type: txt controlDict.txt (1.3 KB, 1 views)
File Type: txt decomposeParDict.txt (1.2 KB, 8 views)
bunni is offline   Reply With Quote

Old   March 30, 2010, 10:58
Default
  #4
Member
 
Antoine Devesa
Join Date: Mar 2010
Posts: 36
Rep Power: 7
A.Devesa is on a distinguished road
i'm quite a beginner with OF, but i noticed 2 things, that unfornately do have nothing to do with decomposition...
- you're maxDeltaT is exactly your initial DeltaT. this could may be cause unexpected behaviour.
- i don't know the key word "adjustableRunTime", but just "runTime". did you try this one? did you observe the same behaviour?

If this is not bringing you further, and if for 2 and 3 processors, you were just decomposing in x direction, you should try to investigate what would pose such a problem when you cut in y direction...
A.Devesa is offline   Reply With Quote

Old   March 30, 2010, 11:31
Default
  #5
Member
 
Join Date: Mar 2010
Posts: 30
Rep Power: 7
bunni is on a distinguished road
Hi,

The thing is, it does work for np=4 (and for n=(2 2 1) in dec..ParDict), just for a different writeInterval. (and endtime. I didn't keep the long endtime with the short writeInterval, too much data!).

Anyway, I will look into the other variables. I'm sure you're quite more accomplished in openFoam than I am.

Last edited by bunni; March 30, 2010 at 11:33. Reason: clarification
bunni is offline   Reply With Quote

Old   April 7, 2010, 18:02
Default Running parallel
  #6
Member
 
Join Date: Nov 2009
Posts: 48
Rep Power: 7
farhagim is on a distinguished road
Hi there,

I am a beginner user of the OpenFoam...I really need to learn how to run openfoam with parallel processing. I read the all the available guideline in this site www.OpenFoam.com)...but I have basic problem with that. First of all I can not find "etc/hosts " in OpenFoam Directories where I should define the host names of the machines. Since I am not familiar with that kind of file
, I would be so grateful if anybody provide me one of these kind of files.

2. I read somewhere that I have to create a "file that contains the
host names of the machines. The file can be given any name and located at any path. " I could not understand what does it means ...or what kind of the files do i have to create....can anybody give a hint or example about that.

3. In decompose Dict, the roots entry is a list of root paths, <root0>, <root1>, . . . , for each node:
roots
<nRoots>
(
"<root0>"
"<root1>"
...
);

I need somebody help me throough this too..


Thank you very much guyss,
Tommaso

Quote:
Originally Posted by bunni View Post
Hi,

Sorry if this is not the right place. I'm having a very strange problem with running openfoam with parallel processing. I've run the job with 1 processor, no problem. Then I run the job with 2 processors, it goes faster, the data's in the right place, no problem. Three processors, no problem. 4 processors - problem. I get no communication back from the program after about 4 or 5 minutes of run time. Nothing is updated to the logfile I redirect to, nothing is written to the processor? directories. But the jobs are still running. But, if I change the output write time (writeInterval in controlDict) to be a lesser value, I get no problem with 4 procs. So, somehow writeInterval changes whether or not the program hangs? any ideas as to what's going on?
farhagim is offline   Reply With Quote

Old   April 8, 2010, 03:36
Default
  #7
Member
 
Antoine Devesa
Join Date: Mar 2010
Posts: 36
Rep Power: 7
A.Devesa is on a distinguished road
Hi Tommaso

I assume, if you're a beginner, you would like to run in parallel but first on one machine only. In this case, you need nothing about 1, 2 and 3. You just will have to decompose your case with a decomposeParDict, suppose on 2 processors if you possess a dualcore, on 4 if its a quadcore, and run your case with the command foamJob -p name_of_solver.

Only if you want to run on several machines at a time, you'll need:
1- that the machines know each other on the network (with help of your /etc/hosts), and connect you without password...
2- a file on the machine you will start the simulation, called e.g. machines and looking like:
machine1 cpu=2 (if you want to run on 2 CPUS on machine1)
machine2 cpu=2
3- the pathes will only be needed if you are not using the same absolute path for your simulation, e.g. on one machine computing on /scratch1/, on the other, on /scratch2/. But i have no experience with particular point.

Hope that helps!
Antoine
A.Devesa is offline   Reply With Quote

Old   April 8, 2010, 03:40
Default turbDyMFoam and parallel running
  #8
Member
 
Aldo Iannetti
Join Date: Feb 2010
Posts: 48
Rep Power: 7
aldo.iannetti is on a distinguished road
Hi foamers,
Did you ever use turbDyMFoam? Well, I have to run it in parallel (rotor-stator application with sliding interfaces). I have an error message saying that there's a non conformal master-slave zone or something like that.
Can someone tell me the detailed workflow of how to set up my job?
P.S.
I have no problem in serial running
aldo.iannetti is offline   Reply With Quote

Old   April 8, 2010, 03:48
Default
  #9
Member
 
Antoine Devesa
Join Date: Mar 2010
Posts: 36
Rep Power: 7
A.Devesa is on a distinguished road
Aldo,
i'm sorry, i had this problem once 2 months ago or so, and i can't remember what was wrong, but it was something in the definition of my dynamicDict.

Well i would say you have to follow roughly these steps:
- merge your static and moving meshes
- create sets for your static and moving interfaces (faceSet)
- transform them to zones (setsToZones)
- adapt your polymesh/boundary with ggi type (try overlap true in your case, if its not yet the case) as well as your 0/ initial conditions
- then decompose your case, preserving the zones of GGI
- run

may be some more experienced users could help you further, i'm rather new to OF and GGI...
A.Devesa is offline   Reply With Quote

Old   April 9, 2010, 15:53
Default Pararllel Running
  #10
Member
 
Join Date: Nov 2009
Posts: 48
Rep Power: 7
farhagim is on a distinguished road
Thanks for your answer.. YEs I want to run on Network at a time.

1-Can you explain alittle more about /etc/hosts... I can not Find the etc/hosts file or directory in my OpenFoam directory..
2- where Should I put the names of the machines and Cpu...in specific file???
does it have special Format?? and Do i have to put the File in specific directory??

Do I have to Install the OpenFoam on All the machiness ???

Thanks again for your help!!

MEhran




Quote:
Originally Posted by A.Devesa View Post
Hi Tommaso

I assume, if you're a beginner, you would like to run in parallel but first on one machine only. In this case, you need nothing about 1, 2 and 3. You just will have to decompose your case with a decomposeParDict, suppose on 2 processors if you possess a dualcore, on 4 if its a quadcore, and run your case with the command foamJob -p name_of_solver.



Only if you want to run on several machines at a time, you'll need:
1- that the machines know each other on the network (with help of your /etc/hosts), and connect you without password...
2- a file on the machine you will start the simulation, called e.g. machines and looking like:
machine1 cpu=2 (if you want to run on 2 CPUS on machine1)
machine2 cpu=2
3- the pathes will only be needed if you are not using the same absolute path for your simulation, e.g. on one machine computing on /scratch1/, on the other, on /scratch2/. But i have no experience with particular point.

Hope that helps!
Antoine
farhagim is offline   Reply With Quote

Old   April 9, 2010, 16:22
Default
  #11
Member
 
Antoine Devesa
Join Date: Mar 2010
Posts: 36
Rep Power: 7
A.Devesa is on a distinguished road
1- /etc/hosts has nothing to do with Foam, its just a file on your machine, where you define which others machines are know or belonging to the network. But on this point, i'm not an expert at all...

2- the file containing the hosts where you want to run, must only be at one single place, namely the place where you start your job from. In this file, you just need the name of the machine and then the number of cpus you want to use on it. Nothing else! So refer to my previous post, what i wrote is really the content of this file. Then you just give this file as an option of your command line: mpirun --hostfile File your_Foam_executable -parallel

Of course the executable must be accessible on every machine, using the same command. so you have to install foam on every machine, or to access your executable through your network fileserver...

Good luck!
A.Devesa is offline   Reply With Quote

Reply

Tags
parallel writeinterval

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
running OpenFoam in parallel vishwa OpenFOAM 22 Yesterday 08:53
Running dieselFoam in parallel. Palminchi OpenFOAM 0 February 17, 2010 05:00
Kubuntu uses dash breaks All scripts in tutorials platopus OpenFOAM Bugs 8 April 15, 2008 07:52
User fortran error when running CFX-10 in parallel CFDworker CFX 2 October 26, 2007 16:01
running multiple Fluent parallel jobs Michael Bo Hansen FLUENT 8 June 7, 2006 08:52


All times are GMT -4. The time now is 06:40.