[bunni]

Hi,

Sorry if this is not the right place. I'm having a very strange problem with running openfoam with parallel processing. I've run the job with 1 processor, no problem. Then I run the job with 2 processors, it goes faster, the data's in the right place, no problem. Three processors, no problem. 4 processors - problem. I get no communication back from the program after about 4 or 5 minutes of run time. Nothing is updated to the logfile I redirect to, nothing is written to the processor? directories. But the jobs are still running. But, if I change the output write time (writeInterval in controlDict) to be a lesser value, I get no problem with 4 procs. So, somehow writeInterval changes whether or not the program hangs? any ideas as to what's going on?

[A.Devesa]

could you please post your controlDict and decomposeDict?

[bunni]

Sure,
(I've also included these as attachments with a txt at the end, due to the attachment manager).

It works fine for 2,3,4 processors for writeInterval =1e-6, and endTime 2e-5

but when the writeInterval is 1e-5 and endTime 1e-4, it seems to hang at (by looking at logfile) between t=7e-6 and 8e-6 - for 4 procs only. 2 or 3 work fine for 1e-5 and endTime 1e-4.

For 4 procs, I do a decomposition equally in x,y: (2,2,1). Again this works fine for writeInterval is 1e-6 and endTime 2e-5.

Cheers



controlDict :

FoamFile
{
version 2.0;
format ascii;
class dictionary;
location "system";
object controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application interFoam;

startFrom latestTime;

startTime 0;

stopAt endTime;

endTime 1e-04;

deltaT 1e-08;

writeControl adjustableRunTime;

writeInterval 1e-05;

purgeWrite 0;

writeFormat ascii;
writePrecision 8;

writeCompression uncompressed;

timeFormat general;

timePrecision 6;

runTimeModifiable yes;

adjustTimeStep yes;

maxCo 0.5;

maxDeltaT 1e-08;

/******/

decomposeParDict:

FoamFile
{
version 2.0;
format ascii;
class dictionary;
location "system";
object decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 3;

method simple;

simpleCoeffs
{
n ( 3 1 1 );
delta 0.001;
}

hierarchicalCoeffs
{
n ( 1 1 1 );
delta 0.001;
order xyz;
}

metisCoeffs
{
processorWeights ( 1 1 1 1 );
}

manualCoeffs
{
dataFile "";
}

distributed no;

roots ( );

[A.Devesa]

i'm quite a beginner with OF, but i noticed 2 things, that unfornately do have nothing to do with decomposition...
- you're maxDeltaT is exactly your initial DeltaT. this could may be cause unexpected behaviour.
- i don't know the key word "adjustableRunTime", but just "runTime". did you try this one? did you observe the same behaviour?

If this is not bringing you further, and if for 2 and 3 processors, you were just decomposing in x direction, you should try to investigate what would pose such a problem when you cut in y direction...

[bunni]

Hi,

The thing is, it does work for np=4 (and for n=(2 2 1) in dec..ParDict), just for a different writeInterval. (and endtime. I didn't keep the long endtime with the short writeInterval, too much data!).

Anyway, I will look into the other variables. I'm sure you're quite more accomplished in openFoam than I am.

[farhagim]

Hi there,

I am a beginner user of the OpenFoam...I really need to learn how to run openfoam with parallel processing. I read the all the available guideline in this site www.OpenFoam.com)...but I have basic problem with that. First of all I can not find "etc/hosts " in OpenFoam Directories where I should define the host names of the machines. Since I am not familiar with that kind of file
, I would be so grateful if anybody provide me one of these kind of files.

2. I read somewhere that I have to create a "file that contains the
host names of the machines. The file can be given any name and located at any path. " I could not understand what does it means ...or what kind of the files do i have to create....can anybody give a hint or example about that.

3. In decompose Dict, the roots entry is a list of root paths, <root0>, <root1>, . . . , for each node:
roots
<nRoots>
(
"<root0>"
"<root1>"
...
);

I need somebody help me throough this too..


Thank you very much guyss,
Tommaso

Originally Posted by bunni

Hi,

Sorry if this is not the right place. I'm having a very strange problem with running openfoam with parallel processing. I've run the job with 1 processor, no problem. Then I run the job with 2 processors, it goes faster, the data's in the right place, no problem. Three processors, no problem. 4 processors - problem. I get no communication back from the program after about 4 or 5 minutes of run time. Nothing is updated to the logfile I redirect to, nothing is written to the processor? directories. But the jobs are still running. But, if I change the output write time (writeInterval in controlDict) to be a lesser value, I get no problem with 4 procs. So, somehow writeInterval changes whether or not the program hangs? any ideas as to what's going on?

[A.Devesa]

Hi Tommaso

I assume, if you're a beginner, you would like to run in parallel but first on one machine only. In this case, you need nothing about 1, 2 and 3. You just will have to decompose your case with a decomposeParDict, suppose on 2 processors if you possess a dualcore, on 4 if its a quadcore, and run your case with the command foamJob -p name_of_solver.

Only if you want to run on several machines at a time, you'll need:
1- that the machines know each other on the network (with help of your /etc/hosts), and connect you without password...
2- a file on the machine you will start the simulation, called e.g. machines and looking like:
machine1 cpu=2 (if you want to run on 2 CPUS on machine1)
machine2 cpu=2
3- the pathes will only be needed if you are not using the same absolute path for your simulation, e.g. on one machine computing on /scratch1/, on the other, on /scratch2/. But i have no experience with particular point.

Hope that helps!
Antoine

[aldo.iannetti]

Hi foamers,
Did you ever use turbDyMFoam? Well, I have to run it in parallel (rotor-stator application with sliding interfaces). I have an error message saying that there's a non conformal master-slave zone or something like that.
Can someone tell me the detailed workflow of how to set up my job?
P.S.
I have no problem in serial running

[A.Devesa]

Aldo,
i'm sorry, i had this problem once 2 months ago or so, and i can't remember what was wrong, but it was something in the definition of my dynamicDict.

Well i would say you have to follow roughly these steps:
- merge your static and moving meshes
- create sets for your static and moving interfaces (faceSet)
- transform them to zones (setsToZones)
- adapt your polymesh/boundary with ggi type (try overlap true in your case, if its not yet the case) as well as your 0/ initial conditions
- then decompose your case, preserving the zones of GGI
- run

may be some more experienced users could help you further, i'm rather new to OF and GGI...

[farhagim]

Thanks for your answer.. YEs I want to run on Network at a time.

1-Can you explain alittle more about /etc/hosts... I can not Find the etc/hosts file or directory in my OpenFoam directory..
2- where Should I put the names of the machines and Cpu...in specific file???
does it have special Format?? and Do i have to put the File in specific directory??

Do I have to Install the OpenFoam on All the machiness ???

Thanks again for your help!!

MEhran

Originally Posted by A.Devesa

Hi Tommaso

I assume, if you're a beginner, you would like to run in parallel but first on one machine only. In this case, you need nothing about 1, 2 and 3. You just will have to decompose your case with a decomposeParDict, suppose on 2 processors if you possess a dualcore, on 4 if its a quadcore, and run your case with the command foamJob -p name_of_solver.



Only if you want to run on several machines at a time, you'll need:
1- that the machines know each other on the network (with help of your /etc/hosts), and connect you without password...
2- a file on the machine you will start the simulation, called e.g. machines and looking like:
machine1 cpu=2 (if you want to run on 2 CPUS on machine1)
machine2 cpu=2
3- the pathes will only be needed if you are not using the same absolute path for your simulation, e.g. on one machine computing on /scratch1/, on the other, on /scratch2/. But i have no experience with particular point.

Hope that helps!
Antoine

[A.Devesa]

1- /etc/hosts has nothing to do with Foam, its just a file on your machine, where you define which others machines are know or belonging to the network. But on this point, i'm not an expert at all...

2- the file containing the hosts where you want to run, must only be at one single place, namely the place where you start your job from. In this file, you just need the name of the machine and then the number of cpus you want to use on it. Nothing else! So refer to my previous post, what i wrote is really the content of this file. Then you just give this file as an option of your command line: mpirun --hostfile File your_Foam_executable -parallel

Of course the executable must be accessible on every machine, using the same command. so you have to install foam on every machine, or to access your executable through your network fileserver...

Good luck!