[stru]

Hello,

I want to run a chtMultiRegionFoam case in parallel. To decompose this case im using

decomposePar -allRegions

and everything is fine, if i want to split my case to 4 processes like in the tutorials. Now I want to change the number of subdomains like this:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.4.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 4 -> 8;

method          scotch;

If I run now decomposePar -alllRegions I get only 4 subdomains per region and there isn't an error message in the log file.

Create time

Decomposing all regions in regionProperties



Decomposing mesh fluid1

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 6.94 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 1034188
    Number of faces shared with processor 1 = 4896
    Number of faces shared with processor 3 = 5955
    Number of processor patches = 2
    Number of processor faces = 10851
    Number of boundary faces = 273919

Processor 1
    Number of cells = 1031395
    Number of faces shared with processor 0 = 4896
    Number of faces shared with processor 2 = 3133
    Number of faces shared with processor 3 = 2245
    Number of processor patches = 3
    Number of processor faces = 10274
    Number of boundary faces = 273788

Processor 2
    Number of cells = 1033559
    Number of faces shared with processor 1 = 3133
    Number of faces shared with processor 3 = 5202
    Number of processor patches = 2
    Number of processor faces = 8335
    Number of boundary faces = 272371

Processor 3
    Number of cells = 1039273
    Number of faces shared with processor 0 = 5955
    Number of faces shared with processor 1 = 2245
    Number of faces shared with processor 2 = 5202
    Number of processor patches = 3
    Number of processor faces = 13402
    Number of boundary faces = 282097

Number of processor faces = 21431
Max number of cells = 1039273 (0.451308% above average 1034604)
Max number of processor patches = 3 (20% above average 2.5)
Max number of faces between processors = 13402 (25.07116% above average 10715.5)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer


Decomposing mesh fluid2

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 1.77 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 507416
    Number of faces shared with processor 1 = 156
    Number of processor patches = 1
    Number of processor faces = 156
    Number of boundary faces = 382890

Processor 1
    Number of cells = 509015
    Number of faces shared with processor 0 = 156
    Number of faces shared with processor 3 = 168
    Number of processor patches = 2
    Number of processor faces = 324
    Number of boundary faces = 383122

Processor 2
    Number of cells = 511338
    Number of faces shared with processor 3 = 188
    Number of processor patches = 1
    Number of processor faces = 188
    Number of boundary faces = 385024

Processor 3
    Number of cells = 510299
    Number of faces shared with processor 1 = 168
    Number of faces shared with processor 2 = 188
    Number of processor patches = 2
    Number of processor faces = 356
    Number of boundary faces = 385170

Number of processor faces = 512
Max number of cells = 511338 (0.3573973% above average 509517)
Max number of processor patches = 2 (33.33333% above average 1.5)
Max number of faces between processors = 356 (39.0625% above average 256)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer


Decomposing mesh solid

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 0.78 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 227051
    Number of faces shared with processor 1 = 78
    Number of faces shared with processor 2 = 71
    Number of processor patches = 2
    Number of processor faces = 149
    Number of boundary faces = 137585

Processor 1
    Number of cells = 226546
    Number of faces shared with processor 0 = 78
    Number of processor patches = 1
    Number of processor faces = 78
    Number of boundary faces = 137340

Processor 2
    Number of cells = 225765
    Number of faces shared with processor 0 = 71
    Number of faces shared with processor 3 = 71
    Number of processor patches = 2
    Number of processor faces = 142
    Number of boundary faces = 136506

Processor 3
    Number of cells = 225724
    Number of faces shared with processor 2 = 71
    Number of processor patches = 1
    Number of processor faces = 71
    Number of boundary faces = 136687

Number of processor faces = 220
Max number of cells = 227051 (0.3444976% above average 226271.5)
Max number of processor patches = 2 (33.33333% above average 1.5)
Max number of faces between processors = 149 (35.45455% above average 110)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.

My decomposeParDict is located in the system folder of the case like in the tutorials of this solver.

Can anybody tell me what I'm missing?
Thanks a lot.
Stefan

[vabishek]

Originally Posted by stru

Hello,

I want to run a chtMultiRegionFoam case in parallel. To decompose this case im using

and everything is fine, if i want to split my case to 4 processes like in the tutorials. Now I want to change the number of subdomains like this:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.4.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 4 -> 8;

method          scotch;

If I run now decomposePar -alllRegions I get only 4 subdomains per region and there isn't an error message in the log file.

Create time

Decomposing all regions in regionProperties



Decomposing mesh fluid1

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 6.94 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 1034188
    Number of faces shared with processor 1 = 4896
    Number of faces shared with processor 3 = 5955
    Number of processor patches = 2
    Number of processor faces = 10851
    Number of boundary faces = 273919

Processor 1
    Number of cells = 1031395
    Number of faces shared with processor 0 = 4896
    Number of faces shared with processor 2 = 3133
    Number of faces shared with processor 3 = 2245
    Number of processor patches = 3
    Number of processor faces = 10274
    Number of boundary faces = 273788

Processor 2
    Number of cells = 1033559
    Number of faces shared with processor 1 = 3133
    Number of faces shared with processor 3 = 5202
    Number of processor patches = 2
    Number of processor faces = 8335
    Number of boundary faces = 272371

Processor 3
    Number of cells = 1039273
    Number of faces shared with processor 0 = 5955
    Number of faces shared with processor 1 = 2245
    Number of faces shared with processor 2 = 5202
    Number of processor patches = 3
    Number of processor faces = 13402
    Number of boundary faces = 282097

Number of processor faces = 21431
Max number of cells = 1039273 (0.451308% above average 1034604)
Max number of processor patches = 3 (20% above average 2.5)
Max number of faces between processors = 13402 (25.07116% above average 10715.5)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer


Decomposing mesh fluid2

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 1.77 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 507416
    Number of faces shared with processor 1 = 156
    Number of processor patches = 1
    Number of processor faces = 156
    Number of boundary faces = 382890

Processor 1
    Number of cells = 509015
    Number of faces shared with processor 0 = 156
    Number of faces shared with processor 3 = 168
    Number of processor patches = 2
    Number of processor faces = 324
    Number of boundary faces = 383122

Processor 2
    Number of cells = 511338
    Number of faces shared with processor 3 = 188
    Number of processor patches = 1
    Number of processor faces = 188
    Number of boundary faces = 385024

Processor 3
    Number of cells = 510299
    Number of faces shared with processor 1 = 168
    Number of faces shared with processor 2 = 188
    Number of processor patches = 2
    Number of processor faces = 356
    Number of boundary faces = 385170

Number of processor faces = 512
Max number of cells = 511338 (0.3573973% above average 509517)
Max number of processor patches = 2 (33.33333% above average 1.5)
Max number of faces between processors = 356 (39.0625% above average 256)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer


Decomposing mesh solid

Create mesh

Calculating distribution of cells
Selecting decompositionMethod scotch

Finished decomposition in 0.78 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 227051
    Number of faces shared with processor 1 = 78
    Number of faces shared with processor 2 = 71
    Number of processor patches = 2
    Number of processor faces = 149
    Number of boundary faces = 137585

Processor 1
    Number of cells = 226546
    Number of faces shared with processor 0 = 78
    Number of processor patches = 1
    Number of processor faces = 78
    Number of boundary faces = 137340

Processor 2
    Number of cells = 225765
    Number of faces shared with processor 0 = 71
    Number of faces shared with processor 3 = 71
    Number of processor patches = 2
    Number of processor faces = 142
    Number of boundary faces = 136506

Processor 3
    Number of cells = 225724
    Number of faces shared with processor 2 = 71
    Number of processor patches = 1
    Number of processor faces = 71
    Number of boundary faces = 136687

Number of processor faces = 220
Max number of cells = 227051 (0.3444976% above average 226271.5)
Max number of processor patches = 2 (33.33333% above average 1.5)
Max number of faces between processors = 149 (35.45455% above average 110)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.

My decomposeParDict is located in the system folder of the case like in the tutorials of this solver.

Can anybody tell me what I'm missing?
Thanks a lot.
Stefan

Hello Stefan,

I believe your case has three regions i.e. fluid1, fluid2 and solid. In order for your decomposeparDict file to work correctly you need to make sure that the contents of decomposeparDict in system/decomposeparDict are the same as the ones in the subfolders i.e. system/fluid1/decomposeparDict , system/fluid2/decomposeparDict and system/solid/decomposeparDict. So, whenever you change the decomposeparDict file in system/decomposeparDict, you can just copy the file and paste it in the sub-folders (system/<region name>/).

Hope that helps!

[stru]

Now it's working.

Thanks vabishek.