CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Pre-Processing

Manual decomposition using setFields

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree14Likes
  • 14 Post By RL-S

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   September 28, 2017, 07:27
Default Manual decomposition using setFields
  #1
New Member
 
Lennart Steffen
Join Date: Mar 2017
Location: Braunschweig, Germany
Posts: 17
Rep Power: 4
RL-S is on a distinguished road
Hey there,

I found a couple of old threads about the manual option in decomposeParDict, and how to conveniently prepare the file for it with the setFields utility:

DecomposePar utility
Manually divide sub-domain for parallel computing

It was possible to piece the answer together from these threads, but it took me a couple of hours, so I wanted to share my solution here. There might be more elegant ways and I'm open to suggestions, but it works and it's fast enough.
First, I'll explain the problem, then I'll describe my solution.

The problem:

The manual option in the decomposeParDict file offers a lot of versatility, but is difficult to use, because you have to provide a file with a labelList the size of your mesh cell count. Each entry signifies the processor that cell is on. The exact file formatting is not completely trivial to find in the docs, but that's doable. However, actually creating that file manually doesn't really make sense, especially for larger meshes.
While the setFields utility provides a very convenient way of writing that list, its input- and output formatting is slightly different to the one for the needed file.

My solution:

Make sure your mesh is created and your system/decomposeParDict looks something like this:
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  4.x                                   |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    note        "mesh decomposition control dictionary";
    object      decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains  2;

method          scotch;

scotchCoeffs
{
}

manualCoeffs
{
    dataFile    "cellDist";
}

// ************************************************************************* //
Then open a terminal in your case directory and get your current decomposition file by executing
Code:
decomposePar -cellDist
That will write a file called cellDist in 0/.

Now we need to write our desired decomposition into the system/setFieldsDict. Just as an example, it can be sth like this:
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  1.7.1                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      setFieldsDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

defaultFieldValues ( volScalarFieldValue cellDist 0 );

regions 
(
      boxToCell 
      {
           box ( 0.5 0 0 ) ( 1 0.5 1 ) ;
           fieldValues ( volScalarFieldValue cellDist 1); 
      }
      boxToCell 
      {
           box ( 0 0.5 0 ) ( 0.5 1 1 ) ;
           fieldValues ( volScalarFieldValue cellDist 2); 
      }
      boxToCell 
      {
           box ( 0.5 0.5 0 ) ( 1 1 1 ) ;
           fieldValues ( volScalarFieldValue cellDist 3); 
      }
);


// ************************************************************************* //
For a cube with an edge length of 1, this would cut it into four regions of equal size. That's really just an example, and you'll know better what suits your case. In my case, I needed to put boundary patches on opposite sides of the mesh on the same processor, and that was easier with this method than with the others.

Now go back to your terminal and execute
Code:
setFields ; touch prepDecomp.py
This will rewrite your file cellDist in 0/ according to your desired decomposition. It also creates an empty file called prepDecomp.py (you may change the name if you want), which we open and paste the following into:
Code:
#read file
with open("0/cellDist", "r") as cdFile:
    lines = cdFile.readlines()

#replace and delete unwanted lines
lines[11] = lines[11].replace("volScalarField", "labelList")
lines[12] = lines[12].replace("0", "constant")
del lines[17:21]
bfline = 0
for i, line in enumerate(lines):
    if (line.find("boundaryField") != -1):
        bfline = i
        break
del lines[i:]

#write file
with open("constant/cellDist", "w") as cdFile:
    for line in lines:
        cdFile.write(line)
Now we execute that python script through the terminal with
Code:
python prepDecomp.py
This will modify the file according to the needed input format for the decomposePar utility.

Next, we set the keyword method in system/decomposeParDict to manual, and we're good to go.

Your workflow for designing your preferred decomposition may be this:

  1. Edit setFieldsDict
  2. Execute
    Code:
    rm -r processor* ; setFields ; python prepDecomp.py ; decomposePar
  3. Check the cell and boundary face counts and see whether it looks good. If yes, you may run your solver, if not, start over from 1.


I needed this, because I used a codedFixedValue BC that accessed a patch on another processor core. This way, I could just place them on the same core, at least as a stopgap solution.


Hope this might help someone.


Lennart

Edit: I didn't try this method with binary format, so if you run into problems, make sure the writeFormat keyword in system/controlDict is set to ascii for the whole process. That way you can at least see what's happening.
RL-S is offline   Reply With Quote

Old   February 7, 2018, 07:30
Default
  #2
New Member
 
Timm Feigel
Join Date: Sep 2017
Location: Braunschweig, Germany
Posts: 3
Rep Power: 4
Timm Feigel is on a distinguished road
Works perfectly, thank you!
Timm Feigel is offline   Reply With Quote

Old   March 20, 2018, 03:18
Default Works Perfectly.
  #3
b_k
New Member
 
Join Date: Aug 2009
Posts: 4
Rep Power: 12
b_k is on a distinguished road
Thanks for sharing. Works perfectly.
b_k is offline   Reply With Quote

Old   August 25, 2018, 01:18
Default Perfect!
  #4
New Member
 
Join Date: Mar 2016
Posts: 8
Rep Power: 5
Eldrael is on a distinguished road
Works great!


Thanks for the help!
Eldrael is offline   Reply With Quote

Old   October 2, 2018, 10:04
Smile
  #5
New Member
 
Mateusz
Join Date: Nov 2016
Posts: 1
Rep Power: 0
maTTe778 is on a distinguished road
It works great. Many thanks !
maTTe778 is offline   Reply With Quote

Reply

Tags
codedfixedvalue, decomposepar, manual decomposition, python, setfields

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Looking for an example/help with manual decomposition KTG OpenFOAM Pre-Processing 2 March 28, 2017 00:06
[snappyHexMesh] How to define to right point for locationInMesh Mirage12 OpenFOAM Meshing & Mesh Conversion 7 March 13, 2016 15:07
Manual Decomposition Method smraniaki OpenFOAM Pre-Processing 3 November 24, 2013 17:53
Manual decomposition of domain pss OpenFOAM Pre-Processing 0 April 26, 2012 02:33
Description file for manual decomposition A.Devesa OpenFOAM Running, Solving & CFD 2 July 4, 2011 09:09


All times are GMT -4. The time now is 02:44.