[dasbrot]

Hi everyone,

I have run into a problem regarding parallel computing in OpenFOAM.

As a background (I can't see how it would matter, but nevertheless..) I have set up a multi-region case where I have two solids and one fluid. I've written a script which does - in short - the following:

1. create background mesh from UNV file
2. decompose mesh with redistributePar
3. snappyHexMesh
4. create cell sets for regions
5. reconstruct mesh with redistributePar
6. surfaceToPatch
7. decompose mesh with redistributePar
8. ...
9. run case

The step with surface to patch is neccessary in order to avoid creating a domain0-region, you can find a couple of post here regarding that problem. Surface to patch only runs in serial mode that is why I have to reconstruct the case again.

Everything is working fine in serial mode, with 2 and also with 3 cores. But when I try to run it on 4 cases, step 5 - the reconstructing - fails with the message

--> FOAM FATAL IO ERROR: 
Cannot find patchField entry for procBoundary3to0

The problem is, there simply is none and there never was a boundary from processor 0 to 3, because they don't share any faces.

Processor 0
    Number of cells = 27324
    Number of faces shared with processor 1 = 1188
    Number of faces shared with processor 2 = 759
    Number of processor patches = 2
    Number of processor faces = 1947
    Number of boundary faces = 3603

Processor 1
    Number of cells = 27324
    Number of faces shared with processor 0 = 1188
    Number of faces shared with processor 3 = 759
    Number of processor patches = 2
    Number of processor faces = 1947
    Number of boundary faces = 3603

Processor 2
    Number of cells = 27324
    Number of faces shared with processor 0 = 759
    Number of faces shared with processor 3 = 1188
    Number of processor patches = 2
    Number of processor faces = 1947
    Number of boundary faces = 3603

Processor 3
    Number of cells = 27324
    Number of faces shared with processor 1 = 759
    Number of faces shared with processor 2 = 1188
    Number of processor patches = 2
    Number of processor faces = 1947
    Number of boundary faces = 3603

(the boundaries for i.e. processor0 to processor1 and processor2 are created as expected)

So, the question is, is this a bug, which only happens in rare cases and has not been adressed yet, or is there another solution? I did only find this post but I already added the include statement.

Thanks a lot and cheers

Edit: replaced reconstructPar with redistributePar (which I was already using but I mixed things up posting this)

[simrego]

Hi!


You shouldn't be able to use reconstructPar after snappy. reconstructPar is only for field reconstruction, but since your mesh is changed (probably changed after snappy or snappy did nothing) you have to reconstruct the mesh too.
You should use reconstructParMesh and then reconstructPar, or you can use simply redistributePar with the -reconstruct switch which will do the same but in one run.


Also I have no idea how did you decomposed your case with reconstructPar, but you should decompose with decomposePar.

[dasbrot]

Hi simrego,

thanks for your answer! I'm sorry, I mixed things up, I'm already using redistributePar, editing my post now ..

Thank you anyways, cheers!

[dasbrot]

In case anyone stumbles over this issue, it was already solved and answered here, but I did not find it before (e.g. in this post).

The error lies within snappyHexMesh, which overwrites the processor boundaries. For example, if the decomposing led to a boundary procBoundary0to1, snappyHexMesh would overwrite it to e.g. procBoundary0to2. So the reconstructing (and the cell set creation before) would fail due to the missing boundary.

The solution to this problem lies in overwriting all the boundary files in the '0'-folder in all processor*-directories with empty files, e.g. for temperature

...
dimensions      [0 0 0 1 0 0 0];

internalField uniform 300;

boundaryField
{
	#include "$WM_PROJECT_DIR/etc/caseDicts/setConstraintTypes"
	
	".*"
    {
        type            calculated;
        value           $internalField;
    }
}

right after snappyHexMesh finished. This is done by the command

ls -d processor* | xargs -i cp -r 0.orig/* ./{}/0/ $1

which one might have seen before. The left part of the command simply lists all existent processor folders, which are then fed into the {} expression for the copy command on the right.

cheers