CFD Online Discussion Forums - Mesh gets out of sync during parallel 2D adaptive mesh refinement

Dear Co-Foamers,

I am working on a 2D adaptive mesh refinement and managed to create a dynamicFvMesh using some of OpenFOAM's high-level mesh operations. It runs fine and gives good results (I successfully validated a backward-facing step) but as soon as I try to run it in parallel, it won't work anymore. Maybe somebody understands why my code does not work in parallel and knows how I could make it work? I believe I need to synchronize some information between the processors after the first refinement step but I don't know exactly which information to transfer.

Let me become a bit more precise about what I've done so far:

I'm doing mesh refinement in 2 passes, one for each direction (say x and y). The first step works fine but when it comes to the second step it fails. It appears to me that the meshes have become out of sync at the coupled processor boundaries. One processor does not know which cells are connected from its neighbour processor over the coupled patch. I'm getting errors like

Code:

[4] --> FOAM FATAL ERROR: 

[4] in patch:procBoundary4to3 : Local size of patch is 62 (faces).

Received from neighbour 64 faceCentres!

Code:

[2] Cannot find point in pts1 matching point 3 

coord:(0.0015 0.001761904762 0) in pts0 

when using tolerance 2.670915379e-08

The code when reduced to the essential looks as follows

Code:

bool Foam::dynamic2DRefineFvMesh::update()

{

    // I left out some unnecessary code

    // all variables used for the x-refinement are marked with "1"

    // the variables for the y-refinement have a "2"



    // the list of cell labels which we want to refine later

    labelList refCells1;



    // actually determine the cells we want to refine

    determineRefinementCells(refCells1);



    // a list of "refineCells" which holds the same labels as refCells1 but together with a vector

    List<refineCell> refineCellList1;



    // first refinement in x direction

    const vector vec1 = (1, 0, 0); 



    // copy refCells1 into refineCellList1

    transferToRefineCellList(refCells1, refineCellList1, vec1);



    // create a cellLooper for the mesh

    autoPtr<cellLooper> cellLooper1(new geomCellLooper(*this));



    // create a meshCutter, undoability switched off

    undoableMeshCutter undoableMeshCutter1(*this, false)



    // create a polyTopoChange which will perform the actual mesh modifications

    polyTopoChange polyTopoChange1(*this);



    // construct cellCuts for our refinement cells

    cellCuts cc1(*this, cellLooper1(), refineCellList1);



    // play the refinement operation into the polyTopoChange

    undoableMeshCutter1.setRefinement(cc1, polyTopoChange1);



    // create a map which holds the information for the mesh changes

    autoPtr<mapPolyMesh> map1 = polyTopoChange1.changeMesh(*this, false);



    // update fields, works fine this time but won't work for the second direction

    updateMesh(map1);



    //-- x-direction is done. y-direction is analogous

    // between both steps we need to find out the new labels of the cells which have been refined in step 1 so we know which cells we must still refine in step 2 (the labels will have changed)

    // the code will crash as soon as I call updateMesh(map2) (or even earlier if I try other things like parallel redistributing)



    // ...

}

I'd be very happy if anybody could point me in the right direction. Thank you in advance for any support!