Problem with Dynamic Load Balacing via PyFoam
 

I am trying to implement a so to say load balancing by reconstructing and decomposing every output time during an InterDyMFoam run with dynamicMeshRefinement based on alpha.

My problems are the reconstructParMesh and probably the decomposePar Utilities do not treat respective refinementHistory and pointLevel outputs for the dynamicRefineFvMesh class.

Does anyone have an Idea how to implement the necessary changes to these Utilities?

Edit: I just realized that instead of reconstruct+decomposepar one can just take redistributePar. But the Problem stays the same. I need the labelList pointlevel in the polymesh folder redistributed along with all the other Fields such as u or p hoping that the dynamicrefinemesh class doesnt refine already refined cells anylonger, without the need to have the possibilty of coarsening these cells again, as refinementHistory is a tough output to redistribute. I would greatly appreciate any input.

OK. I don't see the connection to PyFoam (that's what made me notice the thread). Probably you're using it for writing the driver-script.

Anyway: are you aware of the utility redistributeParMesh? Haven't used it myself, but in the source I read "Redistribute existing decomposed mesh ... " and "... balances mesh". That sounds like what you want to do in a single package

As I stated (unclearly, sorry about that) in my thread, the problem is that neither redistributePar, nor reconstructPar(Mesh) take the generated outputs of the dynamicRefineMesh class with them, namely pointLevel and refinementHistory, located in the processorX/polymesh folders. Without this information the automatic Mesh refinement starts again at given refined mesh, and refines it further, generating way to much cells each loop in phython.

My planned phython loop looks like that:

-decompose mesh

-solver solves to first outputtime with dynamic mesh refinement

-mesh reconstructed, endTime is increased

-decompose new Mesh with more cells -> loadbalancing

-solver solves to increased endtime .. and so forth

Those utilities only decompose/reconstruct proper GeometricFields. So what might be a possibility would be to "fill" those informations into appropriate geometric fields (volScalarField/pointScalarField). Then the reconstruct/decompose (or the resdistribute) will transfer them. Afterwards create the labelLists and go on

Or you take the sources of these utilities, create your own variant that reads and redistributes these labelListrs (at some point in these utilities there are lists available that describe the mappings of cells/points form/to parallel/serial (they're needed for distributing the GeometricField)

Or: you write your own dynamicMesh-class (by copy/paste/adapt from resdistributePar) that basically extends the one you're currently using to redistribute everything in memory from time to time

But I'm afraid all of these solutions involve some C++-programming (maybe someone here has a better idea)

Thanks for your input. I guess sooner or later you have to program something, my hope was that this something already exists out there.

I'm currently working on extending load balancer to be able to map the refinementHistory from one distribution to another. I'll let you know if we get it working!

Here is some previous discussion on the matter:
http://www.cfd-online.com/Forums/ope...tion-maps.html

Any news?
 

I am wondering if you managed to solve the issue. I read other similar threads without success. I am using version 2.3.x.

Summary of problem:
If you run a dynamically refined solver in parallel (example interdymfoam), then it is not possible to reconstruct the case properly. The reason is that the refinement associated files (cellLevel, pointLevel, refinementHistory) cannot be reconstructed by the command 'reconstructparMesh'. This leads to re-refinement of already refined cells, if solver runs from latest time.

Is there any steps that i am missing? I haven't tested the redistributepar command yet. But it would be silly, if we could redistribute the refined mesh properly, but not reconstruct it.

I understand there may be limitations to the code but why are you reconstructing your case only to decompose it again?

I have also tested redistributepar, but it also fails to process refinement files.

I would like to reconstruct the case for several reasons;
If reconstruct process works fine, then I can decompose again for "manual" load balancing. Also I could change the number of processors.

Moreover, it would be easier to make changes in boundary conditions and internal fields on reconstructed case. One could easily add a new field. Also archiving becomes easier. But these are minor issues, since one can hopefully handle them with simple scripts.