[timbojones]

All,

I'm running the incompressibleVoF solver module with adaptive mesh refinement in OpenFOAM 11. As the calculation proceeds, I'd like to rebalance the mesh distribution to make the best use of parallel processing. I've tried using both the "distributor" and "loadBlanacer". The snippet below is from my dynamicMeshDict, and it shows how I've defined the "loadBalancer."

distributor
{
    type                    loadBalancer;

    libs                    ("libfvMeshDistributors.so");

    // How often to redistribute
    redistributeInterval    10;

    // Maximum fractional cell distribution imbalance
    // before rebalancing
    maxImbalance            0.1;
}

The "distributor" setup is the same, but the type entry is changed to "distributor." Neither of these types works for me. If I use "distributor", the case crashes telling me that I need to define internal patches. If I use the "loadBalancer", the case crashes telling me that cpu loads have not been recorded.

Encouraged a bit by the results from "loadBalancer", I tried to activate the cpuLoads debug switch in my controlDict, but this didn't do anything.

Is there anyone who has succeeded in dynamically redistributing the mesh as part of an adaptive mesh refinement case? If so, I'd be very grateful for instructions, advice, or anything that might be helpful to me.

Thanks,
TimboJones

[aeronerd]

I don't have a direct answer for you, but was searching how to best implement this in an upcoming simulation. These may be useful to us:

https://github.com/OpenFOAM/OpenFOAM...83fc482040a37c


It does seem odd that the "demo case" referenced by that description does not include the maxImbalance parameter:

https://github.com/OpenFOAM/OpenFOAM...ynamicMeshDict

[Kellis]

Hi Timbo,

I've been encountering the same issue with my simulations. Did you ever find a solution?

I've attached a simple test case that replicates the error you are encountering. It's a solid block heated from the top with the mesh set to refine within a certain temperature range. I'm trying to use the 3rd party Zoltan library to redistribute, but am always getting this error:

Courant Number mean: 2.13896e-17 max: 1.2843e-15
Selected 0 cells for refinement out of 8000.
Selected 0 split points out of a possible 900.
Redistributing mesh with imbalance 0.875
refinementHistory add : found refinementHistory
refinementHistory add : adding refinementHistory 
refinementHistory apply : found refinementHistory
refinementHistory apply : adding refinementHistory 
[2] 
[2] 
[2] --> FOAM FATAL ERROR: 
[2] Cannot find a internal patch in 
4
(
top
defaultFaces
procBoundary2to0
procBoundary2to3
)

    of types 
4
(
wall
patch
processor
processor
)

    An internal patch must be provided for the exposed internal faces.
[2] 
[2]     From function Foam::label Foam::fvMeshDistribute::findInternalPatch() const
[2]     in file fvMeshDistribute/fvMeshDistribute.C at line 385.
[2] 
FOAM parallel run exiting

The floatingObject tutorial case works fine for me with the zoltan balancing, I'm not sure where the problem lies exactly. I am trying to learn the new modular solver system which is complicating my debugging efforts. Any input would be appreciated.

Thanks,
Kellis

[Kellis]

Following up on this after inspecting the blockMeshDict in the floatingObject tutorial case, which has the entry:

    internal
    {
        type internal;
        faces ();
    }

Adding this to my test case eliminated the error described above and it seems to work properly now. Attached is the functional case setup for use with the Zoltan load balancing package. If I've overlooked something in the setup, please let me know, but I think it is working as expected. I tested the case runtimes with the distributor type entry in dynamicMeshDict set to "none" and "distributor," with the following results:

none: 231s
distributor: 122s

This seems like a dramatic improvement, but I set the case up as a worst-case scenario where all of the refinement happens on a physical boundary away from the initial processor boundaries.

The optionalCpuLoad class used for the loadBalancing type only appears to get called by chemistryModel (and even then only when the flag is set), so I don't think you'll be able to use it in a solver that doesn't have chemistry enabled unless you want to do some coding. Check out the multicomponentFluid / counterFlowFlame2D_GRI tutorial for an example on that one.