[dzordz]

Dear foamers,

I am dealing with a transient problem which needs adaptive meshing to manage the calculation. The issue is that the number of cells grows enormously from the start of the simulation till the end. Therefore the decomposition I make via decomposePar and scotch pre-run (time 0) is not gonna be an optimal after some elapsed time in fact the calculation time needed from one writing to the other will grow due to the fact that particular core will receive more cells and the others will have to wait for it to finish. Ultimately this problem grows with time.

My question is the following. Does there exist an utility that would redistribute cells during the run of the simulation and where one can specify at what intervals redistribution should happen? I can't seem to find such capability anywhere.

At the moment I have a bash code that stops simulation multiple times at specific intervals, redistributes the cells and reruns the sim. But the times have to be put in manually and since they are different for different simulations this becomes unmanageable.

My idea would be to somehow put it in controlDict as sort of a codded function that would basically do the following (it would literally call the utilities at specific times):

1) every n-th writeInterval do:
2) reconstructParMest --> saves your mesh for each time step
3) reconstructPar --> saves the fields for each time step
4) decompose -force --> decomposes mesh & fields for latest time step
5) continue with simulation with more optimal distribution

I have seen libraries and codes being called in controlDict but not utilities. Is this possible?

Thanks!

[massive_turbulence]

Originally Posted by dzordz

Dear foamers,

I am dealing with a transient problem which needs adaptive meshing to manage the calculation. The issue is that the number of cells grows enormously from the start of the simulation till the end. Therefore the decomposition I make via decomposePar and scotch pre-run (time 0) is not gonna be an optimal after some elapsed time in fact the calculation time needed from one writing to the other will grow due to the fact that particular core will receive more cells and the others will have to wait for it to finish. Ultimately this problem grows with time.

My question is the following. Does there exist an utility that would redistribute cells during the run of the simulation and where one can specify at what intervals redistribution should happen? I can't seem to find such capability anywhere.

At the moment I have a bash code that stops simulation multiple times at specific intervals, redistributes the cells and reruns the sim. But the times have to be put in manually and since they are different for different simulations this becomes unmanageable.

My idea would be to somehow put it in controlDict as sort of a codded function that would basically do the following (it would literally call the utilities at specific times):

1) every n-th writeInterval do:
2) reconstructParMest --> saves your mesh for each time step
3) reconstructPar --> saves the fields for each time step
4) decompose -force --> decomposes mesh & fields for latest time step
5) continue with simulation with more optimal distribution

I have seen libraries and codes being called in controlDict but not utilities. Is this possible?

Thanks!

Sounds like load balancing? See if this thread helps Run-Time Parallel Load Balancing

I only know of the term, never used more than Scotch myself.

[dzordz]

If anyone is interested I solved the load balancing problem and incorporated it in ofv1812 with an easy added function in controlDict. This functionality does not compare the loads and decide when to re-balance based on criteria given (would need a more complex coding, but I do not need this at the moment). It just re-balances it on user specified bases every n-th write time.

functions
{
redistribution
{
    libs        ("libutilityFunctionObjects.so");

    type coded;
    writeControl    writeTime;
    writeInterval     2;
    name redistribution;
    codeWrite
    #{
    system ("reconstructParMesh");
    system ("reconstructPar");    
    system ("decomposePar -force");
    #};
}
}

Hopefully something like this will be useful to someone.


UPDATE: This seems to NOT work at large number of processors. The re-composition and decomposition are also made many times (basically equals to number of processors set). All this results in some time steps missing fields files. This would be workable if somehow system ("reconstructParMesh"); could be forced to be performed on 1 processors, while others do nothing. Is such a thing possible?

Cheers