CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Meshing & Mesh Conversion (http://www.cfd-online.com/Forums/openfoam-meshing/)
-   -   Load balancing with dynamic mesh (http://www.cfd-online.com/Forums/openfoam-meshing/103602-load-balancing-dynamic-mesh.html)

tikulju June 21, 2012 07:10

Load balancing with dynamic mesh
 
Hi!
I implemnetd mesh refinement from interDyMFoam to multiphaseEulerFoam and it seems to work well. But after a while I'm stuck with load balancing issues. Does anybody know, is there a way to redistribute the mesh during the simulation? Or if I have to code it, does anybody have idea where to start?

Thanks,
Timo

kmooney June 21, 2012 15:42

Quote:

Originally Posted by tikulju (Post 367622)
Hi!
I implemnetd mesh refinement from interDyMFoam to multiphaseEulerFoam and it seems to work well. But after a while I'm stuck with load balancing issues. Does anybody know, is there a way to redistribute the mesh during the simulation? Or if I have to code it, does anybody have idea where to start?

Thanks,
Timo

I believe that the fvMeshDistribute library might do it for you but its implementation might not be so straightforward. The source code is in the dynamicMesh sub directories.

tikulju June 25, 2012 01:23

Thanks Kyle, I'll try to start from that one.

- Timo

tikulju June 26, 2012 09:20

Quote:

Originally Posted by kmooney (Post 367722)
I believe that the fvMeshDistribute library might do it for you but its implementation might not be so straightforward. The source code is in the dynamicMesh sub directories.

I managed to do it somehow by applying redistributePar code. But the mesh is not the same as is the used in a simulation (it is read from the directory). Any idea how to operate with the mesh, which is used in the main code (i.e. dynamicFvMesh and apparently stored in memory)?

- Timo

tikulju June 28, 2012 06:11

Quote:

Originally Posted by tikulju (Post 368369)
I managed to do it somehow by applying redistributePar code. But the mesh is not the same as is the used in a simulation (it is read from the directory). Any idea how to operate with the mesh, which is used in the main code (i.e. dynamicFvMesh and apparently stored in memory)?

- Timo

Ok, I ended up using the redistributePar-code by modifying it a bit. Now my implementation involves writing and reading of mesh and fields to and from HD, but I couldn't make up any wiser way to do it. But now it works anyway :)

- Timo

kmooney October 2, 2012 16:32

Timo,

I am currently working in coupling interDyMFoam to fvMeshDistribute. I can properly load balance the mesh, however, the refinement engine seems to get confused about the old-to-new cell refinement levels after the balance.

Were you able to fix this?

tikulju October 3, 2012 05:27

Kyle,

I didn't get it actually to work. It hanged after repartition... With manual repartition the problem was the same. If I remember correctly, it had something to do with refinementHistory, which probably needs also to be updated. If you have any ideas how it could be fixed, I would be curious to know.

- Timo

kmooney October 3, 2012 09:13

I have not figured it out quite yet. The refinementHistory class as a ::distribute(mapDisributePolyMesh) member function, as does the hexRef8 class which actually calls refinementHistory::distribute(...).

I added some extra pointers to dyanmicFvMesh and hacked the redistributePar source to load the dynamic meshes. This way, after the load balance I can send the processor-processor map to the mesh cutter and related structures as such:

in redistributePar.C ~line 1000
Code:

    // Do actual sending/receiving of mesh
    autoPtr<mapDistributePolyMesh> map = distributor.distribute(finalDecomp);

    mesh.meshCutter().distribute(map());

I get a seg fault at line 282 in mapDistributeTemplates.C at
Code:

                    forAll(map, i)
                    {
                        subField[i] = field[map[i]];
                    }

The segfault occurs when its preparing to send the cellLevel integer lists to processor neighbors. This is getting a little deep into mpi comms for my programming skills so it will take me some time to proceed at this point.

I've attempted to use the cellLevel volScalarField that is dumped out for post processing but without the pointLevel field I don't think the hexRef8 engine knows what to do with it. I attempted to map a pointLevel pointScalarField but unfortunately I couldn't get it to map during redistribution.

wyldckat October 3, 2012 10:29

Greetings to all!

This bug report might interest you guys: http://www.openfoam.org/mantisbt/view.php?id=638
Try using the latest 2.1.x and see if you still have the same problem!

Best regards,
Bruno

kmooney October 3, 2012 10:42

Quote:

Originally Posted by wyldckat (Post 384737)
Greetings to all!

This bug report might interest you guys: http://www.openfoam.org/mantisbt/view.php?id=638
Try using the latest 2.1.x and see if you still have the same problem!

Best regards,
Bruno

Hi Bruno,

I am indeed using the latest and greatest 2.1.x repo version. This bug report contains some useful discussion on mapping, however, my problem surrounds load balancing which appears to demand quite a lot of subtle processor communication. I will try and gain some insight from it regardless. I appreciate your help!

Cheers,

Kyle


All times are GMT -4. The time now is 06:25.