CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Meshing & Mesh Conversion

Load balancing with dynamic mesh

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By kmooney

Reply
 
LinkBack Thread Tools Display Modes
Old   June 21, 2012, 07:10
Default Load balancing with dynamic mesh
  #1
New Member
 
Timo Kulju
Join Date: Aug 2009
Posts: 21
Rep Power: 7
tikulju is on a distinguished road
Hi!
I implemnetd mesh refinement from interDyMFoam to multiphaseEulerFoam and it seems to work well. But after a while I'm stuck with load balancing issues. Does anybody know, is there a way to redistribute the mesh during the simulation? Or if I have to code it, does anybody have idea where to start?

Thanks,
Timo
tikulju is offline   Reply With Quote

Old   June 21, 2012, 15:42
Default
  #2
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 268
Rep Power: 8
kmooney is on a distinguished road
Quote:
Originally Posted by tikulju View Post
Hi!
I implemnetd mesh refinement from interDyMFoam to multiphaseEulerFoam and it seems to work well. But after a while I'm stuck with load balancing issues. Does anybody know, is there a way to redistribute the mesh during the simulation? Or if I have to code it, does anybody have idea where to start?

Thanks,
Timo
I believe that the fvMeshDistribute library might do it for you but its implementation might not be so straightforward. The source code is in the dynamicMesh sub directories.
kmooney is offline   Reply With Quote

Old   June 25, 2012, 01:23
Default
  #3
New Member
 
Timo Kulju
Join Date: Aug 2009
Posts: 21
Rep Power: 7
tikulju is on a distinguished road
Thanks Kyle, I'll try to start from that one.

- Timo
tikulju is offline   Reply With Quote

Old   June 26, 2012, 09:20
Default
  #4
New Member
 
Timo Kulju
Join Date: Aug 2009
Posts: 21
Rep Power: 7
tikulju is on a distinguished road
Quote:
Originally Posted by kmooney View Post
I believe that the fvMeshDistribute library might do it for you but its implementation might not be so straightforward. The source code is in the dynamicMesh sub directories.
I managed to do it somehow by applying redistributePar code. But the mesh is not the same as is the used in a simulation (it is read from the directory). Any idea how to operate with the mesh, which is used in the main code (i.e. dynamicFvMesh and apparently stored in memory)?

- Timo
tikulju is offline   Reply With Quote

Old   June 28, 2012, 06:11
Default
  #5
New Member
 
Timo Kulju
Join Date: Aug 2009
Posts: 21
Rep Power: 7
tikulju is on a distinguished road
Quote:
Originally Posted by tikulju View Post
I managed to do it somehow by applying redistributePar code. But the mesh is not the same as is the used in a simulation (it is read from the directory). Any idea how to operate with the mesh, which is used in the main code (i.e. dynamicFvMesh and apparently stored in memory)?

- Timo
Ok, I ended up using the redistributePar-code by modifying it a bit. Now my implementation involves writing and reading of mesh and fields to and from HD, but I couldn't make up any wiser way to do it. But now it works anyway

- Timo
tikulju is offline   Reply With Quote

Old   October 2, 2012, 16:32
Default
  #6
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 268
Rep Power: 8
kmooney is on a distinguished road
Timo,

I am currently working in coupling interDyMFoam to fvMeshDistribute. I can properly load balance the mesh, however, the refinement engine seems to get confused about the old-to-new cell refinement levels after the balance.

Were you able to fix this?
kmooney is offline   Reply With Quote

Old   October 3, 2012, 05:27
Default
  #7
New Member
 
Timo Kulju
Join Date: Aug 2009
Posts: 21
Rep Power: 7
tikulju is on a distinguished road
Kyle,

I didn't get it actually to work. It hanged after repartition... With manual repartition the problem was the same. If I remember correctly, it had something to do with refinementHistory, which probably needs also to be updated. If you have any ideas how it could be fixed, I would be curious to know.

- Timo
tikulju is offline   Reply With Quote

Old   October 3, 2012, 09:13
Default
  #8
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 268
Rep Power: 8
kmooney is on a distinguished road
I have not figured it out quite yet. The refinementHistory class as a ::distribute(mapDisributePolyMesh) member function, as does the hexRef8 class which actually calls refinementHistory::distribute(...).

I added some extra pointers to dyanmicFvMesh and hacked the redistributePar source to load the dynamic meshes. This way, after the load balance I can send the processor-processor map to the mesh cutter and related structures as such:

in redistributePar.C ~line 1000
Code:
    // Do actual sending/receiving of mesh
    autoPtr<mapDistributePolyMesh> map = distributor.distribute(finalDecomp);

    mesh.meshCutter().distribute(map());
I get a seg fault at line 282 in mapDistributeTemplates.C at
Code:
                    forAll(map, i)
                    {
                        subField[i] = field[map[i]];
                    }
The segfault occurs when its preparing to send the cellLevel integer lists to processor neighbors. This is getting a little deep into mpi comms for my programming skills so it will take me some time to proceed at this point.

I've attempted to use the cellLevel volScalarField that is dumped out for post processing but without the pointLevel field I don't think the hexRef8 engine knows what to do with it. I attempted to map a pointLevel pointScalarField but unfortunately I couldn't get it to map during redistribution.
hua1015 likes this.
kmooney is offline   Reply With Quote

Old   October 3, 2012, 10:29
Default
  #9
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,253
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings to all!

This bug report might interest you guys: http://www.openfoam.org/mantisbt/view.php?id=638
Try using the latest 2.1.x and see if you still have the same problem!

Best regards,
Bruno
wyldckat is online now   Reply With Quote

Old   October 3, 2012, 10:42
Default
  #10
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 268
Rep Power: 8
kmooney is on a distinguished road
Quote:
Originally Posted by wyldckat View Post
Greetings to all!

This bug report might interest you guys: http://www.openfoam.org/mantisbt/view.php?id=638
Try using the latest 2.1.x and see if you still have the same problem!

Best regards,
Bruno
Hi Bruno,

I am indeed using the latest and greatest 2.1.x repo version. This bug report contains some useful discussion on mapping, however, my problem surrounds load balancing which appears to demand quite a lot of subtle processor communication. I will try and gain some insight from it regardless. I appreciate your help!

Cheers,

Kyle
kmooney is offline   Reply With Quote

Reply

Tags
dynamic mesh, load balance, parallel, refine mesh

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Incylinder dynamic mesh with volumetric reaction mas FLUENT 4 May 3, 2012 10:22
Dynamic mesh update problem. David FLUENT 3 March 15, 2012 06:02
pls help. mesh collapsed with dynamic mesh. wlt_1985 FLUENT 1 July 28, 2011 01:53
Dynamic Mesh moving interface help akash.iitb FLUENT 0 August 23, 2010 23:53
Dynamic mesh + grid adapt = Crash! (Files included BillH FLUENT 4 July 24, 2007 15:31


All times are GMT -4. The time now is 06:57.