CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Programming & Development

Problem with Dynamic Load Balacing via PyFoam

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   December 11, 2012, 07:44
Default Problem with Dynamic Load Balacing via PyFoam
  #1
New Member
 
Waldemar Hübert
Join Date: Sep 2011
Posts: 6
Rep Power: 6
knalldi is on a distinguished road
I am trying to implement a so to say load balancing by reconstructing and decomposing every output time during an InterDyMFoam run with dynamicMeshRefinement based on alpha.

My problems are the reconstructParMesh and probably the decomposePar Utilities do not treat respective refinementHistory and pointLevel outputs for the dynamicRefineFvMesh class.

Does anyone have an Idea how to implement the necessary changes to these Utilities?

Edit: I just realized that instead of reconstruct+decomposepar one can just take redistributePar. But the Problem stays the same. I need the labelList pointlevel in the polymesh folder redistributed along with all the other Fields such as u or p hoping that the dynamicrefinemesh class doesnt refine already refined cells anylonger, without the need to have the possibilty of coarsening these cells again, as refinementHistory is a tough output to redistribute. I would greatly appreciate any input.

Last edited by knalldi; December 11, 2012 at 10:29.
knalldi is offline   Reply With Quote

Old   December 11, 2012, 13:52
Default
  #2
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,920
Rep Power: 41
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by knalldi View Post
I am trying to implement a so to say load balancing by reconstructing and decomposing every output time during an InterDyMFoam run with dynamicMeshRefinement based on alpha.

My problems are the reconstructParMesh and probably the decomposePar Utilities do not treat respective refinementHistory and pointLevel outputs for the dynamicRefineFvMesh class.

Does anyone have an Idea how to implement the necessary changes to these Utilities?

Edit: I just realized that instead of reconstruct+decomposepar one can just take redistributePar. But the Problem stays the same. I need the labelList pointlevel in the polymesh folder redistributed along with all the other Fields such as u or p hoping that the dynamicrefinemesh class doesnt refine already refined cells anylonger, without the need to have the possibilty of coarsening these cells again, as refinementHistory is a tough output to redistribute. I would greatly appreciate any input.
OK. I don't see the connection to PyFoam (that's what made me notice the thread). Probably you're using it for writing the driver-script.

Anyway: are you aware of the utility redistributeParMesh? Haven't used it myself, but in the source I read "Redistribute existing decomposed mesh ... " and "... balances mesh". That sounds like what you want to do in a single package
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   December 11, 2012, 14:30
Default
  #3
New Member
 
Waldemar Hübert
Join Date: Sep 2011
Posts: 6
Rep Power: 6
knalldi is on a distinguished road
As I stated (unclearly, sorry about that) in my thread, the problem is that neither redistributePar, nor reconstructPar(Mesh) take the generated outputs of the dynamicRefineMesh class with them, namely pointLevel and refinementHistory, located in the processorX/polymesh folders. Without this information the automatic Mesh refinement starts again at given refined mesh, and refines it further, generating way to much cells each loop in phython.

My planned phython loop looks like that:

-decompose mesh

-solver solves to first outputtime with dynamic mesh refinement

-mesh reconstructed, endTime is increased

-decompose new Mesh with more cells -> loadbalancing

-solver solves to increased endtime .. and so forth
knalldi is offline   Reply With Quote

Old   December 11, 2012, 15:26
Default
  #4
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,920
Rep Power: 41
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by knalldi View Post
As I stated (unclearly, sorry about that) in my thread, the problem is that neither redistributePar, nor reconstructPar(Mesh) take the generated outputs of the dynamicRefineMesh class with them, namely pointLevel and refinementHistory, located in the processorX/polymesh folders. Without this information the automatic Mesh refinement starts again at given refined mesh, and refines it further, generating way to much cells each loop in phython.

My planned phython loop looks like that:

-decompose mesh

-solver solves to first outputtime with dynamic mesh refinement

-mesh reconstructed, endTime is increased

-decompose new Mesh with more cells -> loadbalancing

-solver solves to increased endtime .. and so forth
Those utilities only decompose/reconstruct proper GeometricFields. So what might be a possibility would be to "fill" those informations into appropriate geometric fields (volScalarField/pointScalarField). Then the reconstruct/decompose (or the resdistribute) will transfer them. Afterwards create the labelLists and go on

Or you take the sources of these utilities, create your own variant that reads and redistributes these labelListrs (at some point in these utilities there are lists available that describe the mappings of cells/points form/to parallel/serial (they're needed for distributing the GeometricField)

Or: you write your own dynamicMesh-class (by copy/paste/adapt from resdistributePar) that basically extends the one you're currently using to redistribute everything in memory from time to time

But I'm afraid all of these solutions involve some C++-programming (maybe someone here has a better idea)
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   December 14, 2012, 07:12
Default
  #5
New Member
 
Waldemar Hübert
Join Date: Sep 2011
Posts: 6
Rep Power: 6
knalldi is on a distinguished road
Quote:
Originally Posted by gschaider View Post
Those utilities only decompose/reconstruct proper GeometricFields. So what might be a possibility would be to "fill" those informations into appropriate geometric fields (volScalarField/pointScalarField). Then the reconstruct/decompose (or the resdistribute) will transfer them. Afterwards create the labelLists and go on

Or you take the sources of these utilities, create your own variant that reads and redistributes these labelListrs (at some point in these utilities there are lists available that describe the mappings of cells/points form/to parallel/serial (they're needed for distributing the GeometricField)

Or: you write your own dynamicMesh-class (by copy/paste/adapt from resdistributePar) that basically extends the one you're currently using to redistribute everything in memory from time to time

But I'm afraid all of these solutions involve some C++-programming (maybe someone here has a better idea)
Thanks for your input. I guess sooner or later you have to program something, my hope was that this something already exists out there.
knalldi is offline   Reply With Quote

Old   December 19, 2012, 22:27
Default
  #6
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 319
Rep Power: 9
kmooney is on a distinguished road
I'm currently working on extending load balancer to be able to map the refinementHistory from one distribution to another. I'll let you know if we get it working!

Here is some previous discussion on the matter:
Parallel AMR Redistribution - agglomeration maps
kmooney is offline   Reply With Quote

Old   August 6, 2015, 05:40
Default Any news?
  #7
New Member
 
alp albayrak
Join Date: Apr 2014
Posts: 3
Rep Power: 4
alpalbayrak is on a distinguished road
I am wondering if you managed to solve the issue. I read other similar threads without success. I am using version 2.3.x.

Summary of problem:
If you run a dynamically refined solver in parallel (example interdymfoam), then it is not possible to reconstruct the case properly. The reason is that the refinement associated files (cellLevel, pointLevel, refinementHistory) cannot be reconstructed by the command 'reconstructparMesh'. This leads to re-refinement of already refined cells, if solver runs from latest time.

Is there any steps that i am missing? I haven't tested the redistributepar command yet. But it would be silly, if we could redistribute the refined mesh properly, but not reconstruct it.
alpalbayrak is offline   Reply With Quote

Old   August 7, 2015, 01:58
Default
  #8
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 319
Rep Power: 9
kmooney is on a distinguished road
I understand there may be limitations to the code but why are you reconstructing your case only to decompose it again?
kmooney is offline   Reply With Quote

Old   August 7, 2015, 08:19
Default
  #9
New Member
 
alp albayrak
Join Date: Apr 2014
Posts: 3
Rep Power: 4
alpalbayrak is on a distinguished road
I have also tested redistributepar, but it also fails to process refinement files.

I would like to reconstruct the case for several reasons;
If reconstruct process works fine, then I can decompose again for "manual" load balancing. Also I could change the number of processors.

Moreover, it would be easier to make changes in boundary conditions and internal fields on reconstructed case. One could easily add a new field. Also archiving becomes easier. But these are minor issues, since one can hopefully handle them with simple scripts.
alpalbayrak is offline   Reply With Quote

Reply

Tags
dynamicrefinefvmesh, load balacing, problem, pyfoam

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
dynamic mesh problem chris d. FLUENT 3 March 2, 2009 07:02
a problem about dynamic mesh satum FLUENT 2 June 25, 2008 01:37
Problem with Dynamic Mesh and Fluent Claude Main CFD Forum 0 December 12, 2006 07:03
Dynamic nature of problem Amit Kulkarni Main CFD Forum 0 November 9, 2002 18:15
Help: Flow-Response Dynamic Coupled problem wavefront Main CFD Forum 3 September 26, 2001 10:28


All times are GMT -4. The time now is 20:28.