|
[Sponsors] |
June 21, 2012, 08:10 |
Load balancing with dynamic mesh
|
#1 |
New Member
Timo Kulju
Join Date: Aug 2009
Posts: 22
Rep Power: 17 |
Hi!
I implemnetd mesh refinement from interDyMFoam to multiphaseEulerFoam and it seems to work well. But after a while I'm stuck with load balancing issues. Does anybody know, is there a way to redistribute the mesh during the simulation? Or if I have to code it, does anybody have idea where to start? Thanks, Timo |
|
June 21, 2012, 16:42 |
|
#2 | |
Senior Member
Kyle Mooney
Join Date: Jul 2009
Location: San Francisco, CA USA
Posts: 323
Rep Power: 18 |
Quote:
|
||
June 25, 2012, 02:23 |
|
#3 |
New Member
Timo Kulju
Join Date: Aug 2009
Posts: 22
Rep Power: 17 |
Thanks Kyle, I'll try to start from that one.
- Timo |
|
June 26, 2012, 10:20 |
|
#4 | |
New Member
Timo Kulju
Join Date: Aug 2009
Posts: 22
Rep Power: 17 |
Quote:
- Timo |
||
June 28, 2012, 07:11 |
|
#5 | |
New Member
Timo Kulju
Join Date: Aug 2009
Posts: 22
Rep Power: 17 |
Quote:
- Timo |
||
October 2, 2012, 17:32 |
|
#6 |
Senior Member
Kyle Mooney
Join Date: Jul 2009
Location: San Francisco, CA USA
Posts: 323
Rep Power: 18 |
Timo,
I am currently working in coupling interDyMFoam to fvMeshDistribute. I can properly load balance the mesh, however, the refinement engine seems to get confused about the old-to-new cell refinement levels after the balance. Were you able to fix this? |
|
October 3, 2012, 06:27 |
|
#7 |
New Member
Timo Kulju
Join Date: Aug 2009
Posts: 22
Rep Power: 17 |
Kyle,
I didn't get it actually to work. It hanged after repartition... With manual repartition the problem was the same. If I remember correctly, it had something to do with refinementHistory, which probably needs also to be updated. If you have any ideas how it could be fixed, I would be curious to know. - Timo |
|
October 3, 2012, 10:13 |
|
#8 |
Senior Member
Kyle Mooney
Join Date: Jul 2009
Location: San Francisco, CA USA
Posts: 323
Rep Power: 18 |
I have not figured it out quite yet. The refinementHistory class as a ::distribute(mapDisributePolyMesh) member function, as does the hexRef8 class which actually calls refinementHistory::distribute(...).
I added some extra pointers to dyanmicFvMesh and hacked the redistributePar source to load the dynamic meshes. This way, after the load balance I can send the processor-processor map to the mesh cutter and related structures as such: in redistributePar.C ~line 1000 Code:
// Do actual sending/receiving of mesh autoPtr<mapDistributePolyMesh> map = distributor.distribute(finalDecomp); mesh.meshCutter().distribute(map()); Code:
forAll(map, i) { subField[i] = field[map[i]]; } I've attempted to use the cellLevel volScalarField that is dumped out for post processing but without the pointLevel field I don't think the hexRef8 engine knows what to do with it. I attempted to map a pointLevel pointScalarField but unfortunately I couldn't get it to map during redistribution. |
|
October 3, 2012, 11:29 |
|
#9 |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128 |
Greetings to all!
This bug report might interest you guys: http://www.openfoam.org/mantisbt/view.php?id=638 Try using the latest 2.1.x and see if you still have the same problem! Best regards, Bruno
__________________
|
|
October 3, 2012, 11:42 |
|
#10 | |
Senior Member
Kyle Mooney
Join Date: Jul 2009
Location: San Francisco, CA USA
Posts: 323
Rep Power: 18 |
Quote:
I am indeed using the latest and greatest 2.1.x repo version. This bug report contains some useful discussion on mapping, however, my problem surrounds load balancing which appears to demand quite a lot of subtle processor communication. I will try and gain some insight from it regardless. I appreciate your help! Cheers, Kyle |
||
May 25, 2017, 10:28 |
Big demand on the feature of dynamic load balance
|
#11 |
New Member
Xinze
Join Date: Mar 2009
Location: Shenyang, China
Posts: 15
Rep Power: 17 |
I am not sure whether you have got the solution or any update for this problem. Recently I also encountered this neck of performance, especially using ~DyMFoam. So I want to active this thread again.
I find this codes but it seems exist some questions. https://github.com/tgvoskuilen/meshBalancing https://bugs.openfoam.org/view.php?id=2461 https://bugs.openfoam.org/view.php?id=638 If possible, anyone may update the latest progress on this problem. XZ |
|
May 25, 2017, 13:25 |
|
#12 | |
Senior Member
Kyle Mooney
Join Date: Jul 2009
Location: San Francisco, CA USA
Posts: 323
Rep Power: 18 |
Quote:
|
||
May 26, 2017, 00:04 |
|
#13 |
New Member
Xinze
Join Date: Mar 2009
Location: Shenyang, China
Posts: 15
Rep Power: 17 |
@kmooney
Thank you for you constructive information. I will try to adapt this trick. Addition, there two clues deposit here and is possible useful for improving the feature: (1) https://bugs.openfoam.org/view.php?id=2461 PIMPLE: iteration 1 Selected 119 cells for refinement out of 115612. Refined from 115612 to 116445 cells. Selected 217 split points out of a possible 10399. Unrefined from 116445 to 114926 cells. Maximum imbalance = 27.2854 % Re-balancing dynamically refined mesh Selecting decompositionMethod ptscotch (1): ERROR: dgraphFold2: out of memory (2) (2): ERROR: dgraphFold2: out of memory (2) (4): ERROR: dgraphFold2: out of memory (2) This bug can be tracked on here: https://github.com/poulson/scotch/bl.../dgraph_fold.c fldvelolocnbr = (orggrafptr->veloloctax != NULL) ? fldvertlocnbr : 0; if (memAllocGroup ((void **) (void *) /* Allocate distributed graph public data */ &fldgrafptr->vertloctax, (size_t) ((fldvertlocnbr + 1) * sizeof (Gnum)), &fldgrafptr->vnumloctax, (size_t) ( fldvertlocnbr * sizeof (Gnum)), &fldgrafptr->veloloctax, (size_t) ( fldvelolocnbr * sizeof (Gnum)), NULL) == NULL) { errorPrint ("dgraphFold2: out of memory (2)"); cheklocval = 1; } (2) Alternatively, it maybe solve these problems if update to a new parallel partitioning libraray, such as Zoltan. http://www.cs.sandia.gov/~kddevin/pa...dagstuhl09.pdf http://www.cs.sandia.gov/zoltan/Zoltan.html XZ |
|
Tags |
dynamic mesh, load balance, parallel, refine mesh |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Dynamic Mesh | kennyboy | FLUENT | 1 | February 23, 2019 02:52 |
Local mesh refinement definition in a DEFORMING dynamic mesh zone using Dynamic Mesh | Emanuele88 | FLUENT | 0 | February 9, 2016 12:39 |
dynamic mesh for drop interface | IndrajitW | FLUENT | 0 | March 30, 2013 09:03 |
dynamic mesh refinement and rhoCentralFoam | ChrisA | OpenFOAM Running, Solving & CFD | 1 | March 21, 2013 09:00 |
Dynamic Mesh moving interface help | akash.iitb | FLUENT | 0 | August 24, 2010 00:53 |