CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Bugs

Particle Tracking brokes at processor interface

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   March 11, 2010, 11:57
Default Particle Tracking brokes at processor interface
  #1
Member
 
Franco Marra
Join Date: Mar 2009
Location: Napoli - Italy
Posts: 52
Rep Power: 8
francescomarra is on a distinguished road
Dear all,

I suspect a bug in the procedure that determines the transfer of particles between processor zones: as reported in the figure attached, particles un-physically accumulates on processor patch boundaries near solid walls.
The case regards a parallel simulation in a cylindrical channel with a strong swirl that drives the particles injected near the axis towards the walls.
In the figure two processor zones to highlight the problem are also indicated: accumulation of particles clearly happens along the processor zone boundaries.

Any hint on this problem or the indication of the routines mostly involved will be greatly appreciated.

Thank you in advance,

Franco Marra
Attached Images
File Type: png Particle_ProcInterface1.PNG (69.8 KB, 54 views)
francescomarra is offline   Reply With Quote

Old   March 11, 2010, 12:20
Default
  #2
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Do you have a testcase you can post?
mattijs is offline   Reply With Quote

Old   March 11, 2010, 13:38
Default
  #3
Member
 
Franco Marra
Join Date: Mar 2009
Location: Napoli - Italy
Posts: 52
Rep Power: 8
francescomarra is on a distinguished road
Dear Mattijs,

thank you so much for the so fast reply.

I do not know how exactly I can post the test case. This simulation is relatively big (about 220'000 cells, the same order of magnitude of particles) and is running with 64 processors.
Probably this is not a testcase to immediately reproduce the problem. However, I can certainly send the necessary files to restart the run from the time step the figure represents.
Otherwise I should try to reproduce the problem on a much smaller testcase.

I can just add that I already have done several runs with the same code, slightly modified to include a source term in the momentum equation to simulate a periodic channel with particles. Also running in parallel, this problem never arose, but in this case processor patch interfaces are always aligned with the coordinate directions.

Let me know if you like I send you the files necessary to initialize and run the case together with the last time step to immediately see the problem.

Thank you and best regards,

Franco
francescomarra is offline   Reply With Quote

Old   March 11, 2010, 14:56
Default
  #4
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Can you email it to me? m dot janssens at opencfd.co.uk
mattijs is offline   Reply With Quote

Old   March 12, 2010, 04:09
Default
  #5
Member
 
Andrew King
Join Date: Mar 2009
Location: Perth, Western Australia, Australia
Posts: 81
Rep Power: 8
andersking is on a distinguished road
We are also having problems with particles getting 'stuck' at processor boundaries.

It seems to be worse when communication is across more than one node, as well as being affected by the grid size. We have also seen particles getting stuck well away from walls.

As an illustration, this shows the stuck particles (in blue), as well as some of the processor boundaries.

Any help appreciated.

Regards,
Andrew
Attached Images
File Type: jpg tmp.jpg (85.8 KB, 73 views)
__________________
Dr Andrew King
Fluid Dynamics Research Group
Curtin University
andersking is offline   Reply With Quote

Old   March 15, 2010, 09:42
Default
  #6
New Member
 
Andrew Lowe
Join Date: Mar 2010
Location: Perth, Australia
Posts: 8
Rep Power: 7
BlueyTheDog is on a distinguished road
My name is Andrew Lowe and I'm working with Andrew King, the previous post, on this problem, in fact I was the one who found it in our work. If anyone has any questions regarding this programme feature with regards to our work, or would like me to test any proposed fixes, please feel free to contact me.

Andrew
BlueyTheDog is offline   Reply With Quote

Old   March 16, 2010, 06:14
Default
  #7
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Dear Andrews,

If you could post / send to my email (see above) a simple testcase that replicates the problem we can have a look at it.

The problem could be interpolation, truncation errors, mesh concavity or any other detail.

Regards,

Mattijs
mattijs is offline   Reply With Quote

Old   March 16, 2010, 07:26
Default
  #8
Super Moderator
 
niklas's Avatar
 
Niklas Nordin
Join Date: Mar 2009
Location: Stockholm, Sweden
Posts: 693
Rep Power: 20
niklas will become famous soon enoughniklas will become famous soon enough
I'd also like your case.

If I run on a nice blockMesh-generated mesh, I cant reproduce the problem.

If I use my utility perturbPoints to make the mesh nastier, I am able to produce
particles that gets stuck and I think the problem is related with this, the mesh quality.

continuing to investigate...
niklas is offline   Reply With Quote

Old   March 16, 2010, 09:50
Default
  #9
New Member
 
Fiorenzo Ambrosino
Join Date: Mar 2010
Location: Naples, Italy
Posts: 5
Rep Power: 7
ambros is on a distinguished road
Hi all, I'm Fiorenzo Ambrosino and I'm working with Franco Marra on the topic he posted previously.
In fact the problem of the stuck particles between processor zones is not present in the other case that Franco mentioned. In this latter case the mesh size is very small and we get a good mesh quality. In the case reported in the figure posted by Franco the mesh is very coarse.
So, for our experience in this kind of simulations, the problem could be related to the mesh quality.

Regards,
Fiorenzo
ambros is offline   Reply With Quote

Old   March 16, 2010, 10:29
Default
  #10
Member
 
Andrew King
Join Date: Mar 2009
Location: Perth, Western Australia, Australia
Posts: 81
Rep Power: 8
andersking is on a distinguished road
I will try to prepare a small test case that highlights the problems.

Andrew
__________________
Dr Andrew King
Fluid Dynamics Research Group
Curtin University
andersking is offline   Reply With Quote

Old   March 18, 2010, 08:04
Default
  #11
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Any luck creating a testcase?

Niklas: any success?
mattijs is offline   Reply With Quote

Old   March 18, 2010, 08:33
Default
  #12
Super Moderator
 
niklas's Avatar
 
Niklas Nordin
Join Date: Mar 2009
Location: Stockholm, Sweden
Posts: 693
Rep Power: 20
niklas will become famous soon enoughniklas will become famous soon enough
Im re-writing the tracking completly. A new algorithm called 'incredibly inefficient but straightforward',
which is sort of similar to what the previous algorithm was with triangle-decomposition, since
its dependent on the inCell-functionality, which I also rewrote to handle all types of cells.

works like this.
Code:
while (move)
{
    if ( inCell(endPosition) )
        move to endPosition
    else
        move to the face found in inCell
        switch to new cell
}
but as usual its the fiddling of how to treat positions on the face.

So no success yet, but I can feel im close, hrmm.....
niklas is offline   Reply With Quote

Old   March 23, 2010, 04:01
Default
  #13
Member
 
Andrew King
Join Date: Mar 2009
Location: Perth, Western Australia, Australia
Posts: 81
Rep Power: 8
andersking is on a distinguished road
Hi Mattijs,

I've have made a test case and attached it. To build the mesh use the makemesh script (wouldn't fit within the 97kB limit otherwise). the decomposeParDict is set for 16 processors, but it should display the problem with 4, (or maybe two).

Looking at it more closely, the particles seem to get stuck where the processor boundary is stairstepped, as in the attached image.

Hope this can shed some light on the issue.

Cheers,
Andrew
Attached Images
File Type: png stair.png (6.2 KB, 36 views)
Attached Files
File Type: gz stuck_particle_test2.tar.gz (3.8 KB, 13 views)
__________________
Dr Andrew King
Fluid Dynamics Research Group
Curtin University
andersking is offline   Reply With Quote

Old   March 23, 2010, 05:31
Default
  #14
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Hi Andrew,

got the case. What solver do I run?
mattijs is offline   Reply With Quote

Old   June 5, 2011, 12:30
Default
  #15
Member
 
edison
Join Date: May 2009
Location: Australia
Posts: 33
Rep Power: 8
Edison_Ge is on a distinguished road
Quote:
Originally Posted by mattijs View Post
Hi Andrew,

got the case. What solver do I run?

Run into same problem when trying to implement a PDF method in OF. Any progress with the fix?
My experience is that the more complicated the processorpatch are generated the more likely the stuck particle to happen.
I'm using the latest 1.7.1 and dying to know if anyone has found a solution.

Edison
Edison_Ge is offline   Reply With Quote

Old   June 15, 2011, 09:47
Default
  #16
New Member
 
Fiorenzo Ambrosino
Join Date: Mar 2010
Location: Naples, Italy
Posts: 5
Rep Power: 7
ambros is on a distinguished road
Hi, looking at the code in detail I figured out that there were a problem on calculating the face that the particle has to cross.
Findface function use lambda function to evaluate which face has to be crossed. Lambda takes into account the wallImpactDistance, the distance between particle center and particle surface that impact with a wall (= particle radius for sphere particle). I've changed lambda for fixing two bugs: the first fix is for considering particle radius bigger than the minimal distance between cell center and a wall; the second for preventing particles to stuck on the wall or processor/cyclic patch.

For the first bug I've removed the use of cell center instead of starting particle position; This seems to have no bad consequences for planar cell faces.
For the second bug, that is our case, in lambda the application of wallImpactDistance is applied for all kind of patch (also processor, cyclic and so on) not only for the wall.
So I changed lambda to do:

Code:
    if (!cloud_.internalFace(facei))
    {
        scalar CCf = mag((from - Cf) & Sf);

        label patchi = patch(facei);
        const polyPatch& patch = mesh.boundaryMesh()[patchi];
        if (isA<wallPolyPatch>(patch))
        {
            if ( CCf > static_cast<const ParticleType&>(*this).wallImpactDistance(Sf) )
            {
                  Cf -= static_cast<const ParticleType&>(*this)
                   .wallImpactDistance(Sf)*Sf;
            }
        }
    }
    scalar lambdaNominator = (Cf - from) & Sf;
    scalar lambdaDenominator = (to - from) & Sf;
I no longer have these bugs.

Hope I was helpful.

Fiorenzo
ambros is offline   Reply With Quote

Old   June 15, 2011, 10:14
Default
  #17
Super Moderator
 
niklas's Avatar
 
Niklas Nordin
Join Date: Mar 2009
Location: Stockholm, Sweden
Posts: 693
Rep Power: 20
niklas will become famous soon enoughniklas will become famous soon enough
Quote:
Originally Posted by ambros View Post
For the first bug I've removed the use of cell center instead of starting particle position; This seems to have no bad consequences for planar cell faces.
Ooooo, if you have done what I think, this is not good.
This means that if the particle is lost it will never find it again.

As you say, on planar cell faces this will be less likely, but (and this is a big but, for convex cells)
this is used to screen which faces to check, since if the particle is inside the cell, the path from cell
centre to end position will cross the same faces as the path from particle start position to end position,
but if it is outside the cell, or even on border, this will no longer be true.

so this can then be used as a check for lost particle, since if cell centre -> end says we have face hit
and particle start -> end says no face hit, we know we have a problem.

now, it will just continue to move the particle and you will not see a problem cause you cant find out.

N
niklas is offline   Reply With Quote

Old   June 15, 2011, 10:56
Default
  #18
New Member
 
Fiorenzo Ambrosino
Join Date: Mar 2010
Location: Naples, Italy
Posts: 5
Rep Power: 7
ambros is on a distinguished road
Hi Niklas, you're right but if I 've all planar faces I can't loose particles (or am I wrong?). I understand that blockMesh utility makes all planar faces, at least the internals, so I think that it's not a big mistake for blockMesh generated meshes (again, I'm not sure so feel free to correct me if I'm wrong).
BUT (and I think this is a big one but), this way, I fix the problem of rebounding particles with radius bigger than distance between cellcenter-wall, say it d1, that have starting_position inside the border cell and distance from wall bigger than d1. In this way it is solved also the case of rebounding particle in whatever cell not near wall when the radius is bigger than distance between thatcellcenter-wall.

I'll reread your useful publication "Particle tracking in unstructured, arbitrary polyhedral meshes for use in CFD and molecular dynamics" for better understand your argument in case I messed up.

Fiorenzo
ambros is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
particle tracking Wenqing Zhang CFX 15 August 3, 2013 06:02
Number density tracking rather than particle tracking Rebecca Main CFD Forum 2 April 23, 2009 12:52
DPM UDF particle position using the macro P_POS(p)[i] dm2747 FLUENT 0 April 17, 2009 01:29
Particle tracking - Domain interface / FrozenRotor mohanrao CFX 4 January 23, 2008 04:39
restarting lagrange (particle tracking) simulation dbdias CFX 0 September 22, 2007 19:26


All times are GMT -4. The time now is 21:51.