CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Post-Processing

Mass flow through faceZone using swak4foam when running in parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes

Reply
 
LinkBack Thread Tools Display Modes
Old   April 13, 2013, 09:33
Default Mass flow through faceZone using swak4foam when running in parallel
  #1
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
I try to obtain the mass flow through a faceZone with swak4foam using the following function in my controlDict:

Code:
    flowIntFluid
    {
        type swakExpression;
        valueType faceZone;
        region fluid_water;
        zoneName int_fluid;
        expression "phi*flip()";
        accumulations (
            sum
        );
        verbose true;
    }
I have created a cellSet for the slave cells of the faceZone with the following action in topoSetDict:

Code:
    {
        name    int_fluidSlaveCells;
        type    cellSet;
        action  new;
        source  faceZoneToCell;
        sourceInfo
        {
            name int_fluid;
            option slave;
        }
    }
Everything works okay when I run the simulation on a single processor however when running in parallel the expression returns garbage, which is probably because of the set information missing in the individual processor subdirectories.

Is it possible to distribute this set information to the individual directories?
billie is offline   Reply With Quote

Old   April 13, 2013, 11:52
Default
  #2
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
I figured out that I do not need swak4foam for this task, using the following function in controlDict does the same and works when running in parallel.

Code:
    flowIntFluid
    {
        type            faceSource;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   timeStep;
        region          fluid_water;
        log             true;
        valueOutput     false;
        source          faceZone;
        sourceName      int_fluid;
        operation       sum;

        fields
        (
            phi
        );
    }
billie is offline   Reply With Quote

Old   April 13, 2013, 16:12
Default
  #3
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
Everything works okay when I run the simulation on a single processor however when running in parallel the expression returns garbage, which is probably because of the set information missing in the individual processor subdirectories.

Is it possible to distribute this set information to the individual directories?
No. Sets are not decomposed. But what works is running the utilities that were used to create the sets on the parallel case
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 13, 2013, 16:13
Default
  #4
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
I figured out that I do not need swak4foam for this task, using the following function in controlDict does the same and works when running in parallel.

Code:
    flowIntFluid
    {
        type            faceSource;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   timeStep;
        region          fluid_water;
        log             true;
        valueOutput     false;
        source          faceZone;
        sourceName      int_fluid;
        operation       sum;

        fields
        (
            phi
        );
    }
Of course this result is only correct if all faces in the zone are oriented in the same way
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 13, 2013, 16:36
Default
  #5
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
No. Sets are not decomposed. But what works is running the utilities that were used to create the sets on the parallel case
Running topoSet on the decomosed case creates the sets but I get garbage output as well.
billie is offline   Reply With Quote

Old   April 13, 2013, 16:45
Default
  #6
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
Of course this result is only correct if all faces in the zone are oriented in the same way
As I said running on a single core the swak4foam expression works but not when running on multiple cores, even when the set is created on the parallel case.

Running funkyDoCalc on the reconstructed case with the following dictionary works as well:

Code:
massFlow
{
    valueType faceZone;
    zoneName int_fluid;
    accumulations (
        sum
    );
    expression "phi";
}
So the problem is not related to the face orientation.
billie is offline   Reply With Quote

Old   April 13, 2013, 17:32
Default
  #7
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
Of course this result is only correct if all faces in the zone are oriented in the same way
I checked the face normals in paraview and they are all oriented in the same way. So the slave cell set is not required in this case. I still do however get the wrong output when running in parallel.
billie is offline   Reply With Quote

Old   April 14, 2013, 05:53
Default
  #8
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
Running topoSet on the decomosed case creates the sets but I get garbage output as well.
Define "garbage". Anyway: there may be a corner case where the sums are not correct: if a face in the set is on a processor-boundary. In that case it may be counted twice or not at all
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 14, 2013, 06:01
Default
  #9
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
As I said running on a single core the swak4foam expression works but not when running on multiple cores, even when the set is created on the parallel case.

Running funkyDoCalc on the reconstructed case with the following dictionary works as well:

Code:
massFlow
{
    valueType faceZone;
    zoneName int_fluid;
    accumulations (
        sum
    );
    expression "phi";
}
So the problem is not related to the face orientation.
Then it worked OK in that case. But in general phi is determined by the order of the cells. Suppose you have a "channel" with four cells. Digits are are the cell numbers and letters are the faces in the face Set. Flow goes from left to right (the way it always does):

0 A 1
2 B 3

Here on A and B phi is positive. In another case

1 A 0
2 B 3

on A the phi is negative while on B it is still positive. Summing it up without multiplying it with the flip will give you a wrong result. That is what I mean with "orientation" and am not sure how you'd visualize this with paraview (as you mentioned in another posting)
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 14, 2013, 06:45
Default
  #10
Senior Member
 
immortality's Avatar
 
Ehsan
Join Date: Oct 2012
Location: Iran
Posts: 2,205
Rep Power: 17
immortality is on a distinguished road
what does flip do?
immortality is offline   Reply With Quote

Old   April 14, 2013, 08:03
Default
  #11
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
That is what I mean with "orientation" and am not sure how you'd visualize this with paraview (as you mentioned in another posting)
Hm, I did look up the the face normals in paraview using the NormalGlyphs filter plus the cellNormals all have the same color.

Quote:
Originally Posted by gschaider View Post
Define "garbage". Anyway: there may be a corner case where the sums are not correct: if a face in the set is on a processor-boundary. In that case it may be counted twice or not at all
This may be one reason but even in this case the output should not change randomly or am I wrong. The output below btw is what I mean by garbage.

Output for single processing:

Code:
#         Time          sum
            1     -0.195176
            2    -0.195298
            3    -0.195382
            4     -0.19532
            5     -0.19533
            6    -0.195343
            7    -0.195337
            8    -0.195337
            9    -0.195339
           10    -0.195338
           11    -0.195338
           12    -0.195339
           13    -0.195339
           14    -0.195338
           15    -0.195338
Output for parallel processing:
Code:
#         Time          sum
            1      -172000
            2-6.34742e-310
            3 -0.000473733
            4         -nan
            5    0.0251599
            6     -4042.94
            7  -0.00175415
            8-8.93413e-310
            9  -0.00423105
           10         -nan
           11      5.71386
           12  3.23021e-51
           13       126807
           14     -15.0175
           15-1.42553e+248
One thing I do not understand. The faceZone consists of 204 faces. When runnig topoSet on the parallell case all 204 faces are on one processor and for all other processors but one there are zero faces. On one processor it looks like this:

Quote:
1
(
-1
)
Could this be the problem or is this normal?

Last edited by billie; April 14, 2013 at 10:27.
billie is offline   Reply With Quote

Old   April 14, 2013, 10:00
Default
  #12
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
Hm, I did look up the the face normals in paraview using the NormalGlyphs filter plus the cellNormals all have the same color.
That is not enough as you see from my example. Especially as paraview may have "corrected" the orientation before displaying it

Quote:
Originally Posted by billie View Post
This may be one reason but even in this case the output should not change randomly or am I wrong. The output below btw is what I mean by garbage.

Output for single processing:

Output for parallel processing:
Remark: Use CODE instead of QUOTE . That way your example shows up in your output

That really looks like garbage

Quote:
Originally Posted by billie View Post
One thing I do not understand. The faceZone consists of 204 faces. When runnig topoSet on the parallell case all 204 faces are on one processor and for all other processors but one there are zero faces. On one processor it looks like this:

Could this be the problem or is this normal?
It depends on the decomposition, but it is very well possible that all faces in the zone are on one processor. The "-1" on one processor may be that problem (although I'm a bit surprised that you didn't get a segmentation fault) as it will try to access data that isn't there (I don't remember checking for faces smaller than 0 anywhere)
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 14, 2013, 10:03
Default
  #13
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by immortality View Post
what does flip do?
A function returning -1 or +1 that helps working around the situation I described above (the flipMap is usually set by some other utility)
immortality likes this.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 14, 2013, 14:25
Default
  #14
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
So probably this is meshing related. The mesh comes from Hypermesh and is exported directly to OpenFOAM format. It contains one internal surface which I want to use for mass flow calculation. From the export a faceZone is created for the internal surface as well as a boundary patch which I remove from the boundaries file as else the meshCheck fails. Afterwards I run renumberMesh to get rid of the unordered face warning

Code:
 ***Faces not in upper triangular order.
  <<Writing 186 unordered faces to set upperTriangularFace
Then I run topoSet to create the cell set for the slave cells of the faceZone.

Currently I do not know a better way of creating the faceZone and the required cell set, but maybe there is a more elegant way to do this.
billie is offline   Reply With Quote

Old   April 14, 2013, 17:09
Default
  #15
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
So probably this is meshing related. The mesh comes from Hypermesh and is exported directly to OpenFOAM format. It contains one internal surface which I want to use for mass flow calculation. From the export a faceZone is created for the internal surface as well as a boundary patch which I remove from the boundaries file as else the meshCheck fails. Afterwards I run renumberMesh to get rid of the unordered face warning

Code:
 ***Faces not in upper triangular order.
  <<Writing 186 unordered faces to set upperTriangularFace
Then I run topoSet to create the cell set for the slave cells of the faceZone.

Currently I do not know a better way of creating the faceZone and the required cell set, but maybe there is a more elegant way to do this.
If the serial mesh (which seems to work) is the only thing your mesher wrote, then I guess the problem is with some OpenFOAM-utility. If you didn't decompose the mesh with decomposePar but your mesher wrote the processor-directories (I understand that some meshers are capable of this) then it might be a mesher problem.

Have you tried leaving renumberMesh out (whether then things work)?
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 15, 2013, 05:40
Default
  #16
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
If the serial mesh (which seems to work) is the only thing your mesher wrote, then I guess the problem is with some OpenFOAM-utility. If you didn't decompose the mesh with decomposePar but your mesher wrote the processor-directories (I understand that some meshers are capable of this) then it might be a mesher problem.
The serial mesh is created by Hypermesh and I use decomposePar for decomposing. So there is probably an OpenFOAM-utility at fault. Until now I tried openfoam-git in combination with swak4foam-hg as well as openfoam-2.2.0 in combination with swak4foam-0.2.3.

Quote:
Originally Posted by gschaider View Post
Have you tried leaving renumberMesh out (whether then things work)?
Leaving it out does not help.

My current workflow is as follows (it is a multi-region case btw.):

Code:
runApplication transformPoints -scale '(0.001 0.001 0.001)'
runApplication renumberMesh -overwrite
runApplication checkMesh -allGeometry -allTopology
runApplication splitMeshRegions -cellZonesOnly -overwrite

for i in solid_aluminium solid_steel
do
    rm -f 0*/${i}/{mut,nuTilda,k,epsilon,omega,p_rgh,U}
done

for i in fluid_water
do
    rm -f 0*/${i}/{dummy}
done

for i in fluid_water solid_aluminium solid_steel; do
    if [ -f log.changeDictionary.${i} ]; then
        echo "changeDictionary already run on ${PWD}: remove log to re-run"
    else
        changeDictionary -region ${i} 2>&1 >log.changeDictionary.${i}
    fi
done

runApplication decomposePar -allRegions

COUNT=0
NPROCS=$(getNumberOfProcessors)

while [ "${COUNT}" -lt "${NPROCS}" ]; do
    if [ -f log.topoSetDict.processor${COUNT} ]; then
        echo "topoSetDict already run on ${PWD}: remove log to re-run"
    else
        topoSet -case processor${COUNT} -dict ${PWD}/system/topoSetDict \
            -region fluid_water 2>&1 >log.topoSetDict.processor${COUNT}
    fi

    let 'COUNT += 1'
done

runParallel $(getApplication) ${NPROCS}
runApplication reconstructPar -allRegions -latestTime
billie is offline   Reply With Quote

Old   April 15, 2013, 13:48
Default
  #17
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,915
Rep Power: 40
gschaider will become famous soon enoughgschaider will become famous soon enough
Quote:
Originally Posted by billie View Post
The serial mesh is created by Hypermesh and I use decomposePar for decomposing. So there is probably an OpenFOAM-utility at fault. Until now I tried openfoam-git in combination with swak4foam-hg as well as openfoam-2.2.0 in combination with swak4foam-0.2.3.
If the problem is the faceZone with "face" -1 than older versions of swak will show that behaviour too. The only way to know for sure is if you have a Debug-version of OF to run the case there. In the debug-version an access to index -1 will make the run fail and give a stack-trace

Quote:
Originally Posted by billie View Post
Leaving it out does not help.
Different decomposition? Just curious if this makes the -1 go away

Quote:
Originally Posted by billie View Post
My current workflow is as follows (it is a multi-region case btw.):

Code:
runApplication transformPoints -scale '(0.001 0.001 0.001)'
runApplication renumberMesh -overwrite
runApplication checkMesh -allGeometry -allTopology
runApplication splitMeshRegions -cellZonesOnly -overwrite

for i in solid_aluminium solid_steel
do
    rm -f 0*/${i}/{mut,nuTilda,k,epsilon,omega,p_rgh,U}
done

for i in fluid_water
do
    rm -f 0*/${i}/{dummy}
done

for i in fluid_water solid_aluminium solid_steel; do
    if [ -f log.changeDictionary.${i} ]; then
        echo "changeDictionary already run on ${PWD}: remove log to re-run"
    else
        changeDictionary -region ${i} 2>&1 >log.changeDictionary.${i}
    fi
done

runApplication decomposePar -allRegions

COUNT=0
NPROCS=$(getNumberOfProcessors)

while [ "${COUNT}" -lt "${NPROCS}" ]; do
    if [ -f log.topoSetDict.processor${COUNT} ]; then
        echo "topoSetDict already run on ${PWD}: remove log to re-run"
    else
        topoSet -case processor${COUNT} -dict ${PWD}/system/topoSetDict \
            -region fluid_water 2>&1 >log.topoSetDict.processor${COUNT}
    fi

    let 'COUNT += 1'
done

runParallel $(getApplication) ${NPROCS}
runApplication reconstructPar -allRegions -latestTime
I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   April 15, 2013, 14:17
Default
  #18
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
It looks like the problem was mesh related. The problems occurred with a small test model which consists of a simple channel with the interior surface in between. Today I tested it with a full model with one inlet which splits eight times and combines again afterwards. There I would like to know the mass flux distribution. For this model the flux is reported correct from the swakExpression as well as the faceSource. The sets created for the decomposed case do not include negative face numbers here.

Another thing I noticed the OpenFoam output from Hypermesh does not reliable create the faceZones, thus I exported first to Fluent format and converted to OpenFoam format using fluent3DMeshToFoam. This way I have all required boundaries, cellZones, faceZones etc. with no need to change anything manually. Plus it creates flipMaps for the faceZones. Could this probably why the reason the output from the faceZone is correct?
billie is offline   Reply With Quote

Old   April 15, 2013, 14:23
Default
  #19
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
If the problem is the faceZone with "face" -1 than older versions of swak will show that behaviour too. The only way to know for sure is if you have a Debug-version of OF to run the case there. In the debug-version an access to index -1 will make the run fail and give a stack-trace
Lets see if I can compile a debug version.

Quote:
Originally Posted by gschaider View Post
Different decomposition? Just curious if this makes the -1 go away
I already tried with simple, hierarchical and scotch as decomposition method.

Quote:
Originally Posted by gschaider View Post
I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.
I will have a look at this. I also have to re-check if using the the intermediate fluent fomat fixes this. I am not 100% sure I did check this already.

If the Hypermesh output is buggy I will contact the Altair support about this issue.
billie is offline   Reply With Quote

Old   April 15, 2013, 15:58
Default
  #20
Member
 
Daniel Pielmeier
Join Date: Apr 2012
Posts: 96
Rep Power: 5
billie is on a distinguished road
Quote:
Originally Posted by gschaider View Post
I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.
I did check this and none of the faces of the faceZone seems to be on the processor boundary. The numbers of the faceZone range with gaps from 0 to 1992 and the lowest processor boundary starts at 4630.

While checking this I recognized that sometimes the startFaces of the inter processor boundaries are not the same. I think they should be same as they share the same faces. So I have procBoundary0to1 -> startFace 15775 and procBoundary1to0 -> startFace 15571.

What is strange that I get no other errors. Anyway I think there is a problem with the Hypermesh mesh export. I will take a look at this and report back here.
billie is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Discrete Phase & Mass Flow Rate MagnusZeus FLUENT 0 December 2, 2011 18:57
Mass Flow Rate Error jallison FLUENT 1 May 16, 2011 16:22
different mass flow rates michael FLUENT 4 February 21, 2005 04:48
Target mass flow rate Saturn FLUENT 0 December 10, 2004 05:18
Mass Flow Inlet Pravir Kumar Rai FLUENT 0 February 17, 2003 20:23


All times are GMT -4. The time now is 11:17.