CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Post-Processing (http://www.cfd-online.com/Forums/openfoam-post-processing/)
-   -   Mass flow through faceZone using swak4foam when running in parallel (http://www.cfd-online.com/Forums/openfoam-post-processing/116125-mass-flow-through-facezone-using-swak4foam-when-running-parallel.html)

billie April 13, 2013 09:33

Mass flow through faceZone using swak4foam when running in parallel
 
I try to obtain the mass flow through a faceZone with swak4foam using the following function in my controlDict:

Code:

    flowIntFluid
    {
        type swakExpression;
        valueType faceZone;
        region fluid_water;
        zoneName int_fluid;
        expression "phi*flip()";
        accumulations (
            sum
        );
        verbose true;
    }

I have created a cellSet for the slave cells of the faceZone with the following action in topoSetDict:

Code:

    {
        name    int_fluidSlaveCells;
        type    cellSet;
        action  new;
        source  faceZoneToCell;
        sourceInfo
        {
            name int_fluid;
            option slave;
        }
    }

Everything works okay when I run the simulation on a single processor however when running in parallel the expression returns garbage, which is probably because of the set information missing in the individual processor subdirectories.

Is it possible to distribute this set information to the individual directories?

billie April 13, 2013 11:52

I figured out that I do not need swak4foam for this task, using the following function in controlDict does the same and works when running in parallel.

Code:

    flowIntFluid
    {
        type            faceSource;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled        true;
        outputControl  timeStep;
        region          fluid_water;
        log            true;
        valueOutput    false;
        source          faceZone;
        sourceName      int_fluid;
        operation      sum;

        fields
        (
            phi
        );
    }


gschaider April 13, 2013 16:12

Quote:

Originally Posted by billie (Post 420291)
Everything works okay when I run the simulation on a single processor however when running in parallel the expression returns garbage, which is probably because of the set information missing in the individual processor subdirectories.

Is it possible to distribute this set information to the individual directories?

No. Sets are not decomposed. But what works is running the utilities that were used to create the sets on the parallel case

gschaider April 13, 2013 16:13

Quote:

Originally Posted by billie (Post 420301)
I figured out that I do not need swak4foam for this task, using the following function in controlDict does the same and works when running in parallel.

Code:

    flowIntFluid
    {
        type            faceSource;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled        true;
        outputControl  timeStep;
        region          fluid_water;
        log            true;
        valueOutput    false;
        source          faceZone;
        sourceName      int_fluid;
        operation      sum;

        fields
        (
            phi
        );
    }


Of course this result is only correct if all faces in the zone are oriented in the same way

billie April 13, 2013 16:36

Quote:

Originally Posted by gschaider (Post 420330)
No. Sets are not decomposed. But what works is running the utilities that were used to create the sets on the parallel case

Running topoSet on the decomosed case creates the sets but I get garbage output as well.

billie April 13, 2013 16:45

Quote:

Originally Posted by gschaider (Post 420331)
Of course this result is only correct if all faces in the zone are oriented in the same way

As I said running on a single core the swak4foam expression works but not when running on multiple cores, even when the set is created on the parallel case.

Running funkyDoCalc on the reconstructed case with the following dictionary works as well:

Code:

massFlow
{
    valueType faceZone;
    zoneName int_fluid;
    accumulations (
        sum
    );
    expression "phi";
}

So the problem is not related to the face orientation.

billie April 13, 2013 17:32

Quote:

Originally Posted by gschaider (Post 420331)
Of course this result is only correct if all faces in the zone are oriented in the same way

I checked the face normals in paraview and they are all oriented in the same way. So the slave cell set is not required in this case. I still do however get the wrong output when running in parallel.

gschaider April 14, 2013 05:53

Quote:

Originally Posted by billie (Post 420333)
Running topoSet on the decomosed case creates the sets but I get garbage output as well.

Define "garbage". Anyway: there may be a corner case where the sums are not correct: if a face in the set is on a processor-boundary. In that case it may be counted twice or not at all

gschaider April 14, 2013 06:01

Quote:

Originally Posted by billie (Post 420335)
As I said running on a single core the swak4foam expression works but not when running on multiple cores, even when the set is created on the parallel case.

Running funkyDoCalc on the reconstructed case with the following dictionary works as well:

Code:

massFlow
{
    valueType faceZone;
    zoneName int_fluid;
    accumulations (
        sum
    );
    expression "phi";
}

So the problem is not related to the face orientation.

Then it worked OK in that case. But in general phi is determined by the order of the cells. Suppose you have a "channel" with four cells. Digits are are the cell numbers and letters are the faces in the face Set. Flow goes from left to right (the way it always does):

0 A 1
2 B 3

Here on A and B phi is positive. In another case

1 A 0
2 B 3

on A the phi is negative while on B it is still positive. Summing it up without multiplying it with the flip will give you a wrong result. That is what I mean with "orientation" and am not sure how you'd visualize this with paraview (as you mentioned in another posting)

immortality April 14, 2013 06:45

what does flip do?

billie April 14, 2013 08:03

Quote:

Originally Posted by gschaider (Post 420381)
That is what I mean with "orientation" and am not sure how you'd visualize this with paraview (as you mentioned in another posting)

Hm, I did look up the the face normals in paraview using the NormalGlyphs filter plus the cellNormals all have the same color.

Quote:

Originally Posted by gschaider (Post 420381)
Define "garbage". Anyway: there may be a corner case where the sums are not correct: if a face in the set is on a processor-boundary. In that case it may be counted twice or not at all

This may be one reason but even in this case the output should not change randomly or am I wrong. The output below btw is what I mean by garbage.

Output for single processing:

Code:

#        Time          sum
            1    -0.195176
            2    -0.195298
            3    -0.195382
            4    -0.19532
            5    -0.19533
            6    -0.195343
            7    -0.195337
            8    -0.195337
            9    -0.195339
          10    -0.195338
          11    -0.195338
          12    -0.195339
          13    -0.195339
          14    -0.195338
          15    -0.195338

Output for parallel processing:
Code:

#        Time          sum
            1      -172000
            2-6.34742e-310
            3 -0.000473733
            4        -nan
            5    0.0251599
            6    -4042.94
            7  -0.00175415
            8-8.93413e-310
            9  -0.00423105
          10        -nan
          11      5.71386
          12  3.23021e-51
          13      126807
          14    -15.0175
          15-1.42553e+248

One thing I do not understand. The faceZone consists of 204 faces. When runnig topoSet on the parallell case all 204 faces are on one processor and for all other processors but one there are zero faces. On one processor it looks like this:

Quote:

1
(
-1
)
Could this be the problem or is this normal?

gschaider April 14, 2013 10:00

Quote:

Originally Posted by billie (Post 420413)
Hm, I did look up the the face normals in paraview using the NormalGlyphs filter plus the cellNormals all have the same color.

That is not enough as you see from my example. Especially as paraview may have "corrected" the orientation before displaying it

Quote:

Originally Posted by billie (Post 420413)
This may be one reason but even in this case the output should not change randomly or am I wrong. The output below btw is what I mean by garbage.

Output for single processing:

Output for parallel processing:

Remark: Use CODE instead of QUOTE . That way your example shows up in your output

That really looks like garbage

Quote:

Originally Posted by billie (Post 420413)
One thing I do not understand. The faceZone consists of 204 faces. When runnig topoSet on the parallell case all 204 faces are on one processor and for all other processors but one there are zero faces. On one processor it looks like this:

Could this be the problem or is this normal?

It depends on the decomposition, but it is very well possible that all faces in the zone are on one processor. The "-1" on one processor may be that problem (although I'm a bit surprised that you didn't get a segmentation fault) as it will try to access data that isn't there (I don't remember checking for faces smaller than 0 anywhere)

gschaider April 14, 2013 10:03

Quote:

Originally Posted by immortality (Post 420392)
what does flip do?

A function returning -1 or +1 that helps working around the situation I described above (the flipMap is usually set by some other utility)

billie April 14, 2013 14:25

So probably this is meshing related. The mesh comes from Hypermesh and is exported directly to OpenFOAM format. It contains one internal surface which I want to use for mass flow calculation. From the export a faceZone is created for the internal surface as well as a boundary patch which I remove from the boundaries file as else the meshCheck fails. Afterwards I run renumberMesh to get rid of the unordered face warning

Code:

***Faces not in upper triangular order.
  <<Writing 186 unordered faces to set upperTriangularFace

Then I run topoSet to create the cell set for the slave cells of the faceZone.

Currently I do not know a better way of creating the faceZone and the required cell set, but maybe there is a more elegant way to do this.

gschaider April 14, 2013 17:09

Quote:

Originally Posted by billie (Post 420486)
So probably this is meshing related. The mesh comes from Hypermesh and is exported directly to OpenFOAM format. It contains one internal surface which I want to use for mass flow calculation. From the export a faceZone is created for the internal surface as well as a boundary patch which I remove from the boundaries file as else the meshCheck fails. Afterwards I run renumberMesh to get rid of the unordered face warning

Code:

***Faces not in upper triangular order.
  <<Writing 186 unordered faces to set upperTriangularFace

Then I run topoSet to create the cell set for the slave cells of the faceZone.

Currently I do not know a better way of creating the faceZone and the required cell set, but maybe there is a more elegant way to do this.

If the serial mesh (which seems to work) is the only thing your mesher wrote, then I guess the problem is with some OpenFOAM-utility. If you didn't decompose the mesh with decomposePar but your mesher wrote the processor-directories (I understand that some meshers are capable of this) then it might be a mesher problem.

Have you tried leaving renumberMesh out (whether then things work)?

billie April 15, 2013 05:40

Quote:

Originally Posted by gschaider (Post 420509)
If the serial mesh (which seems to work) is the only thing your mesher wrote, then I guess the problem is with some OpenFOAM-utility. If you didn't decompose the mesh with decomposePar but your mesher wrote the processor-directories (I understand that some meshers are capable of this) then it might be a mesher problem.

The serial mesh is created by Hypermesh and I use decomposePar for decomposing. So there is probably an OpenFOAM-utility at fault. Until now I tried openfoam-git in combination with swak4foam-hg as well as openfoam-2.2.0 in combination with swak4foam-0.2.3.

Quote:

Originally Posted by gschaider (Post 420509)
Have you tried leaving renumberMesh out (whether then things work)?

Leaving it out does not help.

My current workflow is as follows (it is a multi-region case btw.):

Code:

runApplication transformPoints -scale '(0.001 0.001 0.001)'
runApplication renumberMesh -overwrite
runApplication checkMesh -allGeometry -allTopology
runApplication splitMeshRegions -cellZonesOnly -overwrite

for i in solid_aluminium solid_steel
do
    rm -f 0*/${i}/{mut,nuTilda,k,epsilon,omega,p_rgh,U}
done

for i in fluid_water
do
    rm -f 0*/${i}/{dummy}
done

for i in fluid_water solid_aluminium solid_steel; do
    if [ -f log.changeDictionary.${i} ]; then
        echo "changeDictionary already run on ${PWD}: remove log to re-run"
    else
        changeDictionary -region ${i} 2>&1 >log.changeDictionary.${i}
    fi
done

runApplication decomposePar -allRegions

COUNT=0
NPROCS=$(getNumberOfProcessors)

while [ "${COUNT}" -lt "${NPROCS}" ]; do
    if [ -f log.topoSetDict.processor${COUNT} ]; then
        echo "topoSetDict already run on ${PWD}: remove log to re-run"
    else
        topoSet -case processor${COUNT} -dict ${PWD}/system/topoSetDict \
            -region fluid_water 2>&1 >log.topoSetDict.processor${COUNT}
    fi

    let 'COUNT += 1'
done

runParallel $(getApplication) ${NPROCS}
runApplication reconstructPar -allRegions -latestTime


gschaider April 15, 2013 13:48

Quote:

Originally Posted by billie (Post 420610)
The serial mesh is created by Hypermesh and I use decomposePar for decomposing. So there is probably an OpenFOAM-utility at fault. Until now I tried openfoam-git in combination with swak4foam-hg as well as openfoam-2.2.0 in combination with swak4foam-0.2.3.

If the problem is the faceZone with "face" -1 than older versions of swak will show that behaviour too. The only way to know for sure is if you have a Debug-version of OF to run the case there. In the debug-version an access to index -1 will make the run fail and give a stack-trace

Quote:

Originally Posted by billie (Post 420610)
Leaving it out does not help.

Different decomposition? Just curious if this makes the -1 go away

Quote:

Originally Posted by billie (Post 420610)
My current workflow is as follows (it is a multi-region case btw.):

Code:

runApplication transformPoints -scale '(0.001 0.001 0.001)'
runApplication renumberMesh -overwrite
runApplication checkMesh -allGeometry -allTopology
runApplication splitMeshRegions -cellZonesOnly -overwrite

for i in solid_aluminium solid_steel
do
    rm -f 0*/${i}/{mut,nuTilda,k,epsilon,omega,p_rgh,U}
done

for i in fluid_water
do
    rm -f 0*/${i}/{dummy}
done

for i in fluid_water solid_aluminium solid_steel; do
    if [ -f log.changeDictionary.${i} ]; then
        echo "changeDictionary already run on ${PWD}: remove log to re-run"
    else
        changeDictionary -region ${i} 2>&1 >log.changeDictionary.${i}
    fi
done

runApplication decomposePar -allRegions

COUNT=0
NPROCS=$(getNumberOfProcessors)

while [ "${COUNT}" -lt "${NPROCS}" ]; do
    if [ -f log.topoSetDict.processor${COUNT} ]; then
        echo "topoSetDict already run on ${PWD}: remove log to re-run"
    else
        topoSet -case processor${COUNT} -dict ${PWD}/system/topoSetDict \
            -region fluid_water 2>&1 >log.topoSetDict.processor${COUNT}
    fi

    let 'COUNT += 1'
done

runParallel $(getApplication) ${NPROCS}
runApplication reconstructPar -allRegions -latestTime


I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.

billie April 15, 2013 14:17

It looks like the problem was mesh related. The problems occurred with a small test model which consists of a simple channel with the interior surface in between. Today I tested it with a full model with one inlet which splits eight times and combines again afterwards. There I would like to know the mass flux distribution. For this model the flux is reported correct from the swakExpression as well as the faceSource. The sets created for the decomposed case do not include negative face numbers here.

Another thing I noticed the OpenFoam output from Hypermesh does not reliable create the faceZones, thus I exported first to Fluent format and converted to OpenFoam format using fluent3DMeshToFoam. This way I have all required boundaries, cellZones, faceZones etc. with no need to change anything manually. Plus it creates flipMaps for the faceZones. Could this probably why the reason the output from the faceZone is correct?

billie April 15, 2013 14:23

Quote:

Originally Posted by gschaider (Post 420746)
If the problem is the faceZone with "face" -1 than older versions of swak will show that behaviour too. The only way to know for sure is if you have a Debug-version of OF to run the case there. In the debug-version an access to index -1 will make the run fail and give a stack-trace

Lets see if I can compile a debug version.

Quote:

Originally Posted by gschaider (Post 420746)
Different decomposition? Just curious if this makes the -1 go away

I already tried with simple, hierarchical and scotch as decomposition method.

Quote:

Originally Posted by gschaider (Post 420746)
I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.

I will have a look at this. I also have to re-check if using the the intermediate fluent fomat fixes this. I am not 100% sure I did check this already.

If the Hypermesh output is buggy I will contact the Altair support about this issue.

billie April 15, 2013 15:58

Quote:

Originally Posted by gschaider (Post 420746)
I'm not sure, but one possibility is that the -1 face means that this face is on a processor boundary. One way to check that would be to look at the boundary and the faceZones-file of the processor that has the 208 processes and from the start-entries in boundary see if any faces in the faceZone are on a processor boundary.

I did check this and none of the faces of the faceZone seems to be on the processor boundary. The numbers of the faceZone range with gaps from 0 to 1992 and the lowest processor boundary starts at 4630.

While checking this I recognized that sometimes the startFaces of the inter processor boundaries are not the same. I think they should be same as they share the same faces. So I have procBoundary0to1 -> startFace 15775 and procBoundary1to0 -> startFace 15571.

What is strange that I get no other errors. Anyway I think there is a problem with the Hypermesh mesh export. I will take a look at this and report back here.


All times are GMT -4. The time now is 06:26.