[mitti]

Hi folks,

my current case (Channel Flow, OF 2.1.x) works fine except one point: I calculate the mass flux during run time via phi. Works fine in single core but in parallel the result is wrong. The code looks like:

scalar inletFlux = 0;
label inletPatch = mesh.boundaryMesh().findPatchID("inlet");
inletFlux  = -1.0*gSum(phi.boundaryField()[inletPatch]);

Even if I use "reduce(inletFlux, sumOp<scalar>());" and "sum" instead "gSum" the result is wrong. If I take a look into the output of every processor in parallel, the results do not sum up to the correct value.

Any guess, idea etc. what could be the problem? Do you need more information?

[mAlletto]

can you provide a bit more information:


What mass flow is calculated in each processor and with is calculated by summing it over the processors.




findPatchId gives back -1 if it does not find any patch with the name specified.



I'm not sure what happens in a decomposed case when the patch looked for is not on a processor and you do the gsum.






did you try to sum the mass flux phi with the function sum only in case findPatchId gives back a number different than -1 and after that the results over all processors

[mitti]

Hi Michael,

thank you for your reply.

Originally Posted by mAlletto

What mass flow is calculated in each processor and with is calculated by summing it over the processors.

Single core:

The fun part is, that the mass flux is different for the number of processors:
4 processors: . With values in three processors, one is zero.
8 processors: . (This is something I really don’t understand)
16 processors: but only one processor has a value, all others are zero.
32 processors: with different values in six processors.

This weird results could be because I use scotch? I will try different decompose methods.

Originally Posted by mAlletto

findPatchId gives back -1 if it does not find any patch with the name specified.

I'm not sure what happens in a decomposed case when the patch looked for is not on a processor and you do the gsum. did you try to sum the mass flux phi with the function sum only in case findPatchId gives back a number different than -1 and after that the results over all processors

I don’t know too, so I modified the code:

scalar inletFlux = 0;
label inletPatch = mesh.boundaryMesh().findPatchID("inlet");

if (inletPatch == -1) {
    Info << "Failure to find patch for mass flow calc" << endl;
}
else {
    inletFlux = gSum(phi.boundaryField()[inletPatch]);
    inletFlux *= -1.0;
}

I also tried this but there is no difference in the result:

scalar inletFlux = 0;
label inletPatch = mesh.boundaryMesh().findPatchID("inlet");

if (inletPatch == -1) {
    Info << "Failure to find patch for mass flow calc" << endl;
}
else {
    inletFlux = sum(phi.boundaryField()[inletPatch]);
}

reduce(inletFlux, sumOp<scalar>());
inletFlux *= -1.0;

[mAlletto]

That's strange. What are the initial and boundary conditions? Do you have a constant pressure gradient which drives the flow? Did you try a newer version of OF?

[mAlletto]

By the way, there is a function object already available in OF which determines the mass flow at a specific patch. See how to calculate mass flow rate /simpleFoam.

Maybe you can compere your output with the output of the function object and this can give you a hint what's going wrong.

[mitti]

Originally Posted by mAlletto

By the way, there is a function object already available in OF which determines the mass flow at a specific patch. See how to calculate mass flow rate /simpleFoam.

Maybe you can compere your output with the output of the function object and this can give you a hint what's going wrong.

I took me some time to modify it for 2.1.x but now it runs. Next time I should read the whole wiki thread first^^
Anyway, calcMassFlow is only single core and it gives me the same result as I get for single core.

[mitti]

Originally Posted by mAlletto

That's strange. What are the initial and boundary conditions? Do you have a constant pressure gradient which drives the flow?

You can find my initial/boundary conditions attached.

I do have a pressure gradient which drives the flow. It is added into the momentum equation.

Originally Posted by mAlletto

Did you try a newer version of OF?

Not yet, will do it today and keep you updated.

[mitti]

Originally Posted by mitti

Not yet, will do it today and keep you updated.

Finally, I found enough time to give it a shoot with of1606 and 1712. It is the same result for both versions, the only major difference is that it works for 2 and 4 processors but as soon as I use 8 or more, it fails again. This time with different results for each processor than in of2.1.x.

However, while reading a bunch of stuff here, I came across this thread and the statement "The main problem could be that faces that are on a processor boundary could be counted twice (or not at all)".

Also, in the wiki for calcMassFlow it's stated that the calculation of the mass flux in parallel is an issues. Nevertheless, all threads are relatively old.

[mAlletto]

Originally Posted by mitti

You can find my initial/boundary conditions attached.

I do have a pressure gradient which drives the flow. It is added into the momentum equation.





Not yet, will do it today and keep you updated.

I just had a look on the initialization and it seems you used fixed value boundary conditions for nut, epsilon and k. I would rather advise you to use fixed value for k and epsilon and calculated for nut, or use wall function for all three quantities. Otherwise you get some inconsistency for the boundary conditions.

A and you do you specify the pressure gradient which drives the flow? Is it specified in the fvOption? If so can you post it too so I can have a look?

[mitti]

Originally Posted by mAlletto

I just had a look on the initialization and it seems you used fixed value boundary conditions for nut, epsilon and k. I would rather advise you to use fixed value for k and epsilon and calculated for nut, or use wall function for all three quantities. Otherwise you get some inconsistency for the boundary conditions.

As far as I know it makes, regarding nut, no difference for Channel Flow. But I will run a simulation and check it. Thanks.

Since I use a low-Re model, wall functions are not necessary.

Originally Posted by mAlletto

A and you do you specify the pressure gradient which drives the flow? Is it specified in the fvOption? If so can you post it too so I can have a look?

I modified the UEqn.h

fvVectorMatrix UEqn
(
   fvm::ddt(U)
 + fvm::div(phi, U)
 + turbulence->divDevReff(U)
 ==
   vector(1,0,0) * gradP
);

UEqn.relax();

if (momentumPredictor)
{
   solve(UEqn == -fvc::grad(p));
}

with a pre-calculation of gradP at the beginning of the simulation. ReTau is the desired ReTau, height is the channel half height.

dimensionedScalar uTau = ReTau*nu/height;
dimensionedScalar gradP = pow(uTau,2)/height;

[mitti]

Originally Posted by mitti

As far as I know it makes, regarding nut, no difference for Channel Flow. But I will run a simulation and check it. Thanks.

Done and the results are the same for U, k and nut. And, so far LaunderSharma is capable, the results are close to DNS data.

Nevertheless, I found the solution for my problem with further and deeper digging in the forum finally in this thread.
The problem is, as I already figured out, the way how openFoam parallelizes: the simple fact that a patch can also be the processor boundary, can cause problems when summing up values on this patch. See both links for further information.
However, with "forcing" the decomposePar-method that all boundaries are on one processor (with preservePatches in the decomposeParDict) this problem is avoided. The impact of a maybe bad decomposition, which could effect the runtime, for my case is another aspect.