[GerhardHolzinger]

The utility checkMesh determines the presence of concave faces by calling the function checkFaceAngles(). The code of this function is attached (OF-2.1.x). Here is the relevant code section of checkGeometry.C

if (allGeometry)
    {
        faceSet faces(mesh, "concaveFaces", mesh.nFaces()/100 + 1);
        if (mesh.checkFaceAngles(true, 10, &faces))
        {
            //noFailedChecks++;

            label nFaces = returnReduce(faces.size(), sumOp<label>());

            if (nFaces > 0)
            {
                Info<< "  <<Writing " << nFaces
                    << " faces with concave angles to set " << faces.name()
                    << endl;
                faces.instance() = mesh.pointsInstance();
                faces.write();
            }
        }
    }

If you search the code of checkFaceAngles() for the part where the concave faces are counted, you will see that this part is behind some conditions.

// Check normal
                    edgeNormal /= magEdgeNormal;

                    if ((edgeNormal & faceNormal) < SMALL)
                    {
                        if (faceI != errorFaceI)
                        {
                            // Count only one error per face.
                            errorFaceI = faceI;
                            nConcave++;
                        }

                        if (setPtr)
                        {
                            setPtr->insert(faceI);
                        }

                        maxEdgeSin = max(maxEdgeSin, magEdgeNormal);
                    }

What bothers me with this piece of code is the following condition

if ((edgeNormal & faceNormal) < SMALL)

I can't really figure out how this condition works, or how it is intended. Why am I thinking this:

faceNormal is the normalized face normal vector. So its magnitude is one and the vector is perpendicular to the face.

edgeNormal is also a normalized vector. edgeNormal was generated by calculating the cross-product of two consecutive edges of the face. As an edge connects two points, an edge can always be represented by a vector pointing from one point to the other point.

However, as all edges of the face lie within a plane to which the faceNormal is prependicular. The cross-product of two edges yields always a vector parallel to the faceNormal. In fact it can be parallel or anti-parallel.

The condition

((edgeNormal & faceNormal) < SMALL)

computed the inner-product of the edgeNormal and the faceNormal. As these two vectors are either parallel or anti-parallel, the inner-product is either 1 or -1.
So, the condition is false for the parallel case and false for the anti-parallel case.

How can this condition detect a concave cell? What I figured out is that the following is a concave cell, meaning that the edges delimiting the face make a bend inwards.

I can't see that the condition behind which the line nConcave++; is located is able to detect this.

o------o
| xxxx/
| xxx/
| xxo
| xxx\
o-----o


What am I missing, or am I somewhere wrong?