CFD Online Discussion Forums - Using PETSc with OpenFoam

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)

- OpenFOAM Programming & Development (https://www.cfd-online.com/Forums/openfoam-programming-development/)

- - Using PETSc with OpenFoam (https://www.cfd-online.com/Forums/openfoam-programming-development/204306-using-petsc-openfoam.html)

Using PETSc with OpenFoam

Hello everybody

I'm trying to include PETSc into my OpenFOAM code in order to be able to perform some topology optimization developments (mainly becase there is a version of the MMA optimizer already coded with PETSc).

So far, I have been unable to include petsc.h, even if I added the directories and the libraries in the option file (but is pretty sure that I make a mistake).

Does anybody has experience doing this kind of linking?

I know that there is an ancient post related to the subject, but the answer was about an ancient version of PETSc and the person who posed the question, was not able to solve the problem.

This is my options file

Code:

EXE_INC = \

    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \

    -I$(LIB_SRC)/transportModels \

    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \

    -I$(LIB_SRC)/finiteVolume/lnInclude \

    -I$(LIB_SRC)/meshTools/lnInclude \

        -I$(PETSC_DIR}/include/lnInclude \

        -I$(PETSC_DIR}/$(PETSC_ARCH)/include/lnInclude \



EXE_LIBS = \

    -lturbulenceModels \

    -lincompressibleTurbulenceModels \

    -lincompressibleTransportModels \

    -lfiniteVolume \

    -lmeshTools \

    -lfvOptions \

        -L$(PETSC_DIR)/lib/petsc/conf \

        -L$(PETSC_LIBDIR)

having that and adding to my main file #include "petsc.h" I get the error

fatal error: petsc.h: No such file or directory
compilation terminated.

The suggestion that was given a long time ago is

Code:

include $(RULES)/mplib$(WM_MPLIB)



EXE_INC = \

     $(PFLAGS) $(PINC) \

    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \

    -I$(LIB_SRC)/finiteVolume/lnInclude



EXE_LIBS = \

    -lfiniteVolume \

    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm

I tried adding the libraries -lpetsc with no luck, so I'm completely lost.
If anybody has a way to include PETSc into OpenFOAM (the simples way will do the job actually) I will be really thankful.

The library entries are completely irrelevant at this point, don't bother. You need to make sure that you get your includes right. You have obvious errors in your bracketing, and I simply doubt you're picking up the right directories for petsc. I'd recommend hard coding the include directory for petsc.h for starters.

I did the hard coding has suggested and now, even if recognize the file petsc.h, once inside that file, doesn't recognize another .h file that is called inside that one, even if I added the directories in this way.

Code:

EXE_INC = \

    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \

    -I$(LIB_SRC)/transportModels \

    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \

    -I$(LIB_SRC)/finiteVolume/lnInclude \

    -I$(LIB_SRC)/meshTools/lnInclude \

        -I/home/pablo/PETSC/3.7.4/include/lnInclude \

        -I/home/pablo/PETSC/3.7.4/include/petsc/finclude/lnInclude \

        -I/home/pablo/PETSC/3.7.4/linux-dbg/include/lnInclude \



EXE_LIBS = \

    -lturbulenceModels \

    -lincompressibleTurbulenceModels \

    -lincompressibleTransportModels \

    -lfiniteVolume \

    -lmeshTools \

    -lfvOptions \

        -L/home/pablo/PETSC/3.7.4/lib/petsc/conf \

        -L/home/pablo/PETSC/3.7.4/linux-dbg/lib

Quote:

Originally Posted by palarcon (Post 699831)

Code:

EXE_INC = \

    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \

    -I$(LIB_SRC)/transportModels \

    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \

    -I$(LIB_SRC)/finiteVolume/lnInclude \

    -I$(LIB_SRC)/meshTools/lnInclude \

    -I/home/pablo/PETSC/3.7.4/include/lnInclude \

    -I/home/pablo/PETSC/3.7.4/include/petsc/finclude/lnInclude \

    -I/home/pablo/PETSC/3.7.4/linux-dbg/include/lnInclude \



EXE_LIBS = \

    -lturbulenceModels \

    -lincompressibleTurbulenceModels \

    -lincompressibleTransportModels \

    -lfiniteVolume \

    -lmeshTools \

    -lfvOptions \

    -L/home/pablo/PETSC/3.7.4/lib/petsc/conf \

    -L/home/pablo/PETSC/3.7.4/linux-dbg/lib

I guess that your petsc installation was using the standard PETSc configure/install. In which case you won't have the wmake-style 'lnInclude' at all, but just the plain include directory. Eg,

Code:

 



    -I/home/pablo/PETSC/3.7.4/include \

If you have any code to share (quality doesn't matter much), I'd appreciate taking a look at it as well.

/mark

Hello and thank you all for the detailed answers.
I'm using the standard adjointshapeoptimizationfoam, where in addition to separate in different files the computation of the primal and dual velocities/pressures without any problem (to make it easier to read), I added an #include to call petsc.h (I will put my modifications in colors).
Also, the PETSc installation was done using the standard PETSc configure/install parameters (in a PETSc test case I compiled using the standard make).

At first my main code looked like this

Code:

#include "fvCFD.H"

#include "singlePhaseTransportModel.H"

#include "turbulentTransportModel.H"

#include "simpleControl.H"

#include "fvOptions.H"

#include "petsc.h"



template<class Type>

void zeroCells

(

    GeometricField<Type, fvPatchField, volMesh>& vf,

    const labelList& cells

)

{

    forAll(cells, i)

    {

        vf[cells[i]] = Zero;

    }

}





// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //



int main(int argc, char *argv[])

{

    #include "postProcess.H"



    #include "setRootCase.H"

    #include "createTime.H"

    #include "createMesh.H"

    #include "createControl.H"

    #include "createFields.H"

    #include "createFvOptions.H"

    #include "initContinuityErrs.H"

    #include "initAdjointContinuityErrs.H"



    turbulence->validate();



    // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //



    Info<< "\nStarting time loop\n" << endl;



    while (simple.loop())

    {

        Info<< "Time = " << runTime.timeName() << nl << endl;



                #include "updatePorosityField.H"

                

        // Pressure-velocity SIMPLE corrector

        {

                        #include "UEqn.H"

                        #include "pEqn.H"                        

        }



        // Adjoint Pressure-velocity SIMPLE corrector

        {

                        #include "UaEqn.H"

                        #include "paEqn.H"

        }



        laminarTransport.correct();

        turbulence->correct();

                

                sens=Ua&U;

                

                #include "costFunction.H"



        runTime.write();



        Info<< "ExecutionTime = "

            << runTime.elapsedCpuTime()

            << " s\n\n" << endl;

    }



    Info<< "End\n" << endl;



    return 0;

}

and the options file like this

Code:

EXE_INC = \

    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \

    -I$(LIB_SRC)/transportModels \

    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \

    -I$(LIB_SRC)/finiteVolume/lnInclude \

    -I$(LIB_SRC)/meshTools/lnInclude \

        -I$(PETSC_DIR}/include/lnInclude \

        -I$(PETSC_DIR}/$(PETSC_ARCH)/include/lnInclude \



EXE_LIBS = \

    -lturbulenceModels \

    -lincompressibleTurbulenceModels \

    -lincompressibleTransportModels \

    -lfiniteVolume \

    -lmeshTools \

    -lfvOptions \

        -L$(PETSC_DIR)/lib/petsc/conf \

        -L$(PETSC_LIBDIR)

Doing that I got the error fatal error: petsc.h: No such file or directory compilation terminated.

That was because I was sloppy with the parenthesis on the options file, but following a subsequent advice I did the hard coding for the petsc.h call and to include the folders in the options file.

After that, my petsc.h call was done like this on the main code

Code:

#include "/home/pablo/PETSC/3.7.4/include/petsc.h"

and the options file looks like this now (I was able to correctly write that using environmental variables)

Code:

EXE_INC = \

    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \

    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \

    -I$(LIB_SRC)/transportModels \

    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \

    -I$(LIB_SRC)/finiteVolume/lnInclude \

    -I$(LIB_SRC)/meshTools/lnInclude \

    -I$(PETSC_DIR)/include \

    -I$(PETSC_DIR)/$(PETSC_ARCH)/include \

    -I$(PETSC_DIR)/include/petsc/finclude \





EXE_LIBS = \

    -lturbulenceModels \

    -lincompressibleTurbulenceModels \

    -lincompressibleTransportModels \

    -lfiniteVolume \

    -lmeshTools \

    -lfvOptions \

    -L/home/pablo/PETSC/3.7.4/lib/petsc/conf \

    -L/home/pablo/PETSC/3.7.4/linux-dbg/lib

Now, the main code is able to find the petsc.h file and all the subsequent .h files, but stop again now with an error related to the language itself related to the "pow" operation (I guess that linking OpenFOAM with PETSc language is not as straightforward).

/home/pablo/PETSC/3.7.4/include/petscmath.h:326:38: error: call of overloaded ‘pow(PetscScalar&, PetscScalar&)’ is ambiguous
#define PetscPowScalar(a,b) pow(a,b)

Any idea on how solve this last problem related to the "pow" operator?

Problem solved

Given the fact that OpenFOAM and PETSc have a definition for the pow operator, this one had an ambiguous definition.
To solve it, I had to add "::" in the pow definition inside the PETSc file, in order to avoir the confusion on the definition.

Hi Pablo:

I am interesting in how you using the MMA in PETSc to manipulate the OpenFOAM fields such as alpha. Can you show some hints in your codes?
Thanks much.

Hello and sorry for the late answer.
Finally, and for a number of reasons (mainly the fact that I couldn't find a reliably C++ version of MMA, even if there are) I declined using, at least for the moment, MMA with PETSc.
At the time I performed a lot of test case and I wasn't happy with the results.
Sorry!