CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Programming & Development

Using PETSc with OpenFoam

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By palarcon

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   July 19, 2018, 10:11
Default Using PETSc with OpenFoam
  #1
Member
 
Pablo Alarcón
Join Date: Mar 2018
Location: Liège
Posts: 59
Rep Power: 8
palarcon is on a distinguished road
Hello everybody

I'm trying to include PETSc into my OpenFOAM code in order to be able to perform some topology optimization developments (mainly becase there is a version of the MMA optimizer already coded with PETSc).

So far, I have been unable to include petsc.h, even if I added the directories and the libraries in the option file (but is pretty sure that I make a mistake).

Does anybody has experience doing this kind of linking?

I know that there is an ancient post related to the subject, but the answer was about an ancient version of PETSc and the person who posed the question, was not able to solve the problem.

This is my options file

Code:
EXE_INC = \
    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \
    -I$(LIB_SRC)/transportModels \
    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
    -I$(LIB_SRC)/finiteVolume/lnInclude \
    -I$(LIB_SRC)/meshTools/lnInclude \
	-I$(PETSC_DIR}/include/lnInclude \
	-I$(PETSC_DIR}/$(PETSC_ARCH)/include/lnInclude \

EXE_LIBS = \
    -lturbulenceModels \
    -lincompressibleTurbulenceModels \
    -lincompressibleTransportModels \
    -lfiniteVolume \
    -lmeshTools \
    -lfvOptions \
	-L$(PETSC_DIR)/lib/petsc/conf \
	-L$(PETSC_LIBDIR)
having that and adding to my main file #include "petsc.h" I get the error

fatal error: petsc.h: No such file or directory
compilation terminated.


The suggestion that was given a long time ago is

Code:
include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
     $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm
I tried adding the libraries -lpetsc with no luck, so I'm completely lost.
If anybody has a way to include PETSc into OpenFOAM (the simples way will do the job actually) I will be really thankful.
palarcon is offline   Reply With Quote

Old   July 19, 2018, 10:48
Default
  #2
Senior Member
 
akidess's Avatar
 
Anton Kidess
Join Date: May 2009
Location: Germany
Posts: 1,377
Rep Power: 29
akidess will become famous soon enough
The library entries are completely irrelevant at this point, don't bother. You need to make sure that you get your includes right. You have obvious errors in your bracketing, and I simply doubt you're picking up the right directories for petsc. I'd recommend hard coding the include directory for petsc.h for starters.
__________________
*On twitter @akidTwit
*Spend as much time formulating your questions as you expect people to spend on their answer.
akidess is offline   Reply With Quote

Old   July 19, 2018, 11:18
Default
  #3
Member
 
Pablo Alarcón
Join Date: Mar 2018
Location: Liège
Posts: 59
Rep Power: 8
palarcon is on a distinguished road
I did the hard coding has suggested and now, even if recognize the file petsc.h, once inside that file, doesn't recognize another .h file that is called inside that one, even if I added the directories in this way.

Code:
EXE_INC = \
    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \
    -I$(LIB_SRC)/transportModels \
    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
    -I$(LIB_SRC)/finiteVolume/lnInclude \
    -I$(LIB_SRC)/meshTools/lnInclude \
	-I/home/pablo/PETSC/3.7.4/include/lnInclude \
	-I/home/pablo/PETSC/3.7.4/include/petsc/finclude/lnInclude \
	-I/home/pablo/PETSC/3.7.4/linux-dbg/include/lnInclude \

EXE_LIBS = \
    -lturbulenceModels \
    -lincompressibleTurbulenceModels \
    -lincompressibleTransportModels \
    -lfiniteVolume \
    -lmeshTools \
    -lfvOptions \
	-L/home/pablo/PETSC/3.7.4/lib/petsc/conf \
	-L/home/pablo/PETSC/3.7.4/linux-dbg/lib
palarcon is offline   Reply With Quote

Old   July 19, 2018, 19:28
Default
  #4
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: https://olesenm.github.io/
Posts: 1,679
Rep Power: 40
olesen has a spectacular aura aboutolesen has a spectacular aura about
Quote:
Originally Posted by palarcon View Post
I did the hard coding has suggested and now, even if recognize the file petsc.h, once inside that file, doesn't recognize another .h file that is called inside that one, even if I added the directories in this way.

Code:
EXE_INC = \
    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \
    -I$(LIB_SRC)/transportModels \
    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
    -I$(LIB_SRC)/finiteVolume/lnInclude \
    -I$(LIB_SRC)/meshTools/lnInclude \
    -I/home/pablo/PETSC/3.7.4/include/lnInclude \
    -I/home/pablo/PETSC/3.7.4/include/petsc/finclude/lnInclude \
    -I/home/pablo/PETSC/3.7.4/linux-dbg/include/lnInclude \

EXE_LIBS = \
    -lturbulenceModels \
    -lincompressibleTurbulenceModels \
    -lincompressibleTransportModels \
    -lfiniteVolume \
    -lmeshTools \
    -lfvOptions \
    -L/home/pablo/PETSC/3.7.4/lib/petsc/conf \
    -L/home/pablo/PETSC/3.7.4/linux-dbg/lib

I guess that your petsc installation was using the standard PETSc configure/install. In which case you won't have the wmake-style 'lnInclude' at all, but just the plain include directory. Eg,


Code:
 

    -I/home/pablo/PETSC/3.7.4/include \

If you have any code to share (quality doesn't matter much), I'd appreciate taking a look at it as well.



/mark
olesen is offline   Reply With Quote

Old   July 20, 2018, 04:35
Default
  #5
Member
 
Pablo Alarcón
Join Date: Mar 2018
Location: Liège
Posts: 59
Rep Power: 8
palarcon is on a distinguished road
Hello and thank you all for the detailed answers.
I'm using the standard adjointshapeoptimizationfoam, where in addition to separate in different files the computation of the primal and dual velocities/pressures without any problem (to make it easier to read), I added an #include to call petsc.h (I will put my modifications in colors).
Also, the PETSc installation was done using the standard PETSc configure/install parameters (in a PETSc test case I compiled using the standard make).

At first my main code looked like this

Code:
#include "fvCFD.H"
#include "singlePhaseTransportModel.H"
#include "turbulentTransportModel.H"
#include "simpleControl.H"
#include "fvOptions.H"
#include "petsc.h"

template<class Type>
void zeroCells
(
    GeometricField<Type, fvPatchField, volMesh>& vf,
    const labelList& cells
)
{
    forAll(cells, i)
    {
        vf[cells[i]] = Zero;
    }
}


// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

int main(int argc, char *argv[])
{
    #include "postProcess.H"

    #include "setRootCase.H"
    #include "createTime.H"
    #include "createMesh.H"
    #include "createControl.H"
    #include "createFields.H"
    #include "createFvOptions.H"
    #include "initContinuityErrs.H"
    #include "initAdjointContinuityErrs.H"

    turbulence->validate();

    // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

    Info<< "\nStarting time loop\n" << endl;

    while (simple.loop())
    {
        Info<< "Time = " << runTime.timeName() << nl << endl;

		#include "updatePorosityField.H"
		
        // Pressure-velocity SIMPLE corrector
        {
			#include "UEqn.H"
			#include "pEqn.H"			
        }

        // Adjoint Pressure-velocity SIMPLE corrector
        {
			#include "UaEqn.H"
			#include "paEqn.H"
        }

        laminarTransport.correct();
        turbulence->correct();
		
		sens=Ua&U;
		
		#include "costFunction.H"

        runTime.write();

        Info<< "ExecutionTime = "
            << runTime.elapsedCpuTime()
            << " s\n\n" << endl;
    }

    Info<< "End\n" << endl;

    return 0;
}
and the options file like this

Code:
EXE_INC = \
    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \
    -I$(LIB_SRC)/transportModels \
    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
    -I$(LIB_SRC)/finiteVolume/lnInclude \
    -I$(LIB_SRC)/meshTools/lnInclude \
	-I$(PETSC_DIR}/include/lnInclude \
	-I$(PETSC_DIR}/$(PETSC_ARCH)/include/lnInclude \

EXE_LIBS = \
    -lturbulenceModels \
    -lincompressibleTurbulenceModels \
    -lincompressibleTransportModels \
    -lfiniteVolume \
    -lmeshTools \
    -lfvOptions \
	-L$(PETSC_DIR)/lib/petsc/conf \
	-L$(PETSC_LIBDIR)
Doing that I got the error fatal error: petsc.h: No such file or directory compilation terminated.

That was because I was sloppy with the parenthesis on the options file, but following a subsequent advice I did the hard coding for the petsc.h call and to include the folders in the options file.

After that, my petsc.h call was done like this on the main code

Code:
#include "/home/pablo/PETSC/3.7.4/include/petsc.h"
and the options file looks like this now (I was able to correctly write that using environmental variables)

Code:
EXE_INC = \
    -I$(LIB_SRC)/TurbulenceModels/turbulenceModels/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/incompressible/lnInclude \
    -I$(LIB_SRC)/TurbulenceModels/compressible/lnInclude \
    -I$(LIB_SRC)/transportModels \
    -I$(LIB_SRC)/transportModels/incompressible/singlePhaseTransportModel \
    -I$(LIB_SRC)/finiteVolume/lnInclude \
    -I$(LIB_SRC)/meshTools/lnInclude \
    -I$(PETSC_DIR)/include \
    -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(PETSC_DIR)/include/petsc/finclude \


EXE_LIBS = \
    -lturbulenceModels \
    -lincompressibleTurbulenceModels \
    -lincompressibleTransportModels \
    -lfiniteVolume \
    -lmeshTools \
    -lfvOptions \
    -L/home/pablo/PETSC/3.7.4/lib/petsc/conf \
    -L/home/pablo/PETSC/3.7.4/linux-dbg/lib

Now, the main code is able to find the petsc.h file and all the subsequent .h files, but stop again now with an error related to the language itself related to the "pow" operation (I guess that linking OpenFOAM with PETSc language is not as straightforward).

/home/pablo/PETSC/3.7.4/include/petscmath.h:326:38: error: call of overloaded ‘pow(PetscScalar&, PetscScalar&)’ is ambiguous
#define PetscPowScalar(a,b) pow(a,b)

Any idea on how solve this last problem related to the "pow" operator?

Last edited by palarcon; July 20, 2018 at 08:19.
palarcon is offline   Reply With Quote

Old   July 23, 2018, 09:48
Default
  #6
Member
 
Pablo Alarcón
Join Date: Mar 2018
Location: Liège
Posts: 59
Rep Power: 8
palarcon is on a distinguished road
Problem solved

Given the fact that OpenFOAM and PETSc have a definition for the pow operator, this one had an ambiguous definition.
To solve it, I had to add "::" in the pow definition inside the PETSc file, in order to avoir the confusion on the definition.
yuminghao likes this.
palarcon is offline   Reply With Quote

Old   December 28, 2018, 04:45
Smile
  #7
New Member
 
Wu Qifeng
Join Date: Mar 2016
Posts: 1
Rep Power: 0
nuaawqf is on a distinguished road
Hi Pablo:


I am interesting in how you using the MMA in PETSc to manipulate the OpenFOAM fields such as alpha. Can you show some hints in your codes?
Thanks much.
nuaawqf is offline   Reply With Quote

Old   January 6, 2019, 09:35
Default
  #8
Member
 
Pablo Alarcón
Join Date: Mar 2018
Location: Liège
Posts: 59
Rep Power: 8
palarcon is on a distinguished road
Hello and sorry for the late answer.
Finally, and for a number of reasons (mainly the fact that I couldn't find a reliably C++ version of MMA, even if there are) I declined using, at least for the moment, MMA with PETSc.
At the time I performed a lot of test case and I wasn't happy with the results.
Sorry!
palarcon is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Frequently Asked Questions about Installing OpenFOAM wyldckat OpenFOAM Installation 3 November 14, 2023 12:58
OpenFOAM Training, London, Chicago, Munich, Sep-Oct 2015 cfd.direct OpenFOAM Announcements from Other Sources 2 August 31, 2015 14:36
OpenFOAM Foundation releases OpenFOAM 2.2.2 opencfd OpenFOAM Announcements from ESI-OpenCFD 0 October 14, 2013 08:18
Adventure of fisrst openfoam installation on Ubuntu 710 jussi OpenFOAM Installation 0 April 24, 2008 15:25
The OpenFOAM extensions project mbeaudoin OpenFOAM 16 October 9, 2007 10:33


All times are GMT -4. The time now is 20:29.