CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM (https://www.cfd-online.com/Forums/openfoam/)
-   -   Linking PETSc with OpenFoam (https://www.cfd-online.com/Forums/openfoam/92504-linking-petsc-openfoam.html)

Argen September 15, 2011 19:38

Linking PETSc with OpenFoam
 
I am trying to link a linear solver based on PETSc with OpenFoam. But I got some errors,

-L/opt/64/intel/fce/9.1.045/lib \ -L/home/s/OpenFOAM/ThirdParty/gcc-4.3.1/platforms/linux64/lib64/ \ ./libP1Module.a -lfiniteVolume
-lOpenFOAM -liberty -ldl \ -lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \ -lm -o /home/s/OpenFOAM/s-1.5/applications
/bin/linux64GccDPOpt/PPPP ./libP.a(mypetsc.o)(.text+0x3c): In function `linearsolvers_petsc_mp_solvelinearsystem_petsc_':
./libP.a(mypetsc.o)(.text+0x57):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:100: undefined reference to `mpi_comm_rank_'
./libP.a(mypetsc.o)(.text+0x72):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:101: undefined reference to `mpi_comm_size_'
./libP.a(mypetsc.o)(.text+0x8d):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:113: undefined reference to `matcreate_'
./libP.a(mypetsc.o)(.text+0xb2):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:114: undefined reference to `matsetsizes_'
./libP.a(mypetsc.o)(.text+0xc3):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:115: undefined reference to `matsetfromoptions_'
./libP.a(mypetsc.o)(.text+0xde):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:121: undefined reference to `matgetownershiprange_'

Any idea to solve it?

boger September 16, 2011 08:17

Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.

Argen September 16, 2011 11:45

Quote:

Originally Posted by boger (Post 324390)
Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.

Hi David,

Yes. I am sure that the appropriate PETSc header files were included. I tested it alone and found no problem. I also added libP.a generated from my entire code with one module based on petsc in the Make/options files. So, can you figure out what I should do next?

Thanks,
Ray

boger September 16, 2011 11:53

Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?

boger September 16, 2011 12:12

Try something like this in your Make/options:

Code:

include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
    $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm

This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.

Argen September 16, 2011 12:14

Quote:

Originally Posted by boger (Post 324425)
Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?

The Make/options file is

EXE_INC = \
-I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
./libP.a \
-lfiniteVolume

where libP.a is generated from my entire code. There is no problem to build libP.a.

I changed the linear-solver module using petsc to a program and then run it.

Argen September 16, 2011 12:21

Quote:

Originally Posted by boger (Post 324429)
Try something like this in your Make/options:

Code:

include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
    $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm

This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.

When the above modification is applied, I got a new error like,
-lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D
/usr/bin/ld: cannot find -lpetscmat
collect2: ld returned 1 exit status
make: *** [/home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D] Error 1

Any idea about it?

boger September 16, 2011 12:39

Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.

Argen September 16, 2011 15:22

Quote:

Originally Posted by boger (Post 324436)
Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.

I already set the first two. But I have no idea about the third one. Can you let me know that?

boger September 16, 2011 15:42

PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.

Argen September 16, 2011 16:06

Quote:

Originally Posted by boger (Post 324450)
PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.

First, PETSCLIB_DIR should be PETSC_LIBDIR mentioned before. Is it right?
In my case, two libaries (lib1.so and lib2.so) are stored in /home/s/PETSc/petsc-3.1-p8 which is the same as $PETSC_DIR. Well, after setting PETSC_LIBDIR, I got almost same error. The only difference is that
/usr/bin/ld: cannot find -lpetsc

Before, it's /usr/bin/ld: cannot find -lpetscmat

boger September 16, 2011 16:29

As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.

Argen September 16, 2011 16:41

Quote:

Originally Posted by boger (Post 324457)
As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.

Do you still remember which version you used?

boger September 16, 2011 16:54

3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.

Argen September 16, 2011 17:20

Quote:

Originally Posted by boger (Post 324459)
3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.

Yes, I think so. But even those -lpet* are deleted, the errors become
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/PPP
./libP1Module.a(mypetsc.o)(.text+0xb2): In function `solvelinearsystem_petsc':
/~/OpenFOAM/s/applications/solvers/p/pp/mypetsc.F90:99: undefined reference to `petscinitialize_'
...

I wonder that you called some petsc subroutines inside OpenFOAM or called them in your own module linked with OF?
Anyway, is it due to something wrong with building libP.a from my code?

boger September 16, 2011 17:24

Sorry, my only experience is calling PETSc directly from within OpenFOAM.

Argen September 16, 2011 17:36

Quote:

Originally Posted by boger (Post 324464)
Sorry, my only experience is calling PETSc directly from within OpenFOAM.

That's fine. Thanks a lot.


All times are GMT -4. The time now is 09:34.