CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

Linking PETSc with OpenFoam

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   September 15, 2011, 18:38
Question Linking PETSc with OpenFoam
  #1
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
I am trying to link a linear solver based on PETSc with OpenFoam. But I got some errors,

-L/opt/64/intel/fce/9.1.045/lib \ -L/home/s/OpenFOAM/ThirdParty/gcc-4.3.1/platforms/linux64/lib64/ \ ./libP1Module.a -lfiniteVolume
-lOpenFOAM -liberty -ldl \ -lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \ -lm -o /home/s/OpenFOAM/s-1.5/applications
/bin/linux64GccDPOpt/PPPP ./libP.a(mypetsc.o)(.text+0x3c): In function `linearsolvers_petsc_mp_solvelinearsystem_petsc_':
./libP.a(mypetsc.o)(.text+0x57):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:100: undefined reference to `mpi_comm_rank_'
./libP.a(mypetsc.o)(.text+0x72):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:101: undefined reference to `mpi_comm_size_'
./libP.a(mypetsc.o)(.text+0x8d):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:113: undefined reference to `matcreate_'
./libP.a(mypetsc.o)(.text+0xb2):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:114: undefined reference to `matsetsizes_'
./libP.a(mypetsc.o)(.text+0xc3):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:115: undefined reference to `matsetfromoptions_'
./libP.a(mypetsc.o)(.text+0xde):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:121: undefined reference to `matgetownershiprange_'

Any idea to solve it?

Last edited by Argen; September 15, 2011 at 19:40.
Argen is offline   Reply With Quote

Old   September 16, 2011, 07:17
Default
  #2
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 10:45
Default
  #3
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.
Hi David,

Yes. I am sure that the appropriate PETSc header files were included. I tested it alone and found no problem. I also added libP.a generated from my entire code with one module based on petsc in the Make/options files. So, can you figure out what I should do next?

Thanks,
Ray
Argen is offline   Reply With Quote

Old   September 16, 2011, 10:53
Default
  #4
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 11:12
Default
  #5
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
Try something like this in your Make/options:

Code:
include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
     $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm
This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 11:14
Default
  #6
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?
The Make/options file is

EXE_INC = \
-I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
./libP.a \
-lfiniteVolume

where libP.a is generated from my entire code. There is no problem to build libP.a.

I changed the linear-solver module using petsc to a program and then run it.
Argen is offline   Reply With Quote

Old   September 16, 2011, 11:21
Default
  #7
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
Try something like this in your Make/options:

Code:
include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
     $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm
This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.
When the above modification is applied, I got a new error like,
-lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D
/usr/bin/ld: cannot find -lpetscmat
collect2: ld returned 1 exit status
make: *** [/home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D] Error 1

Any idea about it?
Argen is offline   Reply With Quote

Old   September 16, 2011, 11:39
Default
  #8
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.
__________________
David A. Boger

Last edited by boger; September 16, 2011 at 11:40. Reason: fix link
boger is offline   Reply With Quote

Old   September 16, 2011, 14:22
Default
  #9
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.
I already set the first two. But I have no idea about the third one. Can you let me know that?
Argen is offline   Reply With Quote

Old   September 16, 2011, 14:42
Default
  #10
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 15:06
Default
  #11
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.
First, PETSCLIB_DIR should be PETSC_LIBDIR mentioned before. Is it right?
In my case, two libaries (lib1.so and lib2.so) are stored in /home/s/PETSc/petsc-3.1-p8 which is the same as $PETSC_DIR. Well, after setting PETSC_LIBDIR, I got almost same error. The only difference is that
/usr/bin/ld: cannot find -lpetsc

Before, it's /usr/bin/ld: cannot find -lpetscmat
Argen is offline   Reply With Quote

Old   September 16, 2011, 15:29
Default
  #12
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 15:41
Default
  #13
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.
Do you still remember which version you used?
Argen is offline   Reply With Quote

Old   September 16, 2011, 15:54
Default
  #14
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 16:20
Default
  #15
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.
Yes, I think so. But even those -lpet* are deleted, the errors become
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/PPP
./libP1Module.a(mypetsc.o)(.text+0xb2): In function `solvelinearsystem_petsc':
/~/OpenFOAM/s/applications/solvers/p/pp/mypetsc.F90:99: undefined reference to `petscinitialize_'
...

I wonder that you called some petsc subroutines inside OpenFOAM or called them in your own module linked with OF?
Anyway, is it due to something wrong with building libP.a from my code?
Argen is offline   Reply With Quote

Old   September 16, 2011, 16:24
Default
  #16
Senior Member
 
David Boger
Join Date: Mar 2009
Location: Penn State Applied Research Laboratory
Posts: 146
Rep Power: 8
boger is on a distinguished road
Sorry, my only experience is calling PETSc directly from within OpenFOAM.
__________________
David A. Boger
boger is offline   Reply With Quote

Old   September 16, 2011, 16:36
Default
  #17
Member
 
Join Date: Jan 2010
Posts: 44
Rep Power: 7
Argen is on a distinguished road
Quote:
Originally Posted by boger View Post
Sorry, my only experience is calling PETSc directly from within OpenFOAM.
That's fine. Thanks a lot.
Argen is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linking ALGLIB to OpenFOAM Phicau OpenFOAM Programming & Development 6 January 12, 2012 17:15
Cross-compiling OpenFOAM 1.7.0 on Linux for Windows 32 and 64bits with Mingw-w64 wyldckat OpenFOAM Announcements from Other Sources 3 September 8, 2010 06:25
Modified OpenFOAM Forum Structure and New Mailing-List pete Site News & Announcements 0 June 29, 2009 05:56
64bitrhel5 OF installation instructions mirko OpenFOAM Installation 2 August 12, 2008 18:07
OpenFOAM Debian packaging current status problems and TODOs oseen OpenFOAM Installation 9 August 26, 2007 13:50


All times are GMT -4. The time now is 01:48.