Recompilation errors MPI related
 

Hi,

I try to recompile the development sources from prof. Jasak. Since I work on a RedHat EL4 machine , I had to work around the 'demangle.h' error by partially re-installing binutils.

At this moment, I am able to successfully compile the libraries in /src, but compilation of all applications give the following errors. It seems that there is a missing link to MPI from the applications sources.

================================================== =========================

make: Nothing to be done for `allFiles'.
make: `Make/linuxGcc4DPOpt/dependencies' is up to date.

/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/wmake/bashScripts/mkObjectDir /home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/applications/bin/linuxGcc4DPOpt/icoF oam
g++ -m32 -Dlinux -DDP -Wall -W -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-30 -I/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/src/finiteVolume/lnInclude -I/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/src/OpenFOAM/lnInclude -IlnInclude -I. -fPIC -pthread Make/linuxGcc4DPOpt/icoFoam.o -L/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt \
-lfiniteVolume -lOpenFOAM -liberty -o /home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/applications/bin/linuxGcc4DPOpt/icoF oam
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `MPI::Is_initialized()'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `PMPI::Comm::mpi_comm_map'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Free()'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `op_intercept'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `PMPI::Intracomm::current_op'
/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/lib/linuxGcc4DPOpt/lam-7.1.1/libPstr eam.so: undefined reference to `PMPI::Comm::mpi_err_map'
collect2: ld returned 1 exit status
make: *** [/home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/applications/bin/linuxGcc4DPOpt/ico Foam] Error 1

================================================== =========================

Any ideas?

regards, Frank

Hi Frank, Try this: Go t
 

Hi Frank,

Try this:

Go to /home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/src/Pstream/

and execute ./Allwmake there. This will rebuild the libPstream.so library (I think).

Then try rebuilding icoFoam.

It tried that, but still the s
 

It tried that, but still the same errors.

Even when I'm in /home/frankl/OpenFOAM/OpenFOAM-1.3.27_06_06/src and conduct ./Allwmake no errors occur at all. So something strange is happening during compilation of the applications (solvers and utilities).

Regards, Frank

Obvious (sorry I didin't spot
 

Obvious (sorry I didin't spot it earlier): your directory is at the wrong place. You either need to adjust the location or put the stuff into:

/home/frankl/OpenFOAM/OpenFOAM-1.3

throwing away the date bit.

Hrv

Thanks, I will try that. What
 

Thanks, I will try that. What is the reason for this? I thought the date extension gives the opportunity to test different development versions next to each other. Now I have to keep track of changes using CVS for example.

Regards, Frank

No you don't. Make multiple c
 

No you don't. Make multiple copies of the source (if this is what you are after) with date or similar extensions and then make OpenFOAM-1.3 a link that points to the one you wish to use. Another alternative would be to call your stuff 1.3.1, 1.3.2 etc and then code in the version you wish to use. This would allow you to move between versions simply by sourcing a different file into your environment. Most places where I've worked use something similar so I don't see a problem.

The reason for the setup is a large number of paths and variables that are managed by the release.

As for the CVS, I have been playing with this idea. I personally rather enjoy working under CVS and there already exists a publicly accessible CVS for OpenFOAM. If I get enough response (and people really want access to my development on a daily basis), I don't mind using it. (Anyone else keen to contribute?)

Hrv

I now copied the sources to Op
 

I now copied the sources to OpenFOAM-1.3.1. Then I made the link OpenFOAM-1.3 -> OpenFOAM-1.3.1. All bash setting are the same as for a standard OpenFOAM-1.3 installation.

Unfortunately the same errors occur during ./Allwmake in OpenFOAM-1.3/applications:

================================================== =========================

make: `Make/linuxGcc4DPOpt/dependencies' is up to date.

/home/frankl/OpenFOAM/OpenFOAM-1.3/wmake/bashScripts/mkObjectDir /home/frankl/OpenFOAM/OpenFOAM-1.3/applications/bin/linuxGcc4DPOpt/icoFoam
g++ -m32 -Dlinux -DDP -Wall -W -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-30 -I/home/frankl/OpenFOAM/OpenFOAM-1.3/src/finiteVolume/lnInclude -I/home/frankl/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude -IlnInclude -I. -fPIC -pthread Make/linuxGcc4DPOpt/icoFoam.o -L/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt \
-lfiniteVolume -lOpenFOAM -liberty -o /home/frankl/OpenFOAM/OpenFOAM-1.3/applications/bin/linuxGcc4DPOpt/icoFoam
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `MPI::Is_initialized()'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Comm::mpi_comm_map'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Free()'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `op_intercept'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Intracomm::current_op'
/home/frankl/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Comm::mpi_err_map'
collect2: ld returned 1 exit status
make: *** [/home/frankl/OpenFOAM/OpenFOAM-1.3/applications/bin/linuxGcc4DPOpt/icoFoam] Error 1

================================================== =========================

Any more ideas?

regards, Frank

all right, I think that I solv
 

all right, I think that I solved this problem. I copied the library libPstream.so from the standard OpenFOAM-1.3 version to the approriate location in my modified version: OpenFOAM-1.3.1/lib/linuxGcc4DPOpt/dummy and OpenFOAM-1.3.1/lib/linuxGcc4DPOpt/lam-7.1.1

Any ideas why this works? Prof. Jasak, did you make any important changes to this library libPstream.so?

Regards, Frank

Sounds like your LAM is either
 

Sounds like your LAM is either configured to include the C++ MPI bindings or when compiling Pstream/mpi the C++ bindings were included.

allright. Do you now how to so
 

allright. Do you now how to solve that?

Finally, someone solved this p
 

Finally, someone solved this pb?
Any help would be greatly appreciated.
Antoine

So, a small reminder. OPenFOA
 

So, a small reminder.
OPenFOAM sources compiles and runs perfectly with the original lam-7.1.1 installation.
Libraries libPstream.a and libPstream.so are OK.
I can recompile Icofoam or any other application.

If I try to generate the same libraries with a local installation of lam-7.1.2, i get some warnings " use of old-style cast" during the wmake of Pstream/mpi, but no error.

Then, the compilation of applications like icoFoam are saying:
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `MPI::Is_initialized()'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `PMPI::Comm::mpi_comm_map'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `MPI:http://www.cfd-online.com/OpenFOAM_D...part/proud.gifp::Free()'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `op_intercept'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `PMPI::Intracomm::current_op'
/d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .so: undefined reference to `PMPI::Comm::mpi_err_map'

while the command nm gives :
nm -C /d1/users/dauptain/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/lam-7.1.2/libPstream .a > ! tmp
flubio03:~/OpenFOAM/OpenFOAM-1.3/src/Pstream/mpi % grep Is_initialized tmp
U MPI::Is_initialized()
U MPI::Is_initialized()
U MPI::Is_initialized()
flubio03:~/OpenFOAM/OpenFOAM-1.3/src/Pstream/mpi % grep PMPI::Comm::mpi_comm_map tmp
U PMPI::Comm::mpi_comm_map
U PMPI::Comm::mpi_comm_map
U PMPI::Comm::mpi_comm_map
and so on...

Any idea is welcome

Cordially.
Ant.

Do you want to compile with th
 

Do you want to compile with the local lam? I take it it's for some cluster.

I had a similar problem when being forced to run mpich instead of lam on a cluster. There wasn't an mpich with the -fPIC option available, thus forcing me to link in mpich when compiling the executable instead of linking it when compiling Pstream. I remember that for mpich, if it isn't compiled with -fPIC it's not possible to link it to a dynamic library like Pstream, you have to link it to the executable. I see no trace of this error message for you however.

Do you have a static library libPstream.a? I think if you have both present, it will pick the dynamically linked one, the .so file.

Did you check that the correct
 

Did you check that the correct lam version is referenced in the .bashrc?

eg.
if [ .$WM_MPLIB = .LAM ]; then
export LAM_VERSION=7.1.2
...

And then re-source the .OpenFOAM-<version>/bashrc
before rebuilding lam or libPstream.

BTW: you might also want to try openmpi. The first 1.2 release candidate is available. I've been using 1.2b3 without problems. We use GridEngine and the openmpi boot proceedure does everything for you.

If you do try openmpi, compile *without* threads:

./configure \
--prefix=$OPENMPI_ARCH_PATH \
--enable-shared \
--disable-static \
--disable-mpi-f77 \
--disable-mpi-f90 \
--disable-mpi-profile \
--disable-mpi-cxx

Thank you for your fast replie
 

Thank you for your fast replies.

First of all, i need to use lam-7.1.2, because a coupler and some applications are already validated and running under this one, and i want OF in the whole coupled application.

I built the libPstream.so to see if the compilation of vanilla solvers (e.g. icoFoam) was still possible using lam-7.1.2 : answer is not for the moment.

Il buidled the libPstream.a to put it in a big static independant library that will be used by
the coupler.

After some tests:
- The nm command is giving the same results with libPstream.so and libPstream.a.
- The lam version in my .cshrc is indeed 7.1.2
setenv LAM_VERSION 7.1.2
setenv LAM_ARCH_PATH $LAMHOME/platforms/$WM_OPTIONS
- lam-7.1.2 is built on my system with gcc-4.1.0, as the whole OpenFOAM stuff

Any ideas to compile Pstream with lam-7.1.2 in a way icoFoam is still compiling?
Cordially.
Ant.

>include the C++ MPI bindings
 

>include the C++ MPI bindings

Just make sure you include mpi.h before any other OF include file in all the files in the mpi Pstream. Or probably undefine the LAM_WANT_MPI2CPP in mpi.h.

Frank, just recompile the mpi library separately.
Smthg like:
libPstream.so will be rebuilt. Then openfoam again