CFD Online Discussion Forums - installation with Intel MPI and Intel compilers

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)

- OpenFOAM Installation (https://www.cfd-online.com/Forums/openfoam-installation/)

- - installation with Intel MPI and Intel compilers (https://www.cfd-online.com/Forums/openfoam-installation/93788-installation-intel-mpi-intel-compilers.html)

achim

October 26, 2011 12:10

installation with Intel MPI and Intel compilers

After running with the standard gcc and open-mpi configuration I'd like to install and test with Intel's compilers and MPI.

Allwmake tells me:
.../OpenFOAM/OpenFOAM-1.7.1/lib/linux64IccDPOpt/libscotch.so: undefined reference to `pthread_detach'

Where should this pthread_detach come from?

Changes to etc/bashrc:
# Compiler: set to Gcc, Gcc43, Gcc44, or Icc (for Intel's icc)
: ${WM_COMPILER:=Icc}; export WM_COMPILER
and added later in the file appropriate compiler options for the Intel compilers

etc/settings.sh:
INTELMPI)
mpi_version=intelmpi-3.2.2.006
export MPI_ARCH_PATH=$I_MPI_ROOT
# echo $I_MPI_ROOT
# /opt/intel/impi/3.2.2.006

# Tell OpenMPI where to find its install directory
# export OPAL_PREFIX=$MPI_ARCH_PATH

# _foamAddPath $MPI_ARCH_PATH/bin
_foamAddPath $MPI_ARCH_PATH/bin64
# _foamAddLib $MPI_ARCH_PATH/lib
_foamAddLib $MPI_ARCH_PATH/lib64
_foamAddMan $MPI_ARCH_PATH/man

export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
unset mpi_version
;;

Regards
Achim

wyldckat

November 4, 2011 14:41

Greetings Achim,

I haven't tried building OpenFOAM 1.7.1 with Icc, but I can tell you that in OpenFOAM 2.0.x the Scotch libraries are built with Gcc, precisely due to those issues you are getting.

You can see this particular fix in action here: https://github.com/OpenFOAM/ThirdPar.../Allwmake#L221 - it starts in line 221.

Best regards,
Bruno

achim

November 6, 2011 08:35

Quote:

Originally Posted by wyldckat (Post 330769)

Thanks Bruno,

so I switched to 2.0.1, but still ran into some error:

+ decompose/Allwmake
+ wmakeLnInclude decompositionMethods
+ '[' -n /work/appl/OpenFOAM/ThirdParty-2.0.1/platforms/linux64Icc/scotch_5.1.11 ']'
+ wmake libso scotchDecomp
Making dependency list for source file scotchDecomp.C
+ '[' -d /work/appl/OpenFOAM/OpenFOAM-2.0.1/platforms/linux64IccDPOpt/lib/ ']'
+ wmakeMpiLib ptscotchDecomp
+ set +x
ptscotchDecomp.C(126): catastrophic error: cannot open source file "ptscotch.h"
#include "ptscotch.h"
^

compilation aborted for ptscotchDecomp.C (code 4)
make: *** [Make/linux64IccDPOptINTELMPI/ptscotchDecomp.o] Error 4

So it seems something in my setup is still wrong ...

Any idea?

Regards
Achim

wyldckat

November 6, 2011 08:43

Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:

Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:

Code:

./Allwmake > make.log 2>&1

This will put every single output generated during the building process into the file "make.log".
Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
Pack the file into a tarball:

Code:

tar -czf make.log.tar.gz make.log
Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.

Best regards,
Bruno

achim

November 6, 2011 09:02

1 Attachment(s)

Quote:

Originally Posted by wyldckat (Post 330929)

Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:

Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:

Code:

./Allwmake > make.log 2>&1

This will put every single output generated during the building process into the file "make.log".
Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
Pack the file into a tarball:

Code:

tar -czf make.log.tar.gz make.log
Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.

Best regards,
Bruno

Hi Bruno,

here we go (I have split out and error into 2 files, hope that's OK).

Best regards
Achim

wyldckat

November 6, 2011 09:18

Hi Achim,

I wanted the output and error messages to be in the same file, so I could have an historic detailing of where/when the errors occurred. This way I have separate history lines...

Either way, my suspicion is that you edited the following code in "etc/config/settings.sh":

Code:

    x86_64)

        case "$WM_ARCH_OPTION" in

        32)

            export WM_COMPILER_ARCH=64

            export WM_CC='gcc'

            export WM_CXX='g++'

            export WM_CFLAGS='-m32 -fPIC'

            export WM_CXXFLAGS='-m32 -fPIC'

            export WM_LDFLAGS='-m32'

            ;;

        64)

            WM_ARCH=linux64

            export WM_COMPILER_LIB_ARCH=64

            export WM_CC='gcc'

            export WM_CXX='g++'

            export WM_CFLAGS='-m64 -fPIC'

            export WM_CXXFLAGS='-m64 -fPIC'

            export WM_LDFLAGS='-m64'

            ;;

If you changed the lines with "WM_CC" and "WM_CXX" to use "icc", then undo those changes! Like I wrote before, Scotch will not build with Icc... at least not without substantial hacking - here's the proof: http://www.openfoam.com/mantisbt/view.php?id=323

By the way, knowing which Gcc version and Linux architecture would make it easier for me to help you. To know the architecture, run:

Code:

uname -m

Best regards,
Bruno

achim

November 7, 2011 03:46

Installation

1 Attachment(s)

Hi Bruno,

here it comes:

# cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1

# uname -m
x86_64

# uname -a
Linux machine 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 x86_64 x86_64 x86_64 GNU/Linux

# gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

Best regards
Achim

wyldckat

November 7, 2011 03:56

Hi Achim,

Could you please run the command with the "2>&1" in the end!?

Code:

./Allwmake > make.log 2>&1

Or:

Code:

./Allwmake 1>> make2in1.log 2>> make2in1.log

Don't forget the new name on this one: "make2in1.log".

Because there seems to have been problems building ptscotch, but those errors aren't in the file you posted!

Best regards,
Bruno

achim

November 7, 2011 04:13

installation

1 Attachment(s)

Dear Bruno,

I ran as you proposed. So for the ThirdParty stuff it seems to be OK ...
Just to be sure, here it is again (, ThirdParty stuff only, after Allclean):
./Allwmake > make.log 2>&1

Best regards
Achim

wyldckat

November 7, 2011 04:21

Then something strange is going on... there is no indication that "ptscotch" built at all :( You'll probably run into the same problem as before.

achim

November 8, 2011 10:02

I checked the settings during excution of ThirdParty.../Allwmake and
found 2 strange things:
#1
SCOTCH_ARCH_PATH in Allwmake is
OpenFOAM/ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11
though normally I'm doing OPT and DP, i.e. I would have expected this to be
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPOpt
This seems wrong to me, right?

#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))

Regards
Achim

wyldckat

November 9, 2011 13:41

Hi Achim,

Quote:

Originally Posted by achim (Post 331206)

Scotch is built by Allwmake with the default of splitting into two main folders:

"ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
"ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.

Quote:

Originally Posted by achim (Post 331206)

#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))

That's normal. To use Intel's MPI, add a new entry to "settings.sh" and adjust accordingly, based on what you can see on the other MPI options. I think your initial modification would work, but something was missing.

Right now I can't spend any time looking into this in detail, so all I can say is what I've already written ;)

Best regards,
Bruno

achim

November 10, 2011 05:56

Quote:

Originally Posted by wyldckat (Post 331419)

Hi Achim,

Scotch is built by Allwmake with the default of splitting into two main folders:

"ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
"ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.

Hi Bruno,

in the meantime I got further: I could link a complete executable and start it, but so far it error exited. (I have another urgent activity, so it may take some time until I can spend cycles on it, but I will keep you updated.)

Many thanks for all your help, Bruno!

Bets regards
Achim

All times are GMT -4. The time now is 18:48.