CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

installation with Intel MPI and Intel compilers

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   October 26, 2011, 12:10
Default installation with Intel MPI and Intel compilers
  #1
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
After running with the standard gcc and open-mpi configuration I'd like to install and test with Intel's compilers and MPI.

Allwmake tells me:
.../OpenFOAM/OpenFOAM-1.7.1/lib/linux64IccDPOpt/libscotch.so: undefined reference to `pthread_detach'

Where should this pthread_detach come from?

Changes to etc/bashrc:
# Compiler: set to Gcc, Gcc43, Gcc44, or Icc (for Intel's icc)
: ${WM_COMPILER:=Icc}; export WM_COMPILER
and added later in the file appropriate compiler options for the Intel compilers

etc/settings.sh:
INTELMPI)
mpi_version=intelmpi-3.2.2.006
export MPI_ARCH_PATH=$I_MPI_ROOT
# echo $I_MPI_ROOT
# /opt/intel/impi/3.2.2.006


# Tell OpenMPI where to find its install directory
# export OPAL_PREFIX=$MPI_ARCH_PATH

# _foamAddPath $MPI_ARCH_PATH/bin
_foamAddPath $MPI_ARCH_PATH/bin64
# _foamAddLib $MPI_ARCH_PATH/lib
_foamAddLib $MPI_ARCH_PATH/lib64
_foamAddMan $MPI_ARCH_PATH/man

export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
unset mpi_version
;;

Regards
Achim
achim is offline   Reply With Quote

Old   November 4, 2011, 15:41
Default
  #2
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings Achim,

I haven't tried building OpenFOAM 1.7.1 with Icc, but I can tell you that in OpenFOAM 2.0.x the Scotch libraries are built with Gcc, precisely due to those issues you are getting.

You can see this particular fix in action here: https://github.com/OpenFOAM/ThirdPar.../Allwmake#L221 - it starts in line 221.

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   November 6, 2011, 09:35
Default
  #3
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
Quote:
Originally Posted by wyldckat View Post
Greetings Achim,

I haven't tried building OpenFOAM 1.7.1 with Icc, but I can tell you that in OpenFOAM 2.0.x the Scotch libraries are built with Gcc, precisely due to those issues you are getting.

You can see this particular fix in action here: https://github.com/OpenFOAM/ThirdPar.../Allwmake#L221 - it starts in line 221.

Best regards,
Bruno
Thanks Bruno,

so I switched to 2.0.1, but still ran into some error:

+ decompose/Allwmake
+ wmakeLnInclude decompositionMethods
+ '[' -n /work/appl/OpenFOAM/ThirdParty-2.0.1/platforms/linux64Icc/scotch_5.1.11 ']'
+ wmake libso scotchDecomp
Making dependency list for source file scotchDecomp.C
+ '[' -d /work/appl/OpenFOAM/OpenFOAM-2.0.1/platforms/linux64IccDPOpt/lib/ ']'
+ wmakeMpiLib ptscotchDecomp
+ set +x
ptscotchDecomp.C(126): catastrophic error: cannot open source file "ptscotch.h"
#include "ptscotch.h"
^

compilation aborted for ptscotchDecomp.C (code 4)
make: *** [Make/linux64IccDPOptINTELMPI/ptscotchDecomp.o] Error 4

So it seems something in my setup is still wrong ...

Any idea?

Regards
Achim
achim is offline   Reply With Quote

Old   November 6, 2011, 09:43
Default
  #4
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:
  1. Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:
    Code:
    ./Allwmake > make.log 2>&1
    This will put every single output generated during the building process into the file "make.log".
  2. Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
  3. Pack the file into a tarball:
    Code:
    tar -czf make.log.tar.gz make.log
  4. Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   November 6, 2011, 10:02
Default
  #5
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
Quote:
Originally Posted by wyldckat View Post
Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:
  1. Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:
    Code:
    ./Allwmake > make.log 2>&1
    This will put every single output generated during the building process into the file "make.log".
  2. Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
  3. Pack the file into a tarball:
    Code:
    tar -czf make.log.tar.gz make.log
  4. Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.
Best regards,
Bruno

Hi Bruno,

here we go (I have split out and error into 2 files, hope that's OK).

Best regards
Achim
Attached Files
File Type: gz make.log_and_err.tar.gz (5.1 KB, 3 views)
achim is offline   Reply With Quote

Old   November 6, 2011, 10:18
Default
  #6
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Achim,

I wanted the output and error messages to be in the same file, so I could have an historic detailing of where/when the errors occurred. This way I have separate history lines...

Either way, my suspicion is that you edited the following code in "etc/config/settings.sh":
Code:
    x86_64)
        case "$WM_ARCH_OPTION" in
        32)
            export WM_COMPILER_ARCH=64
            export WM_CC='gcc'
            export WM_CXX='g++'
            export WM_CFLAGS='-m32 -fPIC'
            export WM_CXXFLAGS='-m32 -fPIC'
            export WM_LDFLAGS='-m32'
            ;;
        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
            export WM_CC='gcc'
            export WM_CXX='g++'
            export WM_CFLAGS='-m64 -fPIC'
            export WM_CXXFLAGS='-m64 -fPIC'
            export WM_LDFLAGS='-m64'
            ;;
If you changed the lines with "WM_CC" and "WM_CXX" to use "icc", then undo those changes! Like I wrote before, Scotch will not build with Icc... at least not without substantial hacking - here's the proof: http://www.openfoam.com/mantisbt/view.php?id=323

By the way, knowing which Gcc version and Linux architecture would make it easier for me to help you. To know the architecture, run:
Code:
uname -m
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   November 7, 2011, 04:46
Default Installation
  #7
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
Hi Bruno,

here it comes:

# cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1

# uname -m
x86_64

# uname -a
Linux machine 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 x86_64 x86_64 x86_64 GNU/Linux

# gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

Best regards
Achim


Attached Files
File Type: gz make.log.gz (3.8 KB, 3 views)
achim is offline   Reply With Quote

Old   November 7, 2011, 04:56
Default
  #8
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Achim,

Could you please run the command with the "2>&1" in the end!?
Code:
./Allwmake > make.log 2>&1
Or:
Code:
./Allwmake 1>> make2in1.log  2>> make2in1.log
Don't forget the new name on this one: "make2in1.log".

Because there seems to have been problems building ptscotch, but those errors aren't in the file you posted!

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   November 7, 2011, 05:13
Default installation
  #9
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
Dear Bruno,

I ran as you proposed. So for the ThirdParty stuff it seems to be OK ...
Just to be sure, here it is again (, ThirdParty stuff only, after Allclean):
./Allwmake > make.log 2>&1

Best regards
Achim

Attached Files
File Type: gz make.log.gz (3.8 KB, 3 views)
achim is offline   Reply With Quote

Old   November 7, 2011, 05:21
Default
  #10
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Then something strange is going on... there is no indication that "ptscotch" built at all You'll probably run into the same problem as before.
wyldckat is offline   Reply With Quote

Old   November 8, 2011, 11:02
Default
  #11
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
I checked the settings during excution of ThirdParty.../Allwmake and
found 2 strange things:
#1
SCOTCH_ARCH_PATH in Allwmake is
OpenFOAM/ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11
though normally I'm doing OPT and DP, i.e. I would have expected this to be
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPOpt
This seems wrong to me, right?

#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))

Regards
Achim

achim is offline   Reply With Quote

Old   November 9, 2011, 14:41
Default
  #12
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,312
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Achim,

Quote:
Originally Posted by achim View Post
I checked the settings during excution of ThirdParty.../Allwmake and
found 2 strange things:
#1
SCOTCH_ARCH_PATH in Allwmake is
OpenFOAM/ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11
though normally I'm doing OPT and DP, i.e. I would have expected this to be
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPOpt
This seems wrong to me, right?
Scotch is built by Allwmake with the default of splitting into two main folders:
  • "ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
  • "ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.

Quote:
Originally Posted by achim View Post
#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))
That's normal. To use Intel's MPI, add a new entry to "settings.sh" and adjust accordingly, based on what you can see on the other MPI options. I think your initial modification would work, but something was missing.

Right now I can't spend any time looking into this in detail, so all I can say is what I've already written

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   November 10, 2011, 06:56
Default
  #13
New Member
 
Achim Boemelburg
Join Date: Oct 2011
Posts: 12
Rep Power: 5
achim is on a distinguished road
Quote:
Originally Posted by wyldckat View Post
Hi Achim,


Scotch is built by Allwmake with the default of splitting into two main folders:
  • "ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
  • "ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.


That's normal. To use Intel's MPI, add a new entry to "settings.sh" and adjust accordingly, based on what you can see on the other MPI options. I think your initial modification would work, but something was missing.

Right now I can't spend any time looking into this in detail, so all I can say is what I've already written

Best regards,
Bruno

Hi Bruno,

in the meantime I got further: I could link a complete executable and start it, but so far it error exited. (I have another urgent activity, so it may take some time until I can spend cycles on it, but I will keep you updated.)

Many thanks for all your help, Bruno!

Bets regards
Achim
achim is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Compiling OpenFOAM on hpc-fe.gbar.dtu.dk kaergaard OpenFOAM Installation 1 June 16, 2011 01:33
Error using LaunderGibsonRSTM on SGI ALTIX 4700 jaswi OpenFOAM 2 April 29, 2008 10:54
Intelbs MPI and performance tools in OpenFOAM hplum OpenFOAM 16 December 16, 2007 15:58
PBS queuing with intel MPI hartinger OpenFOAM Installation 0 August 14, 2006 06:22


All times are GMT -4. The time now is 17:27.