CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

OF15x with MPICH2

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   February 25, 2009, 12:47
Default Hi all, I'm used to work wi
  #1
Member
 
Elisabet Mas de les Valls
Join Date: Mar 2009
Location: Barcelona, Spain
Posts: 61
Rep Power: 8
elisabet is on a distinguished road
Hi all,

I'm used to work with OF using OpenMPI, but now I have access to a cluster that uses MPICH2. Hence, I guess some specific changes have to be done in both bashrc and settings.sh files. However, among the available options I only find MPICH, is it ok to keep MPICH but in settings.sh introduce the right MPICH2 version and location?

Moreover, should I have something else in mind? That is, are there special treats for Pstream compilation with PMICH2? I have not been able to find the information I need in previous posts.

Any information is welcome!

Regards,

elisabet
elisabet is offline   Reply With Quote

Old   February 26, 2009, 07:21
Default Still with this... When com
  #2
Member
 
Elisabet Mas de les Valls
Join Date: Mar 2009
Location: Barcelona, Spain
Posts: 61
Rep Power: 8
elisabet is on a distinguished road
Still with this...

When compiling Pstream I get the following error:

/usr/bin/ld: /shared/source/mpich2-1.0.6p1/lib/libmpich.a(allreduce.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/shared/source/mpich2-1.0.6p1/lib/libmpich.a: could not read symbols: Bad value
collect2: ld returned 1 exit status


I guess it has something to do with the rules in my mplibMPICH file, as stated in the wiki, but I don't know how to correct it. Any ideas?

Regards,

elisabet
elisabet is offline   Reply With Quote

Old   February 27, 2009, 01:54
Default I have a cluster running OF-1.
  #3
Member
 
Martin Aunskjaer
Join Date: Mar 2009
Location: Denmark
Posts: 48
Rep Power: 8
aunola is on a distinguished road
I have a cluster running OF-1.5 in parallel using MPICH2. I had no problems rebuilding PStream. It's been a while since I did it, and I don't remember the exact steps. However, I did take notes. I don't have them at hand right now, but I will try to dig them out.
aunola is offline   Reply With Quote

Old   February 27, 2009, 04:33
Default That would be perfect! Than
  #4
Member
 
Elisabet Mas de les Valls
Join Date: Mar 2009
Location: Barcelona, Spain
Posts: 61
Rep Power: 8
elisabet is on a distinguished road
That would be perfect!

Thanks Martin,

elisabet
elisabet is offline   Reply With Quote

Old   February 27, 2009, 11:34
Default Turns out my notes are for OF-
  #5
Member
 
Martin Aunskjaer
Join Date: Mar 2009
Location: Denmark
Posts: 48
Rep Power: 8
aunola is on a distinguished road
Turns out my notes are for OF-1.4.1. I had a quick look at OF-1.5.x. The names and contents of sources files is different, but this is what I believe you should do, assuming your shell is bash:

1) Open $WM_PROJECT_DIR/etc/bashrc

Uncomment the line (put a # in front of it)

: ${WM_LIB:=OPENMPI}; export WM_LIB

and add this line

: ${WM_LIB:=MPICH2}; export WM_LIB

This will make MPICH2 the default MPI selection.


2) In $WM_PROJECT_DIR/etc/settings.sh find where all the MPI stuff is and add this (your paths and version may differ):

MPICH2)

export MPICH2_VERSION=1.0.5
export MPICH2_ARCH_PATH=/home/mar/mpich2-install
export MPICH2_PATH=$MPICH2_ARCH_PATH
export MPICH2_ROOT=$MPICH2_ARCH_PATH
_foamAddLib $MPICH2_ARCH_PATH/lib
_foamAddLib $MPICH2_ARCH_PATH/bin
export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/mpich2-$MPI_VERSION


3) In $WM_PROJECT_DIR/wmake/rules/linuxGcc add the new file mplibMPICH2 with contents:

PFLAGS =
PINC = -I$(MPICH2_ARCH_PATH)/include
PLIBS = -L$(MPICH2_ARCH_PATH)/lib

This file tells the compiler and linker where to find the include files and MPICH2 libraries respectively.


4) In $WM_PROJECT_DIR/src/Pstream/Allwmake add MPICH2 to the case statement:

... | MPICH2 | ...)



5) In $WM_PROJECT_DIR/src/Pstream

./Allwmake


Let me know if there are problems. Later this year I am planning on building a new cluster with OF running MPICH2-MX on top of OpenMX, so I have a vested interest in any observations you make.
aunola is offline   Reply With Quote

Old   March 11, 2009, 12:16
Default Hi Martin, Sorry for the de
  #6
Member
 
Elisabet Mas de les Valls
Join Date: Mar 2009
Location: Barcelona, Spain
Posts: 61
Rep Power: 8
elisabet is on a distinguished road
Hi Martin,

Sorry for the delay, I've been quite busy with other stuff.

OF-1.5.x, with MPICH2, is now compiled in the cluster and works. The steps I carried out are:

1. In etc/bashrc
Quote:
# : ${WM_MPLIB:=OPENMPI
; export WM_MPLIB
: ${WM_MPLIB:=MPICH}; export WM_MPLIB # I kept MPICH for simplicity, I'm too lazy...
}

2. In etc/settings.sh (following the internal architecture of the cluster)
Quote:
MPICH)
mpi_version=mpich2-1.0.6p1
export MPI_HOME=/shared/source/$mpi_version
export MPI_ARCH_PATH=/shared
export MPICH_ROOT=$MPI_ARCH_PATH

_foamAddPath $MPI_ARCH_PATH/bin
_foamAddLib $MPI_ARCH_PATH/lib

export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
unset mpi_version
;;
3. Pstream library compilation
My old error related with fPIC was solved when I asked the cluster administrator to recompile MPICH2 in a shared way. Once this was changed, everything complied directly.

4. others
I got a strange error when compiling decompositionAgglomeration that was solved, as suggested by the error message, changing the order of the include mpi.h in the top of the file decompositionAgglomeration/parMetisDecomp/parMetisDecomp.C. The error was related to redefinition of SEEK_SET, SEEK_CUR and SEEK_END in mpi.h

Nowadays the code works perfectly in parallel using MPICH2. Thanks to everyone!

Best regards,

elisabet
elisabet is offline   Reply With Quote

Old   August 25, 2009, 23:19
Default MPICH2 Configuration
  #7
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 12
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
This is how I configure the MPICH2:

Code:
./configure \
    --without-mpe \
    --disable-f77 \
    --disable-f90 \
    --disable-f90modules \
    --disable-cxx \
    --disable-mpedbg \
    --disable-devdebug \
    --disable-debug \
    --enable-fast \
    --enable-sharedlibs=gcc \
    --with-device=ch3:nemesis \
    -prefix=$MPI_ARCH_PATH
According to the installation guide of mpich2, enable-fast and with-device=ch3:nemesis options are said to perform better, and enable-sharedlibs=gcc option is the only option for linux, I'm afraid, and it would solve the problem of fPIC complaining.
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
MPICH2 Alfonso FLUENT 2 March 13, 2009 12:48
OF15x sample utility does not accept component elisabet OpenFOAM Bugs 2 February 18, 2009 12:58
Problem compiling OF15x from git sources lhcamilo OpenFOAM Bugs 5 November 28, 2008 07:12
Problem compiling OF15x from git sources lhcamilo OpenFOAM Bugs 1 November 27, 2008 09:36
mpich2 build for hpf taw Main CFD Forum 0 April 6, 2008 03:34


All times are GMT -4. The time now is 23:21.