[alvora]

Hello,

I tried to compile openfoam 210 on cluster and I got the error as:


Making dependency list for source file ptscotchDecomp.C
could not open file mpi.h for source file ptscotchDecomp.C
ptscotchDecomp.C:124:19: warning: missing terminating " character [enabled by default]
ptscotchDecomp.C:124:19: warning: extra tokens at end of #include directive [enabled by default]
In file included from /opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/include/mpi.h:1188:0,
from ptscotchDecomp.C:125:
/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/include/mpicxx.h:2698:34: error: declaration of C function ‘void MPI::Init(int&, char**&)’ conflicts with
/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/include/mpicxx.h:2697:13: error: previous declaration ‘void MPI::Init()’ here
/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/include/mpicxx.h:2700:45: error: declaration of C function ‘int MPI::Init_thread(int&, char**&, int)’ conflicts with
/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/include/mpicxx.h:2699:12: error: previous declaration ‘int MPI::Init_thread(int)’ here
ptscotchDecomp.C: In member function ‘Foam::label Foam:tscotchDecomp::decompose(const Foam::fileName&, const Foam::List<int>&, const Foam::List<int>&, const scalarField&, Foam::List<int>&) const’:
ptscotchDecomp.C:552:39: warning: use of old-style cast [-Wold-style-cast]
make: *** [Make/linux64Gcc46DPOptSYSTEMOPENMPI/ptscotchDecomp.o] Error 1
+ wmake libso decompositionMethods
wmakeLnInclude: linking include files to ./lnInclude
Making dependency list for source file decompositionMethod/decompositionMethod.C
Making dependency list for source file geomDecomp/geomDecomp.C

I am compiling with GCC4.6.1, SystemOpenMPI setting.

I adopted some changes in some files, such as:

in wmake/rules/linux64Gcc46/c : mpicc -m64 instead of gcc -m64,
in wmake/rules/linux64Gcc46/c++ : mpicxx -m64 instead of g++ -m64,

in etc/setting.sh,
export WM_CC='mpicc'
export WM_CXX='mpicxx'
export WM_NCOMPPROCS=10
in SYSTEMOPENMPI in communication Library part,
mpi_version=openmpi-system #mpi2
export MPI_HOME=/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1
export MPI_ARCH_PATH=/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1
export MPICH_ROOT=$MPI_ARCH_PATH

_foamAddPath $MPI_ARCH_PATH/bin
_foamAddLib $MPI_ARCH_PATH/lib

export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
unset mpi_version
;;

I typed command in terminal and result shows:
which mpicc
/opt/parastation/mpi2-5.0.26-1-gcc-4.6.1/bin/mpicc
mpicc --version
gcc (GCC) 4.6.1


except above error, everything compiled perfect.. Can anybody tell me that how can I get rid of above error, please?

Kind Regards
Al

[wyldckat]

Greetings Alpesh,

The "SYSTEMOPENMPI" option is meant for Open-MPI. If you want to proceed with this way of modifying things, you still have to reconfigure the file "wmake/rules/General/mplibSYSTEMOPENMPI". It contains this:

# use the system installed openmpi - mpicc must be installed

PFLAGS     = -DOMPI_SKIP_MPICXX
PINC       = $(shell mpicc --showme:compile)
PLIBS      = $(shell mpicc --showme:link)

At least the "PFLAGS" option needs to be reconfigured for your MPI. The one listed for Open-MPI is for disabling the inclusion of C++ MPI code, which you must find for your version of MPI.

Some other example files are shown if you run the following command, which lists the respective general configuration files for each MPI that OpenFOAM has been tested with:

ls -l wmake/rules/General/mplib*

As you can see, it would be best if you figure out which MPI exactly you are using and if it is derived from one of the ones listed.
Example:

mpirun --version

Additionally, you can also find other custom "mplib" files specific to the architecture you've chosen, by running:

ls -l wmake/rules/$WM_ARCH$WM_COMPILER/mplib*

Best regards,
Bruno

[alvora]

Hi Bruno,

Thank you for reply..
I changed the systemopenmpi part because, Initially I tried as:
in systemopenmpi part in settings.sh
export FOAM_MPI=openmpi-system
# Set compilation flags here instead of in wmake/rules/../mplibSYSTEMOPENMPI
export PINC="`mpicc -show compile`" instead of mpicc --showme:compile
export PLIBS="`mpicc -show link`" instead of mpicc --showme:link
libDir=`echo "$PLIBS" | sed -e 's/.*-L\([^ ]*\).*/\1/'`

# Bit of a hack: strip off 'lib' and hope this is the path to openmpi
# include files and libraries.
export MPI_ARCH_PATH="${libDir%/*}"

if [ "$FOAM_VERBOSE" -a "$PS1" ]
then
echo "Using system installed MPI:"
echo " compile flags : $PINC"
echo " link flags : $PLIBS"
echo " libmpi dir : $libDir"
fi

_foamAddLib $libDir
unset libDir
;;
I got same error as above (in ptscotch.C , UIPread.C, UOPwrite.C and UIPread.C files)

After, I tried to compile with Gcc 4.6.1 and MPICH2.1.1.1p1, I got error in only ThirdParty stuff, But no any errors in foam compilation.. I got errors as below :

/bin/sh: line 2: cd: ch_p4: No such file or directory
In file included from ../../../include/mpiimpl.h:121:0,
from mpidimpl.h:22,
from mpidu_process_locks.h:20,
from mpidu_process_locks.c:6:
../../../include/mpidpre.h:13:28: fatal error: mpidi_ch3_conf.h: No such file or directory
compilation terminated.
make[4]: *** [mpidu_process_locks.o] Error 1
make[3]: *** [all-redirect] Error 2
make[2]: *** [all-redirect] Error 2
make[1]: *** [all-redirect] Error 2
make: *** [all-redirect] Error 2
+ make distclean
/bin/sh: line 2: cd: ch_p4: No such file or directory
/bin/sh: line 2: cd: ch_p4: No such file or directory
/bin/sh: line 2: cd: ch_p4: No such file or directory
make[4]: *** No rule to make target `clean'. Stop.
make[3]: [clean-local] Error 2 (ignored)
+ '[' -r /opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.1.1p1 ']'
+ wmakePrintBuild -check
no git description found

what is the ch_p4? ch_p4 word used in MPICH part in Allwmake file of ThirdParty directory:
./configure \
--prefix=$MPI_ARCH_PATH \
--without-mpe \
--disable-f77 \
--disable-f90 \
--disable-f90modules \
--disable-c++ \
--disable-mpedbg \
--disable-devdebug \
--disable-debug \
--enable-sharedlib=$MPI_ARCH_PATH/lib \
--with-device=ch_p4 \
;

Can you tell me how can I get rid from this error, please?

Kind Regards
Alp

[wyldckat]

Hi Alpesh,

You better upgrade the MPICH2 version to 1.4.1p1: http://www.mcs.anl.gov/research/proj...balls/1.4.1p1/
Then try again building it. Don't forget to modify OpenFOAM's "etc/config/settings.sh" accordingly and start a new terminal, to make sure that the new settings are active!

Good luck!
Bruno

[alvora]

Hi bruno,

Thank you ..

should I have to install also hydra-1.4.1p1 or only mpich2-1.4.1p1 ?

Kind Regards
alpesh

[alvora]

Hi Bruno,

I tried with mpich2-1.4.1p1.
when I compiled with Allwmake file in ThirdParty, I got error as (see awmake_default.err file in attachment):

*Configuring MPICH2 version 1.4.1p1 with '--prefix=/opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.4.1p1' '--without-mpe' '--disable-f77' '--disable-f90' '--disable-f90modules' '--disable-c++' '--disable-mpedbg' '--disable-devdebug' '--disable-debug' '--enable-sharedlib=/opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.4.1p1/lib' '--with-device=ch_p4' 'CC=mpicc' 'CFLAGS=-m64 -configure: error:
--enable-f90 and --disable-f90 are NO longer valid configure options,
use --enable-fc or --disable-fc instead
+ make -j 8

After I made change in Allwmake file as previous error suggest,
I used --disable-fc instead of --disable-f90...
Again I got same error as thread_no.3 + some more errors
please, look the 2nd file awmake_change.err in attachment...
ch_p4 device is still unknown..

What should I have to do?

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

I forgot to mention that you should also try removing this line:

--with-device=ch_p4 \

Best regards,
Bruno

[alvora]

Hi Bruno,

Thank for help..

But, still one more error..

/usr/bin/ld: cannot find -lmpich
collect2: ld returned 1 exit status
make[2]: *** [mpich2version] Error 1
make[1]: *** [all-redirect] Error 2
make: *** [all-redirect] Error 2
+ make distclean
make[4]: *** No rule to make target `clean'. Stop.
make[3]: [clean-local] Error 2 (ignored)
+ '[' -r /opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.1.1p1 ']'

please look in attached file...

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

The log you provided only has the error list. This way I can't figure out where the errors actually occurred

Nonetheless, I remembered a similar simulation where the library files were placed in the "lib64" folder instead of "lib". In your can, check the folder: "/opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.4.1p1" and check if only "lib64" exists.

So, depending on the result you get:

• If only the "lib64" folder exists, run:
  

  cd /opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/mpich2-1.4.1p1
  ln -s lib64 lib

• If the two folders already exist, check the contents for "libmpich.so*".
• If this isn't it, I'll need the complete (stdout+stderr) log for ThirdParty's "Allwmake".

Best regards,
Bruno

[alvora]

Hi Bruno,

Thank you for helping..

I don't have mpich2-1.4.1p1 folder under "/opt/las/OpenFOAM/ThirdParty-2.1.0/platforms/linux64Gcc46/" link..

I have directory structure in "/opt/las/OpenFOAM/ThirdParty-2.1.0/plateforms" as below:
plateforms directory consists three sub directories, 1) linux64, 2)linux64Gcc46, 3)linux64Gcc46DPOpt..
.
|-- linux64
| |-- gcc-4.6.1
| | |-- bin
| | |-- include
| | |-- lib
| | |-- lib64
| | |-- libexec
| | `-- share
| |-- gmp-5.0.2
| | |-- include
| | |-- lib
| | `-- share
| |-- mpc-0.9
| | |-- include
| | |-- lib
| | `-- share
| `-- mpfr-3.0.1
| |-- include
| |-- lib
| `-- share
|-- linux64Gcc46
| |-- cmake-2.8.8
| | |-- bin
| | |-- doc
| | |-- man
| | `-- share
| |-- paraview-3.12.0
| | |-- bin
| | |-- include
| | |-- lib
| | `-- share
| |-- qt-4.7.3
| | |-- bin
| | |-- doc
| | |-- imports
| | |-- include
| | |-- lib
| | |-- mkspecs
| | |-- phrasebooks
| | |-- plugins
| | |-- q3porting.xml
| | `-- translations
| `-- scotch_5.1.11
| |-- bin
| |-- include
| `-- share
`-- linux64Gcc46DPOpt
`-- lib
|-- libscotch.so
|-- libscotcherr.so
|-- libscotcherrexit.so
|-- libscotchmetis.so
`-- mpich2-1.4.1p1

48 directories, 5 files

mpich2-1.4.1p1 directory is under "/plateforms/linux64Gcc46DPOpt/lib/"
and it consists four files as below:

.
|-- libptscotch.so
|-- libptscotcherr.so
|-- libptscotcherrexit.so
`-- libptscotchparmetis.so

0 directories, 4 files


I tried with mpich2-1.1.1p1 as well as with mpich2-1.4.1p1..
For mpich2-1.4-1p1, I made change--disable-fc instead of --disable-f90 in Allwmake file in ThirdParty..

Please see the attach file for both case...

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

I can't figure out these logs because they're out of sequence. Run Allwmake like this:

./Allwmake > make.log 2>&1

This way it will be all registered in the correct sequence.

Best regards,
Bruno

[alvora]

Hi Bruno,

Please look the attachment... for both mpich case..

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

This error seems familiar... try this:

1. Go into the folder "mpich-1.4.1p1".
2. Run:

   ln -s lib lib64

3. Repeat for mpich2 1.1.1.

Then run Allwmake once again.

edit: It's familiar because of this report of mine: http://www.openfoam.org/mantisbt/view.php?id=400
It might be a good idea to use OpenFOAM 2.1.1 instead, since this should already be fixed there...

Best regards,
Bruno

[alvora]

Hi Bruno,

still same error..

please, look attachment

after run above command, there are two folders lib and lib64 under "/opt/las/OpenFOAM/ThirdParty-2.1.0/mpich2-1.4.1p1/" directory..

both folder consist below files:
.
|-- libmpl.a
|-- libmpl.la
|-- libopa.a
|-- libopa.la
`-- pkgconfig
`-- openpa.pc

1 directory, 5 files

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

OK, I've finally tested this myself and figured out what the problem is:

Originally Posted by http://trac.mcs.anl.gov/projects/mpich2/ticket/170

Unfortunately parallel make (make -j [whatever]) is not supported by the mpich2 build process at this time due to the way that we build libraries and the fact that "ar" is not safe for concurrent use.

In other words, edit the Allwmake once more and remove "-j $WM_NCOMPPROCS" from this line for MPICH:

make -j $WM_NCOMPPROCS && make install

It will probably still give some errors, but they are not crucial, because they'll be related to the clean up part. Run Allwmake a second time to confirm that all is well

Best regards,
Bruno

[alvora]

Hi Bruno,

Thank you very much for help...

please, look the attachment.. I run as u told.. there are some minor errors..
I think they are from clean up part.. It seems to be succeed.. Because I didn't get any other error apart from these...
Can you look that are they crucial or not?
I am also interested to install openfoam211.. for openfoam211, should I have to make same changes whatever I made for openfam210?

Thank you very much again for your efforts...

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

It looks like it built correctly, although it looked like there were two logs in that file...

And yes, the same modifications to "ThirdParty-2.1.1/Allwmake" is required. Using MPICH2 1.4.1p1 is also advisable, I guess.

As for being able to use the two or more OpenFOAM versions in the same system: Advanced tips for working with the OpenFOAM shell environment

Best regards,
Bruno

[wyldckat]

Just to wrap this up - I've reported this issue with building MPICH2 in parallel here: http://www.openfoam.org/mantisbt/view.php?id=613

And apparently, since MPICH2 1.5a1 was released, it can build in parallel without any problems: http://wiki.mcs.anl.gov/mpich2/index...parallel_make.

[alvora]

Hi Bruno,

I need one more help from you please...
I tried to install openfoam210 in one cluster...
I got error only in ansysToFoam compilation.. and error as:

I/home/b/bbiavora/OpenFOAM/OpenFOAM-2.1.0/src/OpenFOAM/lnInclude -I/home/b/bbiavora/OpenFOAM/OpenFOAM-2.1.0/src/OSspecific/POSIX/lnInclude -fPIC -Xlinker --add-needed -Xlinker --no-as-needed Make/linux64Gcc46DPOpt/ansysToFoam.o -L/home/b/bbiavora/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64Gcc46DPOpt/lib \
-lOpenFOAM -ldl -lm -o /home/b/bbiavora/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64Gcc46DPOpt/bin/ansysToFoam
/usr/lib/../lib64/crt1.o: In function `_start':
init.c.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status
make: *** [/home/b/bbiavora/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64Gcc46DPOpt/bin/ansysToFoam] Error 1


what to do to compile ansysToFoam successfully? I don't have permission to write in /usr/lib or lib64 directory...

I also adopted below setting in Allwmake file of ThirdParty..

CC=/home/b/bbiavora/OpenFOAM/ThirdParty-2.1.1/platforms/linux64/gcc-4.6.1/bin/gcc
CXX=/home/b/bbiavora/OpenFOAM/ThirdParty-2.1.1/platforms/linux64/gcc-4.6.1/bin/g++
export CC
export CXX

I added above four lines before ./configure line...

Kind Regards
Alpesh

[wyldckat]

Hi Alpesh,

Edit the file "OpenFOAM-2.1.1/etc/bashrc" and find this code:

#- Compiler location:
#    foamCompiler= system | ThirdParty (OpenFOAM)
foamCompiler=system

And change it to:

#- Compiler location:
#    foamCompiler= system | ThirdParty (OpenFOAM)
foamCompiler=ThirdParty

This way OpenFOAM's build system will select that custom build you have of GCC 4.6.1.

By the way, do you have flex installed in your cluster? If so, which version?

flex --version

Best regards,
Bruno