compile error about FJMPI
 

The linux system is "Linux bxfs1 2.6.18-164.6.1.el5 #1 SMP Tue Oct 27 11:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux".
I am trying to use the system's MPI of "/opt/FJSVplang/bin/mpiexec".

tar -xzf OpenFOAM-1.6.General.gtgz
tar -xzf ThirdParty-1.6.General.gtgz
tar -xzf ThirdParty-1.6.linux64Gcc.gtgz

Edit OpenFOAM/OpenFOAM-1.6/etc/cshrc
# WM_MPLIB = | OPENMPI | MPICH | MPICH-GM | HPMPI | GAMMA | MPI | QSMPI
if ( ! $?WM_MPLIB ) setenv WM_MPLIB FJMPI

Edit OpenFOAM/OpenFOAM-1.6/etc/settings.csh
case FJMPI:
setenv MPI_ARCH_PATH /opt/FJSVplang
setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpi
_foamAddPath $MPI_ARCH_PATH/bin
_foamAddPath /opt/FJSVpnple/bin
_foamAddPath /opt/FJSVmpi2/bin
_foamAddLib /opt/FJSVpnidt/lib
_foamAddLib /opt/FJSVplang/lib64
_foamAddLib /opt/FJSVpnple/lib
_foamAddLib /opt/FJSVmpi2/lib/sparcv9
_foamAddLib /opt/FJSVpnple/lib/sparcv9
breaksw

/OpenFOAM/OpenFOAM-1.6
./Allwmake

However, I meet the errors shown in follow. Any suggestion is appreciated.

Hi again, I found it is because mpi/libPstream.so is NOT complied successfully. But what a pity I have NO idea to solve it. Any suggestions are appreciated.

++ '[' FJMPI = MPI -o -d /opt/FJSVplang ']'
++ cd ParMetis-3.1
++ WM_OPTIONS=linux64GccDPOptFJMPI
++ wmake libso METISLib
linux64GccDPOptFJMPI/options:5: /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI: No such file or directory
make: *** No rule to make target `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI'. Stop.
wmake error: file 'Make/linux64GccDPOptFJMPI/objectFiles' could not be created
++ wmake libso ParMGridGen-1.0/MGridGen/IMlib
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libIMlib.so' is up to date.
++ wmake libso ParMGridGen-1.0/MGridGen/Lib
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libMGridGen.so' is up to date.
+ src/Allwmake
+ wmakePrintBuild -check
no git description found
+ /bin/rm -f OpenFOAM/Make/linux64GccDPOpt/global.C OpenFOAM/Make/linux64GccDPOpt/global.o
+ wmakeLnInclude OpenFOAM
+ wmakeLnInclude OSspecific/POSIX
+ Pstream/Allwmake
+ wmake libso dummy
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/dummy/libPstream.so' is up to date.
+ case "$WM_MPLIB" in
+ set +x
Note: ignore spurious warnings about missing mpicxx.h headers
+ WM_OPTIONS=linux64GccDPOptFJMPI
+ wmake libso mpi
linux64GccDPOptFJMPI/options:5: /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI: No such file or directory
make: *** No rule to make target `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI'. Stop.
wmake error: file 'Make/linux64GccDPOptFJMPI/objectFiles' could not be created

There is a mplibFJMPI rule here:
but apparently nothing equivalent for linux64.

If I look at settings.{csh,sh} it appears that the current FJMPI settings are specific for 'sparcv9'. You'll need appropriate values there, and also an appropriate wmake/rules/General/mplibFJMP

Hi, Mark, thanks a lot for the reply.

I look through the file of wmake/rules/SunOS64Gcc/mplibFJMPI. I am afraid this time it is temporary out of my knowledge in the linux to revise a rule.
The purpose I want to use the "/opt/FJSVplang/bin/mpiexec" embeded in the system is to do a parallel calculation with more than one node.

I can use the ThirdParty/openmpi-1.3.3 well only within one node with less than 8 CPU cores. So I am trying to recompile the OF-1.6. But unfortunately...

Any more suggestions.
with best regards,
Chiven

Hi, Mark, again, you are right, the current FJMPI settings of OpenFOAM-1.6 is for the previous version of Fujitsu MPI. Now we are using a new system with new FJMPI version. The differences between the two versions are shown as follows.

･environment variable PATH(Previous version)
/opt/FJSVmpi2/bin、
/opt/FSUNaprun/bin、
INSTALL/FSUNf90/bin
･environment variable(Previous version)
LD_LIBRARY_PATH_64
/opt/FJSVmpi2/lib/sparcv9、
/opt/FSUNaprun/lib/sparcv9、
INSTALL/FSUNf90/lib/sparcv9

･environment variable PATH(This version)
/opt/FJSVmpi2/bin、
/opt/FJSVpnple/bin、
INSTALL/FSUNf90/bin
･environment variable(This version)
LD_LIBRARY_PATH_64
/opt/FJSVmpi2/lib/sparcv9、
/opt/FJSVpnple/lib/sparcv9、
INSTALL/FSUNf90/lib/sparcv9

Using openmpi is really the path of least resistance. But I don't see why you need to recompile things. AFAIK there should already be openmpi libraries are part of the binary pack. You should just need to adjust your settings to select OPENMPI as the mpi library and re-source your OpenFOAM environment.

If you want to use a newer openmpi (eg, openmpi-1.4.1), you'll obviously need to recompile it and the src/Pstream as well, but shouldn't need to recompile everything.

In general, however, it probably best to be using the 1.6.x version with the current bug fixes and other enhancements.

The computer has many nodes, every node has 8 cores (threads). When the processor number is not larger than 8, one node is enough. In this case, the parallel calculation of openmpi is good.
However, if the more than 8 processors are need, 2 nodes or more should be used. This time, the parallel calculation of openmpi is not good (very slow).
I am sorry for not clearly express. If you are interesting in it about the details, I can send the user manual to you.

This time, I am wondering how to set the openmpi for the computer. And the administrator also only know the Fujitsu Job scheduler.

Hi, I revised the settings.csh as follows.
Copy the wmake/rules/SunOS64Gcc/mplibFJMPI to wmake/rules/linux64Gcc/, and revised it as:
Then in the directory of OpenFOAM-1.6/, run
./Allwmake

But I met the following errors.
Any comments? Thanks.

Is there even an 'mpi.h' somewhere under /opt/FJSVmpi2 ?
For example,
should return something, and this path needs to be found by the include.

mpi.h is in the directory of

Hi, good morning, Mark, thank you for the comments. With your helps, I just make a big progress.
The settings.csh is revised as follows.
The mplibFJMPI is revised like,
The compile is done well, and no errors. I further check it through the command of foamInstallationTest, and it seems OK.
But when I try to run the cavity tutorial using icoFoam solver, I meet a problem:
I think it is because some parameters are not set. You know, the original rules for FJMPI is like:
And I don't know what are the means of -lfjgmp64, -lfjidt, -ljrm, and so on. Thus I have no idea to improve it. Any comments?

I have known the simple means of the parameters, such as -lfjgmp64, -lfjidt, -ljrm, etc. They are the abbreviated forms of the files under the directory. And I use FLMPI rule as,
But I still meet the same problem.