[OpenFOAM.com] Parallel mpirun doesn't work (perhaps because of clashes with intel compiler)

TomWaits · October 8, 2021, 08:49

Hi All,

I have compiled OpenFOAMv2006 on a Centos 8 machine and OpenFOAM errors on parallel runs. I think this is because I have an Intel fortran compiler installed which is clashing with openfoam as parallel simulations were working before. I have attempted a re-compile with the same error.

The error I get (on each processor) is (from any parallel run, for eg. the simpleFoam motorbike tutorial):

Code:

mca_base_component_repository_open: unable to open mca_btl_usnic: /opt/intel/compilers_and_libraries_2019.1.144/linux/mpi/intel64/libfabric/lib/libfabric.so.1: version `FABRIC_1.3' not found (required by /usr/lib64/openmpi/lib/openmpi/mca_btl_usnic.so) (ignored)

And then (again on each processor):

Code:

--> FOAM FATAL ERROR:
attempt to run parallel on 1 processor
 
       From static bool Foam::UPstream::init(int&, char**&, bool)
       in file UPstream.C at line 259.

FOAM aborting

and

Code:

 *** Process received signal *** 
Signal: Aborted (6) 
Signal code:  (-6)
 [ 0] /lib64/libpthread.so.0(+0x12b20)[0x7f0761e11b20]
[ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7f0761a7137f]
[ 2] /lib64/libc.so.6(abort+0x127)[0x7f0761a5bdb5]
[ 3] /home/user/openfoam2/OpenFOAM-v2006/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error11exitOrAbortEib+0x252)[0x7f0762ed2052]
[ 4] /home/user/openfoam2/OpenFOAM-v2006/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so(_ZN4Foam8UPstream4initERiRPPcb+0x3d9)[0x7f07618306b9]
[ 5]  at ??:?
 *** Process received signal *** 
Signal: Aborted (6) 
Signal code:  (-6)
 at ??:?

Any help or direction would be much appreciated.

Many thanks in advance,
Tom Waits.

TomWaits · October 8, 2021, 21:31

For anyone that has this (or similar) error - after a lot of bashing my head against the keyboard I have managed to get Openfoam re-compiled using the intel compiler/mpi (instead of the default OpenMPI) so that there are no clashes on my system. This has solved the problem.

olesen · October 9, 2021, 03:38

For what it's worth, you can normally just change the mpi bits independently. Eg, assuming you compiled initially with system gcc and OpenMPI. If you change your prefs to specify IntelMPI and run the compilation again, you will see that it only rebuilds the Pstream library and the ptscotch interface. Takes less than a few minutes. You will still need to be careful to set the proper env variables so that the intel-mpi knows to use gcc instead of icc, but that is fairly standard.
To make things a bit easier we have Allwmake-mpi scrips in the relevant places so you can just run those directly.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[ANSYS Meshing] Cant get parallel meshing to work in V18.2	jones007	ANSYS Meshing & Geometry	17	November 9, 2020 05:56
[OpenFOAM.com] Issue configuring ./makeParaView - Ubuntu 16.04	bjdarrer	OpenFOAM Installation	2	April 20, 2020 13:50
CFX11 + Fortran compiler ?	Mohan	CFX	20	March 30, 2011 18:56
Intel Fortran debugger with mpirun	agg	Main CFD Forum	2	March 21, 2007 17:43
AMD X2 & INTEL core 2 are compatible for parallel?	nikolas	FLUENT	0	October 5, 2006 06:49

October 8, 2021, 21:31		#2
TomWaits Member Tom Waits Join Date: Aug 2018 Posts: 38 Rep Power: 7	For anyone that has this (or similar) error - after a lot of bashing my head against the keyboard I have managed to get Openfoam re-compiled using the intel compiler/mpi (instead of the default OpenMPI) so that there are no clashes on my system. This has solved the problem.

October 9, 2021, 03:38		#3
olesen Senior Member Mark Olesen Join Date: Mar 2009 Location: https://olesenm.github.io/ Posts: 1,685 Rep Power: 40	For what it's worth, you can normally just change the mpi bits independently. Eg, assuming you compiled initially with system gcc and OpenMPI. If you change your prefs to specify IntelMPI and run the compilation again, you will see that it only rebuilds the Pstream library and the ptscotch interface. Takes less than a few minutes. You will still need to be careful to set the proper env variables so that the intel-mpi knows to use gcc instead of icc, but that is fairly standard. To make things a bit easier we have Allwmake-mpi scrips in the relevant places so you can just run those directly.