CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Installation

[OpenFOAM.com] OF1806+ compiles, but fails in parallel with INTEL MPI

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By bestucan

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 17, 2020, 17:51
Default OF1806+ compiles, but fails in parallel with INTEL MPI
  #1
Member
 
Lisandro Maders
Join Date: Feb 2013
Posts: 98
Rep Power: 9
Lisandro Maders is on a distinguished road
Hello,

I am trying to compile OpenFOAM 1806+ with the following setup:
  • INTEL MPI - Intel(R) MPI Library for Linux* OS, Version 2019 Update 7 Build 20200312 (id: 5dc2dd3e9)
  • Gcc 8.4
  • scotch 6.0.3
  • Linux distro - Ubuntu 18.04 LTS

I will attach the output log from Allwmake - a few Paraview libraries didn't compile but I am fine with that.

After compilation apparently successfull, I try to run any tutorial in parallel and it fails. I attached a failed snappyHexMesh.log as example as well.

The step-by-step I have done to compile INTEL MPI was:
  1. Download INTEL MPI libraries and install it
  2. change WM_MPLIB=INTELMPI at .../OpenFOAM-v1806/etc/bashrc
  3. checked if the following paths were correct in etc/config.sh/mpi (they are):
    _foamAddPath $MPI_ARCH_PATH/intel64/bin
    _foamAddLib $MPI_ARCH_PATH/intel64/lib
  4. checked the files at wmake/rules/*/mplibINTELMPI to see if they were pointing to the correct paths (they are)
  5. source ~/intel/impi/2019.7.217/intel64/bin/mpivars.sh
  6. source .../OpenFOAM-v1806/etc/bashrc

After sourcing it, I checked the output of which mpirun --version, and it was pointing to the INTEL MPI (good).
I checked the output of which mpicc --version, and it was pointing to gcc-8.4.0 (good).

I run some tutorials in parallel - all of them crashed.

Some important info: the output of ldd $(which snappyHexMesh) shows to me that there are 2 mpi libraries set:
libmpi.so.20 => /usr/lib/x86_64-linux-gnu/libmpi.so.20 (0x00007f2ef5dfc000)
libmpi.so.12 => /home/lmaders/intel/compilers_and_libraries_2020.1.217/linux/mpi/intel64/lib/release/libmpi.so.12 (0x00007f2ef4c79000)


I think it should be only setting the intel one here. If this is probably the issue, how can I ensure the OF solvers are sourcing/setting only the INTEL MPI?

Hopefully I have been clear enough. Thanks in advance

Lisandro
Attached Files
File Type: gz Allwmake.log.tar.gz (4.3 KB, 1 views)
File Type: gz snappyHexMesh.log.tar.gz (1.2 KB, 1 views)
Lisandro Maders is offline   Reply With Quote

Old   June 18, 2020, 17:43
Default
  #2
HPE
Senior Member
 
Herpes Free Engineer
Join Date: Sep 2019
Location: The Home Under The Ground with the Lost Boys
Posts: 658
Rep Power: 8
HPE is on a distinguished road
Hi,

- I would either go for an older IntelMPI version down to OF1806, or a newer OpenFOAM version up to IMPI-2019. Is there any chance for that?
HPE is offline   Reply With Quote

Old   June 19, 2020, 01:34
Default
  #3
New Member
 
Tu Can
Join Date: Jul 2017
Location: China
Posts: 14
Rep Power: 5
bestucan is on a distinguished road
Quote:
Originally Posted by Lisandro Maders View Post
_foamAddPath $MPI_ARCH_PATH/intel64/bin
_foamAddLib $MPI_ARCH_PATH/intel64/lib
compare with

Quote:
Originally Posted by Lisandro Maders View Post
libmpi.so.12 => /home/lmaders/intel/compilers_and_libraries_2020.1.217/linux/mpi/intel64/lib/release/libmpi.so.12 (0x00007f2ef4c79000)

Just a guess. Is _foamAddPath add path recursively?


the libmpi.so.12 is under "lib/release" and you add "lib". Seems likes OpenFOAM missing some .so.
Lisandro Maders likes this.
bestucan is offline   Reply With Quote

Old   June 19, 2020, 16:59
Default
  #4
Member
 
Lisandro Maders
Join Date: Feb 2013
Posts: 98
Rep Power: 9
Lisandro Maders is on a distinguished road
That was it! I thought it would look for it recursivelly inside "lib" (shouldn't it be recursive?)

I explicitly set the release path and worked..

Thanks!
Lisandro Maders is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem during mpi in server: expected Scalar, found on line 0 the word 'nan' muth OpenFOAM Running, Solving & CFD 3 August 27, 2018 04:18
MPI parallel meet problems jiaojiao OpenFOAM 1 May 10, 2016 02:30
Parallel Problem of CFX 14.5.7 on a Intel Xeon 2690V2 computer with Win7 x64 OS BruceWen Hardware 0 April 29, 2014 23:48
PROBLEM IN PARALLEL PROGRAMMING WITH MPI Niavarani Main CFD Forum 1 April 20, 2004 06:51
MPI and parallel computation Wang Main CFD Forum 7 April 15, 2004 11:25


All times are GMT -4. The time now is 03:34.