CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Installation

[OpenFOAM.org] OpenFOAM 3.01 on Centos 5.11 - MPI Issue

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By vmgbritt

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   August 29, 2016, 13:35
Default OpenFOAM 3.01 on Centos 5.11 - MPI Issue
  #1
New Member
 
britt@alum.mit.edu
Join Date: Feb 2011
Posts: 5
Rep Power: 15
vmgbritt is on a distinguished road
Have been struggling with this for some time. If anyone has any thoughts would be appreciated.

I've been running 3.0.1 on Centos 5.11 successfully on a number of nodes here. Today I encountered one issue with a node that has Intel composer xe installed on it and is defaulting to intels MPI.

All my other nodes run on OpenMPI with success and I would like to execute of301 on this node using the same config.

mpi-selector --list returns:
lam-i386
lam-x86_64
openmpi-1.4-gcc-x86_64

mpi-selector --query returns:
default: openmpi-1.4-gcc-x86_64
level:system

but

mpirun -which returns
/opt/intel/composer_xe_2013_sp1.1.106/mpirt/bin/intel64/mpirun
and throws an error saying /opt/intel/composer_xe_2013_sp1.1.106/mpirt/bin/intel64/mpivars.sh: No such file or directory.

Foam is configured OpenFOAM-3.0.1/etc/bashrc configures WM_MPLIB to SYSTEMOPENMPI. I have tried setting it to OPENMPI but with no luck.

but after initializing with of301 typing $WM_MPLIB at the terminal returns bash: SYSTEMOPENMPI: command not found.

I note in the path statement that the intel path statements appear before the/usr/lib64/openmpi/1.4-gcc/bin directory.


I'd rather not uninstall the intel compiler but am unsure how to force the system to see openMPI as the default.

Any suggestions appreciated.

Britt
vmgbritt is offline   Reply With Quote

Old   August 29, 2016, 16:51
Default
  #2
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Quick answer: Looks like Intel's path is loaded automatically into the PATH variable automatically.

The trick I usually suggest on the wiki for OpenFOAM on Ubuntu, is to use the following symbolic linking commands:
Code:
ln -s /usr/bin/mpicc.openmpi OpenFOAM-3.0.1/bin/mpicc
ln -s /usr/bin/mpirun.openmpi OpenFOAM-3.0.1/bin/mpirun
Therefore, in the nodes where you don't have Intel-MPI installed, you can use these commands to find the actual location of each file:
Code:
which mpirun
which mpicc
and then adjust accordingly the symbolic linking commands.

edit: I deleted you're previous thread that had this same exact question.
__________________

Last edited by wyldckat; August 29, 2016 at 16:51. Reason: see "edit:"
wyldckat is offline   Reply With Quote

Old   August 30, 2016, 08:54
Default MPI Selector
  #3
New Member
 
britt@alum.mit.edu
Join Date: Feb 2011
Posts: 5
Rep Power: 15
vmgbritt is on a distinguished road
Thank you for the quick response. Apologies for asking again!

I followed your suggestion.

ln -s /usr/bin/mpirun.openmpi /usr/lib64/openmpi/1.4-gcc/bin/mpirun
ln -s /usr/bin/mpicc.openmpi /usr/lib64/openmpi/1.4-gcc/bin/mpicc

links established correctly but:
which mpirun still returns:
/opt/intel/composer_xe_2013_sp1.1.106/mpirt/bin/intel64/mpirun

but:

which mpicc is correct. /usr/lib64/openmpi/1.4-gcc/bin/mpicc

clearly still something happening with the path statements. I'll keep at it but let me know if anything else jumps out at you.

Thanks for the assistance.
vmgbritt is offline   Reply With Quote

Old   September 11, 2016, 11:38
Default
  #4
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Quick answer: I don't know if you've solved this issue or not, but another possible solution is to run the following:
Code:
echo PATH= /usr/lib64/openmpi/1.4-gcc/bin/:$PATH
If will enforce that the path to to the Open-MPI binaries is picked up first when you try to run them.

The other possible solutions that I can think of are regarding the environment variables. The following command should give us a few ideas:
Code:
export | grep MPI
There are a few variables that I can think of that may be giving you problems:
Code:
I_MPI_ROOT
MPI_ROOT
OPAL_PREFIX
the last one is commonly used for Open-MPI.
To unset them, run for example:
Code:
unset I_MPI_ROOT
__________________
wyldckat is offline   Reply With Quote

Old   September 15, 2016, 14:42
Default Follow-up MPI
  #5
New Member
 
britt@alum.mit.edu
Join Date: Feb 2011
Posts: 5
Rep Power: 15
vmgbritt is on a distinguished road
@WYLDCKAT thanks for the follow up:/

After a bit of trial and error and some support from Intel I got around this by adding the following lines at the end of my .bashrc

export PATH=/usr/lib64/openmpi/1.4-gcc/bin:$PATH
export LD_LIBRARY_PATH=/usr/lib64/openmpi/1.4-gcc/lib:$LD_LIBRARY_PATH

Hope this helps anyone who encounters similar issues.
wyldckat likes this.
vmgbritt is offline   Reply With Quote

Reply

Tags
mpi libraries

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Suggestion for a new sub-forum at OpenFOAM's Forum wyldckat Site Help, Feedback & Discussions 20 October 28, 2014 09:04
OpenFOAM 1.6.x - CentOS 5.3 x86_64 linnemann OpenFOAM Installation 68 April 22, 2013 11:03
Re : Problem Installing OpenFOAM on Centos -5.3 mohanphy OpenFOAM Installation 1 February 7, 2010 19:09
OpenFOAM 1.6 package - CentOS 5.3 x86_64 linnemann OpenFOAM Installation 36 September 2, 2009 12:46
Is Testsuite on the way or not lakeat OpenFOAM Installation 6 April 28, 2008 11:12


All times are GMT -4. The time now is 10:43.