|
[Sponsors] |
how to confirm that I have already use infiniband in OpenFOAM? |
![]() |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
![]() |
![]() |
#1 |
New Member
Detian Liu
Join Date: Mar 2013
Location: Beijing, P.R.China
Posts: 20
Rep Power: 13 ![]() |
Dear all,
Recently, I am wroking on a case with LES for about 4M mesh, I try to install the infiniband on Ubuntu 12.04 in our cluster, but when I run the case in 48 cores with infiniband connections, I found that it is seems to the be same as I run the case with Ethernet, so is that means I do not use the infiniband at all, or I do not really use ib protocol? In addition, I installed OpenFOAM use the Ubuntu Deb Pack Installation, the openmpi was also installed automatically, so is that means I should recompile the openmpi again to support infiniband, if so, how can I should do? Or weather there is some tools to monitor the flux of infiniband, so that I can ensure that I have already use infiniband, because the monitor of Ubuntu could only monitor TCP/IP. Thanks for your help~ |
|
![]() |
![]() |
![]() |
![]() |
#2 | ||
Member
Laurent Fitschy
Join Date: May 2011
Posts: 40
Rep Power: 14 ![]() |
Hi,
Quote:
Check if your openmpi installation has infiniband support with the following command : Code:
# ompi_info | grep openib MCA btl: openib (MCA v2.0, API v2.0, Component v1.6.3) Code:
configOpt="$configOpt --with-openib --with-psm Quote:
To force openib protocol : mpirun --mca btl openib,self -n 4 icoFoam -parallel To force ethernet protocol mpirun --mca btl tcp,self -n 4 icoFoam -parallel Best Regards. |
|||
![]() |
![]() |
![]() |
![]() |
#3 |
New Member
Detian Liu
Join Date: Mar 2013
Location: Beijing, P.R.China
Posts: 20
Rep Power: 13 ![]() |
Dear Laurent,
Thanks for your reply! As to the problem,when I check the openmpi with: # ompi_info | grep openib I get the result: MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.3) Is that means there is no problem with the openmpi? Then, the two command: mpirun --mca btl openib,self -n 4 icoFoam -parallel mpirun --mca btl tcp,self -n 4 icoFoam -parallel can both run the case, but the efficiency seems to be the same, is that means for a 4M case in 48 cores, both openib and tcp seem to be similar? So do you have some experience that for how many cores, infiniband could emerge its advantage? Thanks again for your patient reply~ Best regards! |
|
![]() |
![]() |
![]() |
![]() |
#4 | ||
Member
Laurent Fitschy
Join Date: May 2011
Posts: 40
Rep Power: 14 ![]() |
Quote:
Quote:
Code:
lsmod | grep ipoib |
|||
![]() |
![]() |
![]() |
![]() |
#5 | |
New Member
lzf
Join Date: Mar 2020
Posts: 1
Rep Power: 0 ![]() |
Quote:
![]() |
||
![]() |
![]() |
![]() |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
OpenFOAM - Validation of Results | Ahmed | OpenFOAM Running, Solving & CFD | 10 | May 13, 2018 18:28 |
Issues with OpenFoam | sanjibdsharma | OpenFOAM | 0 | August 14, 2009 08:41 |
64bitrhel5 OF installation instructions | mirko | OpenFOAM Installation | 2 | August 12, 2008 18:07 |
How to confirm that OpenFoam runs in 64 bit | jdk | OpenFOAM Running, Solving & CFD | 2 | February 5, 2008 04:55 |
The OpenFOAM extensions project | mbeaudoin | OpenFOAM | 16 | October 9, 2007 09:33 |