[derekm]

I see OpenSuse 13.1 has ben added to the Repository version but what are
the OFED and Openmpi versions supported/tesed etc ?

latest OFED is 3.12

latest Openmpi is 1.8 whereas 1.6 was mentioned in the 2.3.0 executable release

[wyldckat]

Greetings Derek,

I'm sorry to say that your questions seems a bit too... diagonal?

I say this because:

1. The RPM package officially provided of OpenFOAM 2.3.0 for SuSE is meant for openSUSE 12.3, not 13.1: http://www.openfoam.org/download/suse.php
   

   Version 2.3.0, SuSE 12.3 released 17/02/14

2. If you build OpenFOAM 2.3.0 from source code using the instructions given here: http://openfoamwiki.net/index.php/In...#openSUSE_13.1 - it will use the Open-MPI that is provided by default in openSUSE. This usually is the best solution, because the MPI installations that are provided from the Linux Distribution's own repositories, usually are the most stable ones for that Distribution and should work out-of-the-box upon installation. Or at least it should .
3. OFED and Open-MPI can only cooperate with each other if Open-MPI has been built with the respective bindings to OFED. I'm not familiar enough with openSUSE to know if OFED is even provided with it, and if Open-MPI provided with it has been built with the necessary connections to OFED. But my guess is that if OFED is provided in openSUSE, then Open-MPI from the same repositories should also cooperate well with each other.
4. Thinking a bit more about this, I believe you're referring to this page: http://www.openfoam.org/download/source.php - then if the previous point #3 is valid (i.e. Open-MPI + OFED is available in openSUSE), then it should work as intended. Although you might want to rebuild OpenFOAM, or at least rebuild "$FOAM_SRC/Pstream", so that it will be built with the latest Open-MPI you have on your system.
5. Beyond this, your mileage may vary. Using the latest MPI and/or OFED version does not mean it will work for your Infiniband network.

Best regards,
Bruno

[derekm]

Infiniband kernel drivers are provided with Opensuse but you still need to get the OFED libraries rpms

Opensuse 12.3 has Openmpi 1.6.0-3 compiled without Infiniband support

To get infiniband to work on Opensuse 12.3 ok, I used Openmpi 1.6.4 compiled with OFED 3.5 libraries. I tried compiling Openmpi 1.6.0-3 with various OFEDS but they didnt work, and it took a lot of work to find the working combination.

Opensuse 12.3 is about to go out of support so i need to find out about 13.1

Opensuse 13.1 has Openmpi 1.7.2 as default and I was trying to reduce "varying mileage" to find out what would work.

[wyldckat]

Mmm... then it's a tough choice. Open-MPI 1.6 and 1.8 are considered to be stable versions and 1.7 is considered a feature version (i.e., most likely to have more bugs): http://www.open-mpi.org/software/

From this point of view, it's best to build from source code either way. From safety point of view, it would be best to use the latest stable Open-MPI 1.6.5.

As for compatibility of OpenFOAM with Open-MPI 1.8, I have no idea.
Nonetheless, as I somewhat implied before, it is possible to build MPI support for OpenFOAM with more than one MPI version. Have a look at the file "etc/config/settings.sh" in OpenFOAM's installation and search for this line:

case "$WM_MPLIB" in

there you'll see that it's possible to add several versions of MPI, by using different names. Then you can use each version by setting "WM_MPLIB" for each MPI version, as exemplified here: http://openfoamwiki.net/index.php/In...#openSUSE_12.3 - step #6

source $HOME/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 WM_MPLIB=SYSTEMOPENMPI

Also check step #7 for more information.

[derekm]

Thanks...
Extra complications are kernel versions

Ofed 3.5 supports kernel 3.5 but I got it working with opensuse 12.3 which has kernel 3.7
but i can only update Mellanox firmware under kernel version 2.4 (openSuse 11.4)
opensuse 13.1 is kernel 3.11 but ofed 3.12 is kernel 3.12.


ahh well it only took me a month to get the last lot working

[wyldckat]

Hi Derek,

You might want to at least start thinking in switching to an enterprise-type Linux Distribution, instead of continuing using openSUSE. For example:

• CentOS, SL (Scientific Linux) or Rocks;
• Ubuntu (server edition) or Debian (stable version);
• or even go with commercial support, such as RHEL or SLES.

I say this because I too use to be a fan of openSUSE, due to it's simplified network configuration and so on, but as time became more scarse and more machines need to be maintained, it felt that it was best to switch to CentOS or similar, just so that we're not forced to reinstall all machines from scratch every 1-2 years, and still be able to easily continue to update for 4 or more years.

Best regards,
Bruno