Changing compiler and compile flags
 

In the bashrc file I can see an option to change the compiler between Gcc, Gcc43 and Icc (Intel compiler), but what if I have, for instance, a PGI compiler, or what if I want to use a system installed Gcc-4.4? How do I change this?

Also, I have proprietary MPI libraries available on some of the systems I use. Is there a way that I can specify these libraries instead of those included in the ThirdParty package?

And lastly, if I want to use a more recent release of Paraview, can I specify that so paraFoam is built to run it rather than what is included in the ThirdParty software (this is the least important point for me)?

If somebody can point me to the right places for these options I would appreciate it. If I manage to come across them myself I will post it here.

After some review I can see that I obviously need to change the 'compilerInstall=OpenFOAM' to 'compilerInstall=System' in the settings.sh file. How do I tell it which system compiler I want to use, then? Also, how do I change the compile flags?

The last thing I am still trying to figure out is how to change which mpi library is used. It seems like the settings.sh file is set up so that only mpi libraries within the ThirdParty folder can be used. I would instead prefer to use vendor provided mpi libraries available on the system.

In your settings.sh is a section named:

# Communications library
# ~~~~~~~~~~~~~~~~~~~~~~

and a environment variable called $WM_MPLIB. This variable has to be changed to switch MPI. Depending on the value of $WM_MPLIB there has to be added or changed the case belonging to this MPI, e.g. if you set $WM_MPLIB to MPI you have to set the path to your MPI:

MPI)
export MPI_ARCH_PATH=<Add path to your MPI here>
export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/mpi
;;

If this is done you should have a look into the $WM_PROJECT_DIR/wmake/rules directory. Depending on the system and the compiler you want to use there have to be a directory there containing the compiler flags and mpi flags. Have a look in linux64Gcc how this looks like.

Fabian

Thank you. This looks like exactly what I needed. I'll post again if I have trouble figuring out how to make the appropriate directory in .../wmake/rules.

Okay, I get that you can change the "compilerInstall" to OpenFOAM (uses ThirdParty compilers) or System (uses system compilers). I am getting a critical error regarding the path being looked in for my system compiler, though. The compiler is in usr/bin/gcc instead of /bin/gcc. On top of this I want to be able to use different versions of the gcc compiler. I can't find where the paths are set for compilerInstall=System. Can anyone point me to that?

This should be easy. First I think if you set compilerInstall to System he will use the system gcc, whose path is defined by the $PATH variable of your system. But have a look at settings.sh, line 103 there should be a possibility to enter a path for your own gcc, but you have to set compilerInstall back to OpenFoam or add a new case like "mygcc".

Fabian

I was looking at that and thinking I might add a case for compilerInstall in System. Then for the various compilers I want to use I would add the paths. I'll try this out and see how it works.

I added a section in settings.sh specifying the binary directories for various compilers. Things seemed to compile fine when I use the system Gcc 4.3 compiler, but when I try the 4.4 compiler I keep getting the following messages throughout the compile:

Needless to say the compile is unsuccessful. Any ideas on what I'm missing?

Changing a compiler, even if it is only a change in version is very tricky. If you compile with intels c++ compiler there are also problems coming up if you use 10.x instead of the 11.1 version. So first I would verify if you made all settings correctly. I would suggest to go to $WM_PROJECT_DIR/src/Pstream/mpi and type "wmake libso" (may be there is a Allwmake, I am not sure) if this works without any errors your MPI is set up correctly and your compiler too. Than you have to go into detail which file is causing the error and why it works with gcc 4.3 and not with the 4.4 version.

fabian

Going to $WM_PROJECT_DIR/src/Pstream/mpi and running "wmake libso" results in the same error message. I should have noted that the "unknown pseudo-op..." errors I was getting were also labeled as assembler messages. So it appears that I have indeed set something up incorrectly. Here is a rundown of what I've changed:

In bashrc:
-Changed $WM_COMPILER to a custom compiler (like Gcc44, Gcc45, etc)

In settings.sh:
-Changed $compilerInstall to System
-Added a section for case "$(compilerInstall)" in System
-Added cases for the different compilers with export WM_COMPILER_DIR=/opt/gcc/4.4 etc.
-under OPENMPI changed mpi_version to openmpi-1.3.3-gcc
-changed MPI_HOME to /opt/$mpi_version
-changed MPI_ARCH_PATH to $MPI_HOME

I also created the appropriate directory under $WM_PROJECT_DIR/wmake/rules.

I suspect that I'm not quite setting up my mpi libraries correctly here. I couldn't really draw all the parallels between what I saw in the ThirdParty directory and what I saw in the system mpi installation. I will try compiling again using the system compiler and the ThirdParty mpi options to see if this is indeed my problem. However, I still would like to utilize the system mpi libraries as I am trying to get some metrics on the performance of these libraries in conjunction with the various compilers available on these systems.

Changing compiler and mpi is very difficult and very time consuming. My experience was that switching mpi to a system own one is much more easier than changing compiler. If you use a standard x86-64 architecture I would use the openmpi and gcc coming along with Openfoam. There is no need to switch to another one. I am not sure which version of OpenMPI comes along with OpenFoam but this is not the oldest and the speed up of using gcc-4.4 instead of 4.3 is negletable compared to the amount of work for the change of the compiler. If you have a need of changing MPI, e.g. you use a Multi-Core System with a very special MPI but still an x86-64 architecture I would first change the MPI until this runs successfully with the OpenFoam own gcc-4.3. If there still is a need to change the compiler I would try this after a successfull change of the MPI. The compiler is in my opinion the much more difficult part to change.

Fabian

May be, you can also have a look at the Module project for easy change of compiler and compiler version in your PATH:

http://modules.sourceforge.net/

My whole purpose right now is to use obtain data on the performance of the various compilers available on these systems in building CFD code. The point is not to actually get OpenFOAM usable. I may be looking into other alternatives as using OpenFOAM for benchmarking like this may be a bit much.

Fabian, could you detail what changes are necessary for swapping out various MPI versions? I would like to see if I can successfully use the different MPI libraries on the system while using the Gcc 4.3 compiler (which does work). Like I said, I may scrap this idea all together if I find a better alternative, but it's slow at work right now and I have time to keep working on this.

For changing the MPI used by OpenFOAM there are in my opinion two places where you have to change something and which are allready mentioned above:

1) settings.sh: MPI_ARCH_PATH and FOAM_MPI_LIBBIN have to be sett correctly
2) There have to be a file for every MPI you want to use in $WM_PROJECT_DIR/wmake/rules/<system and compiler dependend directory> containig flags and options for the MPI you are willing to use. But which flags and options this is MPI dependend and this is beyond my knowlegde.

What I am not understanding you say:
But how would you perform any test if OpenFoam isn't usable?

Fabian