|
[Sponsors] |
[OpenFOAM.org] [GUIDE] How to compile OpenFOAM source on a cluster/server with no admin rights |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
September 6, 2021, 14:14 |
[GUIDE] How to compile OpenFOAM source on a cluster/server with no admin rights
|
#1 |
Member
Gabriel Felix
Join Date: May 2021
Location: Brazil
Posts: 35
Rep Power: 6 |
Hi folks,
I've recently gained access to my university cluster, HPC or server (call it as you prefer) and I wanted to run my OpenFOAM simulations on it. In these clusters you usually do not have any administrator rights so you can't install precompiled builds of OF as you do in your personal computer. Therefore, you will need to download and compile the OpenFOAM source code on the server. I took almost two weeks to succesfully compile OF on the cluster. I have read almost every thread about the subject here on CFD-Online comunity and just a few of them were actually useful to apply in my case. I'll try to compile here in this thread all the information I could gather among these threads and I hope that it helps you if you find yourself in the same situation I was. What I did initially was to follow the OpenFoam.org official instructions constained here. However, my compilation crashed/broke on a mpi.h error. When I set the ./Allwmake -k flag (according to this thread [OpenFOAM.com] Compiling v2006, mpicc -- broken), the compilation skipped the mpi.h, and succesfully compiled the other packages. I could actually run a test case such as pitzDaily with simpleFoam, but I could not run it in parallel as it needs the library OpenMPI. I started then to research about the OpenMPI installation and it seems that it is the error that most people get. The clusters usually have different versions of OpenMPI or it can even not be intalled on the machine. In most of the threads people said to modify the WM_MPLIB flag on etc/bashrc from SYSTEMOPENMPI to OPENMPI and other ones. However you will always get the same error because the MPI might not be running on the server. You will need to discover which version is installed on the server and then you will have to load it. After loading the OpenMPI module, I stopped getting the mpi.h error and compilation advanced until it crashed again in a ld.gold error, which I managed to solve through steps 1 to 4 of this thread Compiling OpenFoam from git : problem with ld. Compilation finally finished. However, when I tried to decompose the case with decomposePar, I got errors again. After many tries, I decided to delete this compilation and start a fresh-new one because I have done many modifications in etc/bashrc and Allwmake and these modifications could have introduced some user-induced errors to the process. To delete the compilation you just got to delete the opt/OpenFoam folder and unpack the .tgz file again to the same directory. Until here I was always using the default "system" compiler and a user in this thread [OpenFOAM.org] OpenFOAM 2.3.0 on CentOS 6.5 suggested to use the "ThirdParty" compiler and modify the WM_MPLIB to OPENMPI. I loaded the OpenMPI module and made the modifications, however I still got errors. Then I only modified the compiler and not the WM_MPLIB and it worked! There finally were no errors during compilation and it succesfully finished compiling all packages. So, in order to succesfully compile OF on the server I followed the steps above. I hope that this set up works for you! 1. Download the source code in your client PC and transfer to the server Code:
https://dl.openfoam.com/source/v2012/OpenFOAM-v2012.tgz Code:
scp -P <port> ~/OpenFOAM-v2012-windows10.tgz <your-user>@XXX.XX.XX.XX:$HOME/ 2. Unpack Code:
tar -xvzf -C OpenFOAM-v2012.tgz $HOME/opt/ Code:
module spider openmpi Code:
$ module spider openmpi -------------------------------------------------------------------------------------------------------------------- OpenMPI: -------------------------------------------------------------------------------------------------------------------- Description: The Open MPI Project is an open source MPI-3 implementation. Versions: OpenMPI/3.1.1-GCC-7.3.0-2.30 OpenMPI/3.1.3-GCC-8.2.0-2.31.1 OpenMPI/3.1.4-GCC-8.3.0 OpenMPI/4.0.5-GCC-10.2.0 -------------------------------------------------------------------------------------------------------------------- For detailed information about a specific "OpenMPI" module (including how to load the modules) use the module's full name. For example: $ module spider OpenMPI/4.0.5-GCC-10.2.0 -------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------- openmpi/3.1.3: openmpi/3.1.3/2019 -------------------------------------------------------------------------------------------------------------------- This module can be loaded directly: module load openmpi/3.1.3/2019 Copy exactly the name of the version that appears in the message returned in step 3. Code:
module load OpenMPI/4.0.5-GCC-10.2.0 || export PATH=$PATH:/usr/lib64/openmpi/bin Code:
WM_COMPILER_TYPE=ThirdParty 6. Source etc/bashrc Code:
source etc/bashrc You can run it in parallel with the -j flag. Change the number of processors according to your machine. You don't need to use the maximum number of processors available. Use just a few. Code:
./Allwmake -j 6 Code:
foamInstallationTest Code:
cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily $HOME/ cd $HOME/pitzDaily blockMesh simpleFoam Create a decomposeParDict similar to the other tutorials Code:
/*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: v2012 | | \\ / A nd | Website: www.openfoam.com | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class dictionary; object decomposeParDict; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // numberOfSubdomains 5; method hierarchical; coeffs { n (1 5 1); } Code:
blockMesh decomposePar mpirun -np 5 simpleFoam -parallel or mpiexec -np 5 simpleFoam -parallel reconstructPar 9. Load OpenMPI and OF automatically You will need to modify the server bashrc to load the OF and the OpenMPI whenever you login in to it so you won't have to load it manually every time. Add these lines to the end of ~/.bashrc Code:
alias of2012='alias of2012='module load OpenMPI/4.0.5-GCC-10.2.0 || export PATH=$PATH:/usr/lib64/openmpi/bin; source /home70/gfelix/opt/OpenFOAM-v2012/etc/bashrc $FOAM_SETTINGS' Code:
of2012 Now the OF is ready to run your cases. You just gotta transfer your run files with scp as done in step 1. NOTE: You might want to set up a passwordless login in order to avoid having to type your user and password everytime you login to the server. Just follow this guide How to Setup Passwordless SSH Login NOTE 2: You can check the information about the cluster's nodes and number of processors with the following commands: Code:
Number of processors (CPU) grep -c ^processor /proc/cpuinfo Infos about the nodes cat /proc/cpuinfo Last edited by gabrielfelix; September 8, 2021 at 08:33. |
|
July 5, 2022, 05:18 |
|
#2 | |
Member
Join Date: Nov 2020
Posts: 53
Rep Power: 5 |
Quote:
I encounter the same problem as you are. I followed almost the same route as what you did, however, if I'll run parallel processing it seems like all the cores in our cluster are hosting. What I am saying is, for example, if I'll ask 1 node which has 20 cores, then all these cores are hosting when in fact 19 cores must be slaves while one is the master. Have you encountered these issues? Regards, Mike |
||
Tags |
cluster, compile, openfoam, server, source |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
polynomial BC | srv537 | OpenFOAM Pre-Processing | 4 | December 3, 2016 09:07 |
SparceImage v1.7.x Issue on MAC OS X | rcarmi | OpenFOAM Installation | 4 | August 14, 2014 06:42 |
Problem compiling a custom Lagrangian library | brbbhatti | OpenFOAM Programming & Development | 2 | July 7, 2014 11:32 |
[swak4Foam] Swak4FOAM 0.2.3 / OF2.2.x installation error | FerdiFuchs | OpenFOAM Community Contributions | 27 | April 16, 2014 15:14 |
centOS 5.6 : paraFoam not working | yossi | OpenFOAM Installation | 2 | October 9, 2013 01:41 |