Error while compiling OpenFOAM on Redhat cluster
 

Hi all,

I met a problem while compiling OF2.1.1 on Redhat. The error information is as follows, I have no idea of the " Protocol error" at all. ..

Can you guys help me? Thanks a lot.

Greetings feiqunali,

If my deduction is correct, what you're experiencing is a limitation on the Lustre installation your cluster is using.
Because the problem is triggered on a file that is using a symbolic link, which OpenFOAM uses for compiling all library files, namely for symbolically linking to almost all source code files that OpenFOAM has got.

In other words, in order to make it easier to compile OpenFOAM's libraries, it uses the "lnInclude" folder on each major library's source code folder, which contains links to all of the files relevant to that library. Then other libraries (and itself) can simply add to the search path a single directory path for each library.

Anyway, there are two possible solutions that I can think of:

1. Talk to your cluster administrator and show him the contents of this folder:
   
   Code:

   ---

   /lustre/home/hpc-mhwen/OpenFOAM/OpenFOAM-2.1.1/src/OpenFOAM/lnInclude

   ---

   Hopefully the administrator will figure out what needs to be tuned in the Lustre file system in order to handle such a large number of symbolic links in a single folder. And keep in mind that there are (or will be) at least 20 or 30 folders similar to that one.
2. The other possibility - which I strongly advise you to follow - is to build only on the master node of the cluster, on it's own local hard drive, not on the Lustre file system. When the build is complete, then you can copy the installation to the Lustre file system.

Best regards,
Bruno

Thanks.

Can you describe the first solution more detailly? What should I tell the administrator to do?

All you can do is describe the problem:

1. Show the administrator the contents of the folder I mentioned before, e.g. by running:
   
   Code:

   ---

   ls -l /lustre/home/hpc-mhwen/OpenFOAM/OpenFOAM-2.1.1/src/OpenFOAM/lnInclude

   ---

2. It should show you something like this:
   
   Code:

   ---

   lrwxrwxrwx  1 user user    26 Sep  9 20:16 addDictOption.H -> ../include/addDictOption.H
lrwxrwxrwx  1 user user    31 Sep  9 20:16 addOverwriteOption.H -> ../include/addOverwriteOption.H
lrwxrwxrwx  1 user user    28 Sep  9 20:16 addRegionOption.H -> ../include/addRegionOption.H
lrwxrwxrwx  1 user user    27 Sep  9 20:16 addTimeOptions.H -> ../include/addTimeOptions.H
lrwxrwxrwx  1 user user    74 Sep  9 20:16 addToGlobalFunctionSelectionTable.H -> ../db/runTimeSelection/globalFunctions/addToGlobalFunctionSelectionTable.H
lrwxrwxrwx  1 user user    74 Sep  9 20:16 addToMemberFunctionSelectionTable.H -> ../db/runTimeSelection/memberFunctions/addToMemberFunctionSelectionTable.H
lrwxrwxrwx  1 user user    64 Sep  9 20:16 addToRunTimeSelectionTable.H -> ../db/runTimeSelection/construction/addToRunTimeSelectionTable.H

   ---

   There are over 1400 symbolic links inside that folder.
3. Furthermore, you can check if the linked file "runTimeSelectionTables.H" is pointing to something:
   
   Code:

   ---

   ls -l /lustre/home/hpc-mhwen/OpenFOAM/OpenFOAM-2.1.1/src/OpenFOAM/lnInclude/runTimeSelectionTables.H
cat /lustre/home/hpc-mhwen/OpenFOAM/OpenFOAM-2.1.1/src/OpenFOAM/lnInclude/runTimeSelectionTables.H

   ---

   The last command might give you the same message of "Protocol error".
4. I searched online for:
   
   Code:

   ---

   lustre "Protocol error"

   ---

   And found this: https://cug.org/5-publications/proce...pitz-Paper.pdf
   
   Quote:

   Protocol error - invalid bits in messages, bad magic, wire version, NID wrong for mailbox, bad timeout. Remote peer will receive NAK.

Anything beyond this is for the administrator to figure out and fix.