OpenFOAM 2.4.0 OpenMPI Epoll warning on parallel job
 

Hi,

I ported my existing OpenFOAM cases from version 2.3.0 to 2.4.0.
The 2.4.0 version compiled without issues on a Fedora 23 Linux system.
When I start a parallel test case with :

>> mpirun -np 8 icoFoam -parallel > log &

it starts, but I get the following openmpi related warning:
"[WARN] Epoll ADD(4) on fd 1 failed ..."

The result is that the log file is not written while the case is running. When the job finishes, the log file appears. I can't find anything wrong with the results, but it is annoying that I can't 'tail' the log file during execution to check progress or errors.

Is this a known issue, can anyone suggest a solution ?

Tom.

Quick answer: From my blog post: Notes about running OpenFOAM in parallel

the output indeed only comes out at the end of the mpirun.

It is clearly the same issue as mentioned in the other post you mentioned (thanks for that). However I'm unable to find the 'mca-params.conf' file at the location given in this other post.

Checking mpirun --version gives me 1.8.8
I don't find any /usr/mpi/gcc/openmpi-1.8.8 directory.
A full search gives me 2 locations with a mca-params.conf file
- one in /etc/openmpi-x86_64
- one in /home/myuser/OpenFOAM/Thirdparty-2.4.0/..openmpi-1.8.5/...
unfortunately both are fully commented out, so I don't see what I could change to disable this epoll function.

Quick answer: When in doubt, ask Open-MPI:
Look inside the file "log.ompi_info" and look for two details:

1. "mca_param_files" - this points out which configuration files are being used.
2. "opal_event_include" - this points out the currently defined polling mechanism.

As far as I can figure out, the default that works as intended is the setting "poll". If the setting is hard coded into the Open-MPI build, then you will need to set the value in the configuration file as "poll". The other hypothesis is setting in the command line when launching, e.g.: