[OnePeople]

Dear all,
I have strange issue with OpenFOAM and mpirun - they are not utilizing all cores if I run multiple jobs at once.

I.e. now I have 2 machines with 2xX5650 - which gives 24 cores in total per machine.

Now I'm trying to run 3 jobs, 8 cores each:
user1: mpirun -np 8 sonicFoam -parallel
user2: mpirun -np 8 simpleFoam -parallel
user2: mpirun -np 8 simpleFoam -parallel

Problem is, that jobs only uses 16 out of 24 cores and I don't have any idea why it happens. Thermal and RAM looks OK, so it doesn't look like throttling.

htop: https://pastebin.com/nV6tT5SD
lm-sensors: https://pastebin.com/9UnnEHSF

Same happens on the machine with: 2x8 core jobs, 2x4 core jobs and 2x2 core jobs. Also 16/24 cores are under load.

Have you any idea why it happens?

[OnePeople]

I found a solution.

Problem was caused by default binding/mapping routines in MPICH. If you use --bind-by socket instead of default bind.