[cmoreno98]

Hello,


I was running interFoam simulations in parallel with no problem until yesterday, that I got the following problem:

cmoreno@clufa01:~/prueba$ mpirun -np 4 interFoam -parallel | tee interFoam.log
[clufa01:24393] opal_os_dirpath_create: Error: Unable to create the sub-directory (/tmp/openmpi-sessions-cmoreno@clufa01_0/51976) of (/tmp/openmpi-sessions-cmoreno@clufa01_0/51976/0/0), mkdir failed [1]
[clufa01:24393] [[51976,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 107
[clufa01:24393] [[51976,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 402
[clufa01:24393] [[51976,0],0] ORTE_ERROR_LOG: Error in file ess_hnp_module.c at line 638
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_session_dir failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------

Anyone knows how to fix it?


Thank you beforehand,


Carlos

[HPE]

Hi

- Could you please retry to run your case?
- If the error persists, can you attach the last few hundred lines of log file?

[cmoreno98]

Originally Posted by HPE

Hi

- Could you please retry to run your case?
- If the error persists, can you attach the last few hundred lines of log file?

Hello,

inside the .log file is the following:

--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  orte_session_dir failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
 --------------------------------------------------------------------------

The simulation doesn't start.

Also when I type "cd + start of the folder name" in the console and I press tab (to not have to write the full directory name) it doesn't work:

cmoreno@clufa01:~/prueba$ cd p-bash: no se puede crear un fichero temporal para el documento-aquí: No queda espacio en el dispositivo

The translation is "a temporary file for the document cannot be created-here: there is no space left on the device"
It is strange because if I try to run the simulation without using mpirun it works perfectly (just typing interFoam), so I think it's not a problem of space.

[tomf]

It looks like your root directory is full.

Please check if there are large tempory files in /tmp or other parts of your root directory that you can remove.

You will need root access to remove those of course.

Regards,
Tom

[cmoreno98]

Originally Posted by tomf

It looks like your root directory is full.

Please check if there are large tempory files in /tmp or other parts of your root directory that you can remove.

You will need root access to remove those of course.

Regards,
Tom

Hello,

I'm not used to work in Ubuntu. How do I check if the are large files in /tmp and remove them? I typed df -h and I got this:

cmoreno@clufa01:~$ df -h
S.ficheros     Tamaño Usados  Disp Uso% Montado en
udev             3,9G      0  3,9G   0% /dev
tmpfs            796M    81M  715M  11% /run
/dev/sda2         11G    11G     0 100% /
tmpfs            3,9G      0  3,9G   0% /dev/shm
tmpfs            5,0M      0  5,0M   0% /run/lock
tmpfs            3,9G      0  3,9G   0% /sys/fs/cgroup
/dev/sda1        511M   3,4M  508M   1% /boot/efi
/dev/sdb1         59G    43G   14G  76% /cluster
tmpfs            796M      0  796M   0% /run/user/1003

[tomf]

Hi,

Ok, thanks for this information. It indeed showed that the root partition (which is mounted as '/' in linux) is full.

I would suggest the following:

su 
cd /
du -h --max-depth=1

su will make you root user (you will have to type in your root password)
the second line may not be necessary, but it takes you to the main root partition.
The third line will show you the disk usage of each folder.
If the /tmp folder is large, you can probably remove some files/folder there.

Please do not touch other folders if you are not that familiar with linux.

If you are unsure, please search online for what is in what folder.

Hope this helps.
Best of luck,
Tom

[cmoreno98]

Originally Posted by tomf

Hi,

Ok, thanks for this information. It indeed showed that the root partition (which is mounted as '/' in linux) is full.

I would suggest the following:

su 
cd /
du -h --max-depth=1

su will make you root user (you will have to type in your root password)
the second line may not be necessary, but it takes you to the main root partition.
The third line will show you the disk usage of each folder.
If the /tmp folder is large, you can probably remove some files/folder there.

Please do not touch other folders if you are not that familiar with linux.

If you are unsure, please search online for what is in what folder.

Hope this helps.
Best of luck,
Tom

I'm not the admin of the server so I don't know the password...

cmoreno@clufa01:~$ su
Password: 
su: Authentication failure
cmoreno@clufa01:~$ cd /
cmoreno@clufa01:/$ du -h --max-depth=1
16M    ./bin
du: cannot read directory './sys/fs/fuse/connections/41': Permission denied
du: cannot read directory './sys/kernel/debug': Permission denied
0    ./sys
4.4G    ./usr
du: cannot read directory './cluster/lost+found': Permission denied
du: cannot read directory './cluster/users/professors': Permission denied
etc...

[tomf]

In that case you would need to contact the administrator. I guess he would be able to help you further.

Regards,
Tom