CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (http://www.cfd-online.com/Forums/openfoam-solving/)
-   -   FOAM FATAL ERROR in Foam::findEtcFile() : could not find mandatory file (http://www.cfd-online.com/Forums/openfoam-solving/69561-foam-fatal-error-foam-findetcfile-could-not-find-mandatory-file.html)

makaveli_lcf October 27, 2009 10:43

FOAM FATAL ERROR in Foam::findEtcFile() : could not find mandatory file
 
Hi all!

I have following problem while running OpenFOAM 1.6 on cluster with queuing system. I compiled OpenFOAM from scratch using GCC from OF distribution and MVAPICH installed on cluster. OpenFOAM runs perfect on login machine in serial, but when I start it with queuing system it fails with such error in error output:

--> FOAM FATAL ERROR in Foam::findEtcFile() : could not find mandatory file
'controlDict'

As I understand, OpenFOAM could not get proper environment variable while running on remote machine. How to provide proper environment for OpenFOAM?

Thank you in advance

makaveli_lcf October 27, 2009 13:03

Problem solved!
Tommorow read about my expereance with OpenFOAM on MVAPICH+InfinityBand

mrv4real September 3, 2011 04:48

Hi,

even if this is an very old thread, do you remember the solution? Do i have to set some environment variables and if yes, which are they? setting the -case unfortunately did not work.

Thanks in advance,
mrv4real

wyldckat September 3, 2011 07:39

Greetings mrv4real and welcome to the forum!

There are at least two ways for doing this:
  1. There is a script named "foamExec" that you can use to launch OpenFOAM applications. Run in your current terminal/console window the following command:
    Code:

    which foamExec
    Now, use that script for calling your desired solver or application. As an example, for running icoFoam in parallel in my machine, instead of simply using:
    Code:

    icoFoam -parallel
    I would use:
    Code:

    /home/me/OpenFOAM/OpenFOAM-2.0.x/bin/foamExec icoFoam -parallel
    This should work on all OpenFOAM versions, but sadly it doesn't always work, due to some bugs that existed on some versions.
  2. Or you can use a job file for the PBS, like the one shown here: http://www.cfd-online.com/Forums/ope...tml#post286124 - post #5
Best regards,
Bruno

mrv4real September 3, 2011 08:19

Hi Bruno,

thanks a lot for the nice welcome message and your fast reply!

I tried your suggestions but am still having problems. I try to run it on a BlueGene/P and it works for serial code.

If I try it with mpirun in a loadleveler script, i had to pass the LD_LIBRARY_PATH environment variable and allthough that i specify the case directory with "-case <...>" he throws the error message about the missing controlDict.

Is there a way to give the directory of the simulation using an environment variable i can pass to the mpirun?

Thanks,
mrv4real

mrv4real September 3, 2011 08:33

... maybe it is best, if I post the script for the loadleveler. Hoping for some sugesstions...

#!/bin/sh
# @ account_no = []
# @ job_name = test_damBreak
# @ output = $(job_name).out
# @ error = $(job_name).err
# @ environment = COPY_ALL
# @ wall_clock_limit = 00:05:00
# @ notification = always
# @ notify_user = my@address.net

# @ job_type = bluegene
# @ bg_size = 4


# @ queue

source ~/OpenFOAM/OpenFOAM-1.6/etc/bashrc

cd /home/my_account/damBreak

mpirun -np 4 -mode SMP -cwd /home/my_account/damBreak -exe /home/my_account/OpenFOAM/OpenFOAM-1.6/applications/bin/linuxGccDPOpt/interFoam -args "-case /home/my_account/damBreak -parallel" -env LD_LIBRARY_PATH=/home/my_account/OpenFOAM/OpenFOAM-1.6/lib/linuxGccDPOpt/mpi:/home/my_account/OpenFOAM/my_account-1.6/lib/linuxGccDPOpt:/home/my_account/OpenFOAM/site/1.6/lib/linuxGccDPOpt:/home/my_account/OpenFOAM/OpenFOAM-1.6/lib/linuxGccDPOpt


Thanks a lot,
mrv4real

wyldckat September 3, 2011 09:43

Hi mrv4real,

I forgot to mention that the controlDict that the solver is complaining about is not the one in your case's system folder; it is complaining about not finding "OpenFOAM-1.6/etc/controlDict". This means that LD_LIBRARY_PATH is far from enough; there are a whole lot more variables that would need exporting.

Mmm... OpenFOAM 1.6 is getting old now... but OK, let's try this:
  1. foamExec in 1.6 allows you to read OF's bashrc file from a personal folder, namely "$HOME/.OpenFOAM/$version/bashrc". So, the following commands will create the personal folder used by OpenFOAM and copy the bashrc file to the newly created folder:
    Code:

    mkdir -p $HOME/.OpenFOAM/1.6
    cp $WM_PROJECT_DIR/etc/bashrc $HOME/.OpenFOAM/1.6/

  2. Edit the copied file if necessary, such as hard coding some of the folders to use your "/home/my_account", instead of $HOME.
  3. Now, as for the loadleveler script, this should do the trick:
    Code:

    #!/bin/sh
    # @ account_no = []
    # @ job_name = test_damBreak
    # @ output = $(job_name).out
    # @ error = $(job_name).err
    # @ environment = COPY_ALL
    # @ wall_clock_limit = 00:05:00
    # @ notification        = always
    # @ notify_user        = my@address.net

    # @ job_type = bluegene
    # @ bg_size = 4


    # @ queue

    source $HOME/.OpenFOAM/1.6/bashrc

    cd /home/my_account/damBreak

    mpirun -np 4 -mode SMP -cwd $PWD -exe `which foamExec`  -args "interFoam -parallel"

    It's simpler this way and more easily adaptable when you need to change something. Otherwise there is another way, but it ain't pretty :(
Good luck!
Bruno

mrv4real September 3, 2011 10:05

Hi,

i already tried it with a script, but then mpirun gives the error "Load failed on 172.16.100.69: Magic value in ELF header of executable file is invalid"

This comes from the fact, that foamExec is a script and not an executable. I found something about a "-m script" switch for mpirun, but the BlueGene doesn´t accept it...

Thanks,
mrv4real

wyldckat September 3, 2011 11:27

I did a bit of googling and found this: http://bluegene.epfl.ch/mpirun.txt
There seems to be an option named "-env_all". This would simplify the script, therefore not needing foamExec nor to manually define each variable to be sent.

If that doesn't work, then it's not going to be pretty :)

If I'm not mistaken, the necessary variables are: PATH LD_LIBRARY_PATH WM_PROJECT_DIR WM_PROJECT_INST_DIR WM_OPTIONS FOAM_LIBBIN FOAM_APPBIN FOAM_USER_APPBIN MPI_BUFFER_SIZE
The last one depends on the MPI you are using in BlueGene. In bold are the ones that are reeeeally necessary. I think the other ones are sometimes necessary...

Something like this should work:
Code:

#!/bin/sh
# @ account_no = []
# @ job_name = test_damBreak
# @ output = $(job_name).out
# @ error = $(job_name).err
# @ environment = COPY_ALL
# @ wall_clock_limit = 00:05:00
# @ notification        = always
# @ notify_user        = my@address.net

# @ job_type = bluegene
# @ bg_size = 4


# @ queue

source ~/OpenFOAM/OpenFOAM-1.6/etc/bashrc

cd /home/my_account/damBreak

mpirun -np 4 -mode SMP -cwd $PWD -exe `which interFoam`  -args "-parallel" -exp_env LD_LIBRARY_PATH -exp_env PATH -exp_env WM_PROJECT_DIR -exp_env WM_PROJECT_INST_DIR -exp_env  WM_OPTIONS -exp_env  FOAM_LIBBIN -exp_env  FOAM_APPBIN -exp_env  FOAM_USER_APPBIN -exp_env MPI_BUFFER_SIZE

If exporting the current variables to the remote nodes can't be done, then... well, the only ones that can have problems are PATH and LD_LIBRARY_PATH, so for the remaining variables, you can use the "-exp_env" option.

Oh, about PWD: this is usually an environment variable that has the path for the current directory, but it might not work on the environment you are working on :(

Best regards,
Bruno

mrv4real September 3, 2011 14:34

thanks a lot !!! now it works.

Best,
mrv4real

tH3f0rC3 December 2, 2011 10:29

Edit: sorry for posting.:D

nandiganavishal April 4, 2014 21:34

problem in running openfoam in parallel
 
Hello Foamers,

I was able to compile my test code (PNPFoam) and could run it in a serial mode in Kraken cluster. However, when I was trying to run a parallel case, I had an error similar to the ones discussed in the above message.

Code:

FOAM FATAL ERROR in Foam::findEtcFiles() : could not find mandatory file
    'controlDict--> FOAM FATAL ERROR in Foam::findEtcFiles() : could not find mandatory file

I tried to follow the above procedure, but none of them worked.

Below is my sample script file to run the parallel case.
Code:

#!/bin/bash
#PBS -A TG-CDA100010
#PBS -l size=72,walltime=00:02:00
#PBS -o a.out
#PBS -N  simout

cd $PBS_O_WORKDIR

aprun -n 72 -N 12 /lustre/scratch/joonho/openfoamv/conc_10mM/2D_simulations/PNP_solve/PNP_Foam

also below are the export environments that are defined in the bashrc file in my home folder

Code:

export LD_LIBRARY_PATH=/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/system-dynlib:/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/platforms/linux64GccDPOpt/lib/mpich2-system:/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/platforms/linux64GccDPOpt/lib/dummy:/opt/gcc/mpc/0.8.1/lib:/opt/gcc/mpfr/2.4.2/lib:/opt/gcc/gmp/4.3.2/lib:/opt/gcc/4.6.2/snos/lib64:/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/platforms/linux64GccDPOpt/lib:/opt/cray/atp/1.4.1/lib

export PATH=/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/wmake:/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/platforms/linux64GccDPOpt/bin:/lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/bin:/opt/cray/atp/1.4.1/bin:/opt/cray/xt-asyncpe/5.11/bin:/opt/cray/pmi/2.1.4-1.0000.8596.15.1.ss/bin:/opt/gcc/4.6.2/bin:/nics/e/sw/xt-cle3.1/xdusage:/sw/xt/globus/5.0.4/binary/bin:/sw/xt/globus/5.0.4/binary/sbin:/sw/altd/bin:/usr/local/hsi/bin:/usr/local/gold/bin:/sw/xt/bin:/opt/moab/6.1.6/bin:/opt/torque/2.5.11/bin:/opt/cray/lustre-cray_ss_s/1.8.4_2.6.27.48_0.12.1_1.0301.5943.18.1-1.0301.27524.1.24/sbin:/opt/cray/lustre-cray_ss_s/1.8.4_2.6.27.48_0.12.1_1.0301.5943.18.1-1.0301.27524.1.24/bin:/opt/cray/MySQL/5.0.64-1.0301.2899.20.1.ss/sbin:/opt/cray/MySQL/5.0.64-1.0301.2899.20.1.ss/bin:/opt/cray/sdb/1.0-1.0301.25929.4.88.ss/bin:/opt/cray/nodestat/2.2-1.0301.25918.4.1.ss/bin:/opt/modules/3.2.6.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/usr/lib64/jvm/jre/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/cray/bin

Can you please let me know how to resolve this error.

Thanks

Regards,
Vishal

wyldckat April 6, 2014 13:46

Greetings Vishal,

You should contact the administrator for the cluster you're using and/or read the instructions on their website. Because it seems that you're using a dedicated installation of OpenFOAM 2.1.1, which very likely will not work how we might expect, since we don't have access to the cluster you're using.

Because from your description, you seem to be confusing the path to your case "/lustre/scratch/joonho/openfoamv/conc_10mM/2D_simulations/PNP_solve/PNP_Foam" with the path to your application "PNPFoam".

My guess is that you should change your script to something like this:
Code:

#!/bin/bash
#PBS -A TG-CDA100010
#PBS -l size=72,walltime=00:02:00
#PBS -o a.out
#PBS -N  simout

cd $PBS_O_WORKDIR

. /lustre/scratch/proj/sw/openfoam/2.1.1/cnl3.1_gnu4.6.2/OpenFOAM-2.1.1/etc/bashrc

aprun -n 72 -N 12 /full/path/to/your/PNPFoam -case /lustre/scratch/joonho/openfoamv/conc_10mM/2D_simulations/PNP_solve/PNP_Foam -parallel

Make sure to change the path "/full/path/to/your/PNPFoam" to the correct path for your solver. You can see its path if you build it again, as it's in the last 1 or 2 lines of the build process.

Best regards,
Bruno

Mehrez April 21, 2015 07:32

Thanks it works !

Quote:

Originally Posted by wyldckat (Post 322759)
Greetings mrv4real and welcome to the forum!

There are at least two ways for doing this:
  1. There is a script named "foamExec" that you can use to launch OpenFOAM applications. Run in your current terminal/console window the following command:
    Code:

    which foamExec
    Now, use that script for calling your desired solver or application. As an example, for running icoFoam in parallel in my machine, instead of simply using:
    Code:

    icoFoam -parallel
    I would use:
    Code:

    /home/me/OpenFOAM/OpenFOAM-2.0.x/bin/foamExec icoFoam -parallel
    This should work on all OpenFOAM versions, but sadly it doesn't always work, due to some bugs that existed on some versions.
  2. Or you can use a job file for the PBS, like the one shown here: http://www.cfd-online.com/Forums/ope...tml#post286124 - post #5
Best regards,
Bruno



All times are GMT -4. The time now is 03:13.