CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

OpenFoam with PBS job manager

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   March 23, 2016, 15:35
Default OpenFoam with PBS job manager
  #1
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Is there any official tutorial or example regarding OpenFoam and PBS job scheduler (the qsub command)? I saw some pages on the web but it seems that they are trying to show how to solve a problem first.

I actually don't care about that since my job is to install (have already done) and check it users can submit their jobs to the cluster.

So, I need an example which shows how to submit a solved problem (meshing, ...) to the job scheduler.

Basically, I write a pbs script and put "ls" command in it and submit it to the scheduler to see if all nodes are working and communicating. So, I am searching for a similar simple command with respect to OpenFoam.

Any idea for that?
mahmoodn is offline   Reply With Quote

Old   March 23, 2016, 22:40
Default
  #2
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Hi mahmoodn,

You can use blockMesh and snappyHexMesh provided by OpenFOAM for meshing.

For testing PBS job scheduler, you can simply pick up a tutorial case from $FOAM_TUTORIALS directory and test with that case.

The $FOAM_TUTORIALS/incompressible/icoFoam/cavity is a good start. You need to copy it to your working directory, and run blockMesh inside the case directory. After meshing is done, run icoFoam to begin your simulation.

P.S.: don't forget to source OpenFOAM bashrc first
wwzhao is offline   Reply With Quote

Old   March 24, 2016, 02:24
Default
  #3
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Do you have any idea about this:

Code:
mahmood@cluster:of-test$ source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc
mahmood@cluster:of-test$ blockMesh
-bash: blockMesh: command not found
mahmood@cluster:of-test$ /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64Gcc48DPOpt/bin/blockMesh
/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64Gcc48DPOpt/bin/blockMesh: error while loading shared libraries: libblockMesh.so: cannot open shared object file: No such file or directory
mahmood@cluster:of-test$
I can test with LD_LIBRARY_PATH and PATH, but just want to know if I am on the correct track.
mahmoodn is offline   Reply With Quote

Old   March 24, 2016, 11:02
Default
  #4
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Hi mahmoodn,

Have you modified the bashrc from OpenFOAM? What's the value of your $FOAM_INST_DIR and $WM_PROJECT_INST_DIR?
wwzhao is offline   Reply With Quote

Old   March 24, 2016, 11:11
Default
  #5
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Yes, I modified it during the installation. Lines associated with that is

Code:
export WM_PROJECT=OpenFOAM
export WM_PROJECT_VERSION=2.3.0
foamInstall=/export/apps/mechanics/$WM_PROJECT
Therefore, I see

Code:
$ ls /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/
Allwmake      bin      doc  platforms    README.org  tutorials
applications  COPYING  etc  README.html  src         wmake
The values you asked are

Code:
$ echo $FOAM_INST_DIR
/export/apps/mechanics/OpenFOAM
$ echo $WM_PROJECT_INST_DIR
/export/apps/mechanics/OpenFOAM
mahmoodn is offline   Reply With Quote

Old   March 24, 2016, 11:43
Default
  #6
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Hmm. Seems the shell variables aren't exported correctly.

What's the output of export | grep -E '(FOAM_|WM_)'? What's the value of your $PATH and $LD_LIBRARY_PATH?
wwzhao is offline   Reply With Quote

Old   March 24, 2016, 13:11
Default
  #7
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Check it

Code:
$ export | grep -E '(FOAM_|WM_)'
declare -x FOAM_APP="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/applications"
declare -x FOAM_APPBIN="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin"
declare -x FOAM_EXT_LIBBIN="/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64GccDPOpt/lib"
declare -x FOAM_INST_DIR="/export/apps/mechanics/OpenFOAM"
declare -x FOAM_JOB_DIR="/export/apps/mechanics/OpenFOAM/jobControl"
declare -x FOAM_LIBBIN="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib"
declare -x FOAM_MPI="openmpi-1.6.5"
declare -x FOAM_RUN="/home/mahmood/OpenFOAM/mahmood-2.3.0/run"
declare -x FOAM_SETTINGS=""
declare -x FOAM_SIGFPE=""
declare -x FOAM_SITE_APPBIN="/export/apps/mechanics/OpenFOAM/site/2.3.0/platforms/linux64GccDPOpt/bin"
declare -x FOAM_SITE_LIBBIN="/export/apps/mechanics/OpenFOAM/site/2.3.0/platforms/linux64GccDPOpt/lib"
declare -x FOAM_SOLVERS="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/applications/solvers"
declare -x FOAM_SRC="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/src"
declare -x FOAM_TUTORIALS="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/tutorials"
declare -x FOAM_USER_APPBIN="/home/mahmood/OpenFOAM/mahmood-2.3.0/platforms/linux64GccDPOpt/bin"
declare -x FOAM_USER_LIBBIN="/home/mahmood/OpenFOAM/mahmood-2.3.0/platforms/linux64GccDPOpt/lib"
declare -x FOAM_UTILITIES="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/applications/utilities"
declare -x WM_ARCH="linux64"
declare -x WM_ARCH_OPTION="64"
declare -x WM_CC="gcc"
declare -x WM_CFLAGS="-m64 -fPIC"
declare -x WM_COMPILER="Gcc"
declare -x WM_COMPILER_LIB_ARCH="64"
declare -x WM_COMPILE_OPTION="Opt"
declare -x WM_CXX="g++"
declare -x WM_CXXFLAGS="-m64 -fPIC"
declare -x WM_DIR="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/wmake"
declare -x WM_LDFLAGS="-m64"
declare -x WM_LINK_LANGUAGE="c++"
declare -x WM_MPLIB="OPENMPI"
declare -x WM_OPTIONS="linux64GccDPOpt"
declare -x WM_OSTYPE="POSIX"
declare -x WM_PRECISION_OPTION="DP"
declare -x WM_PROJECT="OpenFOAM"
declare -x WM_PROJECT_DIR="/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0"
declare -x WM_PROJECT_INST_DIR="/export/apps/mechanics/OpenFOAM"
declare -x WM_PROJECT_USER_DIR="/home/mahmood/OpenFOAM/mahmood-2.3.0"
declare -x WM_PROJECT_VERSION="2.3.0"
declare -x WM_THIRD_PARTY_DIR="/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0"



$ echo $PATH
/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/gperftools-svn/bin:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/ParaView-4.1.0/bin:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/openmpi-1.6.5/bin:/home/mahmood/OpenFOAM/mahmood-2.3.0/platforms/linux64GccDPOpt/bin:/export/apps/mechanics/OpenFOAM/site/2.3.0/platforms/linux64GccDPOpt/bin:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/bin:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/wmake:/opt/openmpi/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/bio/ncbi/bin:/opt/bio/mpiblast/bin:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/tcoffee/bin:/opt/bio/hmmer/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gromacs/bin:/opt/bio/gmap/bin:/opt/bio/tigr/bin:/opt/bio/autodocksuite/bin:/opt/bio/wgs/bin:/opt/eclipse:/opt/ganglia/bin:/opt/ganglia/sbin:/usr/java/latest/bin:/opt/maven/bin:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/condor/bin:/opt/condor/sbin:/opt/gridengine/bin/linux-x64:/export/apps/mathematics/matlab2012a/bin:/home/mahmood/bin



$ echo $LD_LIBRARY_PATH
/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/gperftools-svn/lib:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/ParaView-4.1.0/lib/paraview-4.1:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/openmpi-1.6.5:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64GccDPOpt/lib/openmpi-1.6.5:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/openmpi-1.6.5/lib:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/openmpi-1.6.5/lib64:/home/mahmood/OpenFOAM/mahmood-2.3.0/platforms/linux64GccDPOpt/lib:/export/apps/mechanics/OpenFOAM/site/2.3.0/platforms/linux64GccDPOpt/lib:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib:/export/apps/mechanics/OpenFOAM/ThirdParty-2.3.0/platforms/linux64GccDPOpt/lib:/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/dummy:/opt/gridengine/lib/linux-x64:/opt/openmpi/lib
mahmoodn is offline   Reply With Quote

Old   March 24, 2016, 19:21
Default
  #8
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Seems your applications aren't installed to $FOAM_APPBIN.

Have you exported WM_COMPILER=Gcc48 before building OpenFOAM? This may be the issue. If so, just rebuild OpenFOAM without exporting that.
wwzhao is offline   Reply With Quote

Old   March 25, 2016, 02:37
Default
  #9
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
I ran it again. Please see the variables first

Code:
root@cluster:ThirdParty-2.3.0# module load openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin
root@cluster:ThirdParty-2.3.0# source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI
root@cluster:ThirdParty-2.3.0#

root@cluster:OpenFOAM-2.3.0# echo $WM_NCOMPPROCS
4
root@cluster:OpenFOAM-2.3.0# echo $WM_COMPILER
Gcc48
root@cluster:OpenFOAM-2.3.0# echo $WM_MPLIB
SYSTEMOPENMPI
root@cluster:OpenFOAM-2.3.0# echo $FOAM_APPBIN
/export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64Gcc48DPOpt/bin
Then I ran ./Allwmake again and you can see the lengthy output at pastebin


This time, when I run blockMesh, I see its usage

Code:
root@cluster:OpenFOAM-2.3.0# blockMesh
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.0-f5222ca19ce6
Exec   : blockMesh
Date   : Mar 25 2016
Time   : 11:03:08
Host   : "cluster.scu.ac.ir"
PID    : 10972
Case   : /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time



--> FOAM FATAL IO ERROR:
cannot find file

file: /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/system/controlDict at line 0.

    From function regIOobject::readStream()
    in file db/regIOobject/regIOobjectRead.C at line 73.

FOAM exiting
So it is working....
Question: What commands should I put in the .bashrc or /etc/profile (for all users)? These two?
Code:
root@cluster:ThirdParty-2.3.0# module load openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin
root@cluster:ThirdParty-2.3.0# source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 03:39
Default
  #10
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Hi mahmoodn,

Good to see it working

Quote:
Question: What commands should I put in the .bashrc or /etc/profile (for all users)? These two?
Well, if your users do not need to switch between various versions of OpenFOAM, you can just add this two lines to /etc/profile.d/openfoam.sh (create new one if it does not exist).

Just keep the user variables same with those when you built OpenFOAM .


Regards,
Weiwen
wwzhao is offline   Reply With Quote

Old   March 25, 2016, 03:56
Default
  #11
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
It seems that the following line has no effect in /etc/profile and ~/.bashrc

Code:
alias of230='module load openmpi-x86_64; source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI'
Instead, if I manually run
Code:
source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI
from the terminal, then I will be able to run blockMesh without full path
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 04:01
Default
  #12
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Quote:
Originally Posted by mahmoodn View Post
It seems that the following line has no effect in /etc/profile and ~/.bashrc
Code:
alias of230='module load openmpi-x86_64; source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI'
This line set alias for OpenFOAM, users who want to use OpenFOAM must execute of230 first.

The aliases are useful when there are multiple OpenFOAM versions as it wouldn't mess up the environment variables .


Regards,
Weiwen
wwzhao is offline   Reply With Quote

Old   March 25, 2016, 04:03
Default
  #13
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
OK I got it... So the correct thing is to first run of230 and then blockMesh.

Thanks for that. I will move on to the tutorial for PBS.
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 05:58
Default
  #14
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Hi again,
I ran blockMesh and then icoFoam in the $FOAM_TUTORIALS/incompressible/icoFoam/cavity. I saw some outputs without any error and it seems it is working.

Is there any multithreaded (needing more than core) example which I want to test with PBS?
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 07:00
Default
  #15
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Quote:
Originally Posted by mahmoodn View Post
Is there any multithreaded (needing more than core) example which I want to test with PBS?
Yes, for parallel computing, you need a more complex case, e.g, $FOAM_TUTORIALS/incompressible/pimpleDyMFoam/propeller

The following steps may be helpful.

1. Copy the propeller case to your working directory.

2. Run ./Allrun.pre to generate mesh and set initial conditions.

3. Run decomposePar to decompose the domain into 4 parts.

4. Prepare the PBS submit scripts. A simple example may look like this:

Code:
#!/bin/bash
#PBS -l nodes=1:ppn=4
#
#PBS -l walltime=00:30:00
#PBS -N myJobName
#PBS -A MyProjectName
#PBS -o outputFileName
#PBS -e errorFileName

module load openmpi-x86_64
source /export/apps/mechanics/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc WM_NCOMPPROCS=4 foamCompiler=ThirdParty WM_COMPILER=Gcc48 WM_MPLIB=SYSTEMOPENMPI

mpirun -np 4 pimpleDyMFoam -parallel | tee log
5. Use qsub to submit your job.


Regards,
Weiwen
wwzhao is offline   Reply With Quote

Old   March 25, 2016, 11:58
Default
  #16
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Some good news and some bad news...

Good:
I am able to run the mpi command on master node as well as the compute node. In other words, the following commands works as expected.

Code:
of230
mpirun -np 4 pimpleDyMFoam -parallel | tee log
check the output for the server node (where OpenFoam is installed) and the compute node. 4 cores are utilized.

However, when I submit the job, the output file contains the following error:
Code:
mahmood@cluster:propeller$ qsub submit.tor
49.cluster.scu
mahmood@cluster:propeller$ qstat
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
49.cluster                 myJobName        mahmood                0 R default
mahmood@cluster:propeller$ qstat
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
49.cluster                 myJobName        mahmood         00:00:00 C default
mahmood@cluster:propeller$ cat outputFileName
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.0-f5222ca19ce6
Exec   : pimpleDyMFoam -parallel
Date   : Mar 25 2016
Time   : 20:20:19
Host   : "compute-0-0.local"
PID    : 2102
[0]
[0]
[0] --> FOAM FATAL ERROR:
[0] Cannot read "/home/mahmood/system/decomposeParDict"
[0]
FOAM parallel run exiting
[0]
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun has exited due to process rank 0 with PID 2102 on
node compute-0-0.local exiting improperly. There are two reasons this could occur:

1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.

2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"

This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
Something is wrong with the MPI communicator.
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 13:30
Default
  #17
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
Regarding the error

Code:
Cannot read "/home/mahmood/system/decomposeParDict"
I have to say, the file actually is available but in a different place. As suggested by you, I copied the tutorial folder to my home. So:

Code:
mahmood@cluster:propeller$ pwd
/home/mahmood/of-test/propeller

mahmood@cluster:propeller$ ls
0         Allrun.pre     log.createPatch            log.topoSet     processor1  system
0.org     constant       log.renumberMesh           outputFileName  processor2
Allclean  log            log.snappyHexMesh          postProcessing  processor3
Allrun    log.blockMesh  log.surfaceFeatureExtract  processor0      submit.tor

mahmood@cluster:propeller$ ls system/
controlDict                        forces      readFields                         surfaces
createInletOutletSets.topoSetDict  fvSchemes   snappyHexMeshDict
createPatchDict                    fvSolution  surfaceFeatureExtractDict
decomposeParDict                   Q           surfaceFeatureExtractDictDefaults
So, system/decomposeParDict is present. I suspect that someone (PBS or OpenFoam or MPI) wrongly creates $HOME/system/decomposeParDict which is considered as /home/mahmood/system/decomposeParDict

Any idea for that?
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 13:36
Default
  #18
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
OK. I think I found the problem. According to Basics: Set-up for different desktop machines (post #7 and after that), I found that my home has not been exported to the NFS! (have to say that I created my account on the server only).

So, let me test something and I will be back.
mahmoodn is offline   Reply With Quote

Old   March 25, 2016, 14:16
Default
  #19
New Member
 
Weiwen Zhao
Join Date: Dec 2013
Posts: 14
Rep Power: 4
wwzhao is on a distinguished road
Hi mahmoodn,

You need to setup NIS server and client for sharing account info. Moreover, the home of compute nodes should be mounted via NFS.


Regards,
Weiwen
wwzhao is offline   Reply With Quote

Old   March 25, 2016, 15:53
Default
  #20
New Member
 
Join Date: Mar 2016
Posts: 17
Rep Power: 2
mahmoodn is on a distinguished road
OK I found the problem


In the PBS script, we have to add cd $PBS_O_WORKDIR abd this is an important thing.

Without that and by adding pwd before the mpi command in the PBS script, I saw that the working directory was /home/mahmood which is actually $HOME and for that reason it appended system/decomposeParDict to that.

With cd $PBS_O_WORKDIR, the working directory is correctly set to the location where propeller exists.

So, everything is OK now and I am able to run a OpenFoam job. Have to tell the guys for that.
Thanks.
mahmoodn is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
OpenFOAM v3.0+ ?? SBusch OpenFOAM 21 October 15, 2016 12:52
OpenFOAM Training, London, Chicago, Munich, Sep-Oct 2015 cfd.direct OpenFOAM Announcements from Other Sources 2 August 31, 2015 13:36
Fluent jobs through pbs ibnkureshi FLUENT 5 June 9, 2011 13:43
64bitrhel5 OF installation instructions mirko OpenFOAM Installation 2 August 12, 2008 18:07
cfd job Dr. Don I anyanwu Main CFD Forum 20 May 17, 1999 15:13


All times are GMT -4. The time now is 01:18.