Openfoam on Clusters
 

Hi,
I am running OF on this cluster using pbs script. Everything seems okay, however, OF "hang" after this:

/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 1.7.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 1.7.1-03e7e056c215
Exec : twoPhaseEulerFoam
Date : May 20 2011
Time : 09:38:49
Host : b64
PID : 20776
Case : /home/freeman/rectbed
nProcs : 1

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0


Reading g
Reading transportProperties

Calculating face flux field phia
Calculating face flux field phib
.....

It has been in this position for almost 17hrs. Any help on this will be much appreciated. Please note that this same very "demo" has been ran on my PC with lower specs and works fine. The computing time on my PC is just couple of hours.

Thanks
Freeman

Could you upload a copy of your script file. Also, are you trying to run a parallel version over several processors or the original you ran locally?

Marco,
Nope! Because, I just created account on this cluster and wanted to test it first, so I run "serial", see below:

#! /bin/bash
#PBS -S /bin/bash

cd $PBS_O_WORKDIR
echo "Current working directory is `pwd`"
module load openfoam
twoPhaseEulerFoam > log

thanks

Some observations:

1) Make sure that the default walltime and memory allocation are enough for your application. This is the cluster I am using, and it sounds like they have the same queue submission software:

http://www.westgrid.ca/support/running_jobs

I would suggest giving a very short walltime to test things out before running full jobs, like 0.5-1 hour.

2) Remove the "> log" statement. The output from the job is usually sent automatically to a logfile named file.pbs.oJOBNUM, where JOBNUM is the number given by the system. Error messages are sent to file.pbs.eJOBNUM.

Test these and see what happens. Good luck

Marco,
I am afraid, I got the same error. See blow my new pbs:

#! /bin/bash
#PBS -S /bin/bash
#PBS -l walltime=0:30:00
#PBS -l mem=2gb

cd $PBS_O_WORKDIR
echo "Current working directory is `pwd`"
module load openfoam
twoPhaseEulerFoam

The requested memory should be more than enough since my PC only use 700mb.

Thanks

Marco,
I think it's coming from twoPhaseEulerFoam Solver (see http://www.cfd-online.com/Forums/ope...eulerfoam.html), since it works well for other solvers.

Can you post the output from your pbs.e and pbs.o files? Thanks

pbs.e is below:

#0 Foam::error::printStack(Foam::Ostream&)=>> PBS: job killed: walltime 633 ex$

Note that, this was 30mins wall time and previously when using 72 wall time, it hang for 17 hours so I kill it.

As for pbs.o, please see http://www.cfd-online.com/Forums/ope...eulerfoam.html which is NO difference from my original posting, cc below:

/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 1.7.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 1.7.1-03e7e056c215
Exec : twoPhaseEulerFoam
Date : May 20 2011
Time : 09:38:49
Host : b64
PID : 20776
Case : /home/freeman/rectbed
nProcs : 1

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0


Reading g
Reading transportProperties

Calculating face flux field phia
Calculating face flux field phib
.....

Since you are also on Westgrid, I would recommend running an interactive session by typing and running twoPhaseEulerFoam to see what the stack trace is. Seems like the queue submission is eating the output.

Interactive showed this error:

freeman@bugaboo:~/bed2> twoPhaseEulerFoam
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 1.7.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 1.7.1-03e7e056c215
Exec : twoPhaseEulerFoam
Date : May 24 2011
Time : 11:16:22
Host : bugaboo
PID : 25269
Case : /home/freeman/bed2
nProcs : 1

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0


Reading g
Reading transportProperties

#0 Foam::error::printStack(Foam::Ostream&)

any clue...?

Wow, that it the shortest stack trace I've ever seen. The only thing I can think of is that something is wrong with the transportProperties dictionary. Is this case the same as the tutorial? Which cluster are you running openfoam on and how it is installed. I see you loading an openfoam module; does that source the installation?

yes, it's tutorial. Because, I didn't know what the problem might be..., therefore wanted a "clean" case. Interesting enough, I ran copy of it on my pc and it worked fine.

OF is installed on Bugaboo. I think westgrid has it only on this machine, according to their website (http://westgrid.ca/support/software) or ? How is it install, I don't know! But I assumed that it was install according to procedure listed on OF website. I have to say that I'ven't contacted westgrid support because I wanted to make sure that it'sn't coming from OF itself.

Usually, OF operate a local folder in one's home directory, therefore the "module" is used to addressed that.

Also, note that the same OF WORKED fine with other solvers such as SimpleFoam, interFoam, etc.

I will contact westgrid support then, probably it might be coming from either installation was wrong or their twoPhaseEuler solver doesn't have all the required files including transportProperties dictionary.

Thanks anyway in advance.

Yeah, it seems like there is a library missing or something, as its a run-time error. Since OpenFOAM isn't standard software on any of the Westgrid clusters, it can sometimes be helpful to install your own version. I'm running it on Glacier because I also need Octave on the same machines for post-processing, but I've also managed to get it running on Checkers and Snowpatch (only Snowpatch could get ParaFoam working, and that was with the binary version). Good luck!