CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM (http://www.cfd-online.com/Forums/openfoam/)
-   -   Running OpenFOAM in parallel (http://www.cfd-online.com/Forums/openfoam/126216-running-openfoam-parallel.html)

samiam1000 November 11, 2013 06:32

Running OpenFOAM in parallel
 
1 Attachment(s)
Dear Foamers,

I am dealing with a pretty-big mesh and I am having some troubles.

I am running it on CINECA, a super-computer center and my script is as follow:

Code:

#!/bin/sh

blockMesh
decomposePar
mpirun -np 16 snappyHexMesh
reconstructParMesh -latestTime -mergeTol 1E-06 -noZero

rm -r processor*
cp constant/polyMesh/blockMeshDict .
rm -r constant/polyMesh
cp -r 1/polyMesh constant
cp blockMeshDict constant/polyMesh
rm -r 1
rm blockMeshDict

decomposePar
mpirun -np 16 topoSet -dict ./system/topoSetDict
mpirun -np 16 refineMesh -dict ./system/refineMeshDict1
reconstructParMesh -latestTime -mergeTol 1E-06 -noZero

rm -r processor*
cp constant/polyMesh/blockMeshDict .
rm -r constant/polyMesh
cp -r 1/polyMesh constant
cp blockMeshDict constant/polyMesh
rm -r 1
rm blockMeshDict

decomposePar
mpirun -np 16 simpleFoam
recontructPar

Once I run it, I get the error in the file attached.

As far as the steps are concerned, I am pretty sure they're ok, since if I run them on my laptop (with a smaller mesh, of course) everything is fine.
What is this errore linked to, according to you?

Thanks a lot,
Samuele

colinB November 11, 2013 06:50

Dear Samuele,

the first striking thing I saw is that you miss some
parallel statements (see bold code):

Code:

mpirun -np 16 snappyHexMesh -parallel
and this statement you have to use for all mpiruns otherwise
you will start up 16 different processes which will cause some troubles.

Let me know whether it worked
regards
Colin

samiam1000 November 11, 2013 08:26

Thanks a lot, Colin.

Let me try.

I'll get back to you asap.

Samuele

samiam1000 November 11, 2013 08:36

Hi Colin,

I've added the option -parallel, but I am always getting the same error.

Any other hints?

Thanks,
Samuele

colinB November 11, 2013 09:01

Hi Samuele

well the last line says:

Code:

=>> PBS: job killed: mem 31250244kb exceeded limit 12582912kb
which means that the memory is exceeded.
I thought this came by starting 16 times the same thread and making 16
times the same mesh, but apparently not.

So my question is: How big is your mesh and how much RAM do you have
available?
As a guideline for me I'm using 800k cells per 1GB RAM

Another question I have: why is blockMesh started up several times
while you specified blockMesh to start up as a single thread and not
as a mpirun?
Is this because you are using CINECA?

regards
Colin


All times are GMT -4. The time now is 11:32.