CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

Running OpenFOAM in parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By colinB

Reply
 
LinkBack Thread Tools Display Modes
Old   November 11, 2013, 06:32
Default Running OpenFOAM in parallel
  #1
Senior Member
 
Samuele Z
Join Date: Oct 2009
Location: Mozzate - Co - Italy
Posts: 490
Rep Power: 9
samiam1000 is on a distinguished road
Dear Foamers,

I am dealing with a pretty-big mesh and I am having some troubles.

I am running it on CINECA, a super-computer center and my script is as follow:

Code:
#!/bin/sh

blockMesh
decomposePar
mpirun -np 16 snappyHexMesh
reconstructParMesh -latestTime -mergeTol 1E-06 -noZero

rm -r processor*
cp constant/polyMesh/blockMeshDict .
rm -r constant/polyMesh
cp -r 1/polyMesh constant
cp blockMeshDict constant/polyMesh
rm -r 1
rm blockMeshDict

decomposePar
mpirun -np 16 topoSet -dict ./system/topoSetDict
mpirun -np 16 refineMesh -dict ./system/refineMeshDict1
reconstructParMesh -latestTime -mergeTol 1E-06 -noZero

rm -r processor*
cp constant/polyMesh/blockMeshDict .
rm -r constant/polyMesh
cp -r 1/polyMesh constant
cp blockMeshDict constant/polyMesh
rm -r 1
rm blockMeshDict

decomposePar
mpirun -np 16 simpleFoam
recontructPar
Once I run it, I get the error in the file attached.

As far as the steps are concerned, I am pretty sure they're ok, since if I run them on my laptop (with a smaller mesh, of course) everything is fine.
What is this errore linked to, according to you?

Thanks a lot,
Samuele
Attached Files
File Type: txt launcher.txt (24.7 KB, 14 views)
samiam1000 is offline   Reply With Quote

Old   November 11, 2013, 06:50
Default
  #2
Senior Member
 
Join Date: Aug 2010
Location: Groningen, The Netherlands
Posts: 216
Rep Power: 10
colinB is on a distinguished road
Dear Samuele,

the first striking thing I saw is that you miss some
parallel statements (see bold code):

Code:
mpirun -np 16 snappyHexMesh -parallel
and this statement you have to use for all mpiruns otherwise
you will start up 16 different processes which will cause some troubles.

Let me know whether it worked
regards
Colin
samiam1000 likes this.
colinB is offline   Reply With Quote

Old   November 11, 2013, 08:26
Default
  #3
Senior Member
 
Samuele Z
Join Date: Oct 2009
Location: Mozzate - Co - Italy
Posts: 490
Rep Power: 9
samiam1000 is on a distinguished road
Thanks a lot, Colin.

Let me try.

I'll get back to you asap.

Samuele
samiam1000 is offline   Reply With Quote

Old   November 11, 2013, 08:36
Default
  #4
Senior Member
 
Samuele Z
Join Date: Oct 2009
Location: Mozzate - Co - Italy
Posts: 490
Rep Power: 9
samiam1000 is on a distinguished road
Hi Colin,

I've added the option -parallel, but I am always getting the same error.

Any other hints?

Thanks,
Samuele
samiam1000 is offline   Reply With Quote

Old   November 11, 2013, 09:01
Default
  #5
Senior Member
 
Join Date: Aug 2010
Location: Groningen, The Netherlands
Posts: 216
Rep Power: 10
colinB is on a distinguished road
Hi Samuele

well the last line says:

Code:
=>> PBS: job killed: mem 31250244kb exceeded limit 12582912kb
which means that the memory is exceeded.
I thought this came by starting 16 times the same thread and making 16
times the same mesh, but apparently not.

So my question is: How big is your mesh and how much RAM do you have
available?
As a guideline for me I'm using 800k cells per 1GB RAM

Another question I have: why is blockMesh started up several times
while you specified blockMesh to start up as a single thread and not
as a mpirun?
Is this because you are using CINECA?

regards
Colin
colinB is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
parallel error with cyclic BCs for pimpleDyMFoam and trouble in resuming running sunliming OpenFOAM Bugs 21 November 22, 2013 04:38
Running in parallel Djub OpenFOAM Running, Solving & CFD 3 January 24, 2013 17:01
Large test case for running OpenFoam in parallel fhy OpenFOAM Running, Solving & CFD 22 September 22, 2009 11:13
Statically Compiling OpenFOAM Issues herzfeldd OpenFOAM Installation 21 January 6, 2009 10:38
OpenFOAM 14 stock version parallel bug msrinath80 OpenFOAM Bugs 2 May 30, 2007 14:47


All times are GMT -4. The time now is 10:16.