CFD Online URL
[Sponsors]
Home > Forums > OpenFOAM Native Meshers: snappyHexMesh and Others

multiple instances of SnappyHexMesh

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   April 3, 2012, 15:33
Default multiple instances of SnappyHexMesh
  #1
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road
I'm firing up four cases one after another and want to let them run on the same machine. Seems SHM only allows two instances to run at the same time and kills the other processes.
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 3, 2012, 15:44
Default
  #2
Senior Member
 
lore
Join Date: Mar 2010
Location: Italy
Posts: 454
Rep Power: 8
lovecraft22 is on a distinguished road
Send a message via Skype™ to lovecraft22
Use a script, have a look at any runScript within any of the OF tutorials.
__________________
I use PowerFLOW!
lovecraft22 is offline   Reply With Quote

Old   April 3, 2012, 21:27
Default
  #3
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road
you mean like this one...or is there something simpler?

#!/bin/sh
#------------------------------------------------------------------------------
# ========= |
# \\ / F ield | OpenFOAM: The Open Source CFD Toolbox
# \\ / O peration |
# \\ / A nd | Copyright (C) 2011 OpenFOAM Foundation
# \\/ M anipulation |
#------------------------------------------------------------------------------
# License
# This file is part of OpenFOAM.
#
# OpenFOAM is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with OpenFOAM. If not, see <http://www.gnu.org/licenses/>.
#
# Script
# Allrun
#
# Description
#
#------------------------------------------------------------------------------
cd ${0%/*} || exit 1 # run from this directory

# Source tutorial run functions
. $WM_PROJECT_DIR/bin/tools/RunFunctions

# logReport <logfile>
# Extracts useful info from log file.
logReport()
{
caseName=`dirname $1 | sed s/"\(.*\)\.\/"/""/g`
app=`echo $1 | sed s/"\(.*\)\."/""/g`
appAndCase="Application $app - case $caseName"

fatalError=`grep "FOAM FATAL" $1`
UxSS=`grep -E "Ux[:| ]*solution singularity" $1`
UySS=`grep -E "Uy[:| ]*solution singularity" $1`
UzSS=`grep -E "Uz[:| ]*solution singularity" $1`
completed=`grep -E "^[\t ]*[eE]nd" $1`

if [ "$fatalError" ]
then
echo "$appAndCase: ** FOAM FATAL ERROR **"
elif [ "$UxSS" -a "$UySS" -a "$UzSS" ]
then
echo "$appAndCase: ** Solution singularity **"
elif [ "$completed" ]
then
completionTime=`tail -10 $log | grep Execution | cut -d= -f2 | sed 's/^[ \t]*//'`
if [ "$completionTime" ]
then
completionTime="in $completionTime"
fi
echo "$appAndCase: completed $completionTime"
else
echo "$appAndCase: unconfirmed completion"
fi
}


# Recursively run all tutorials
foamRunTutorials cases


# Analyse all log files
rm testLoopReport > /dev/null 2>&1 &
touch testLoopReport

for appDir in *
do
(
[ -d $appDir ] && cd $appDir || exit

for log in `find . -name "log.*" | xargs ls -rt`
do
logReport $log >> ../testLoopReport
done
echo "" >> ../testLoopReport
)
done

find . -name "log.*" -exec cat {} \; >> logs

# ----------------------------------------------------------------- end-of-file
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 4, 2012, 06:06
Default
  #4
Senior Member
 
lore
Join Date: Mar 2010
Location: Italy
Posts: 454
Rep Power: 8
lovecraft22 is on a distinguished road
Send a message via Skype™ to lovecraft22
Nope, I would say something much easier like:


Code:
#!/bin/sh
cd path_to_first_case
snappyHexMesh

cd path_to_second_case
snappyHexMesh

cd path_to_third_case
snappyHexMesh
__________________
I use PowerFLOW!
lovecraft22 is offline   Reply With Quote

Old   April 4, 2012, 10:21
Default
  #5
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road
that is kind of what I am doing. only two instances can run. if more are open it starts killing the previous processes.
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 4, 2012, 10:27
Default
  #6
Senior Member
 
lore
Join Date: Mar 2010
Location: Italy
Posts: 454
Rep Power: 8
lovecraft22 is on a distinguished road
Send a message via Skype™ to lovecraft22
If you use a script like the one I posted every instance will start after the previous one has ended.
__________________
I use PowerFLOW!
lovecraft22 is offline   Reply With Quote

Old   April 4, 2012, 22:41
Default
  #7
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road
i see...
do you k now if the solvers,rather than the meshers, for multiple cases can run simultaneously?
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 5, 2012, 05:20
Default
  #8
Senior Member
 
lore
Join Date: Mar 2010
Location: Italy
Posts: 454
Rep Power: 8
lovecraft22 is on a distinguished road
Send a message via Skype™ to lovecraft22
Don't know, sorry…
__________________
I use PowerFLOW!
lovecraft22 is offline   Reply With Quote

Old   April 5, 2012, 09:21
Default
  #9
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 7,511
Blog Entries: 33
Rep Power: 74
wyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the rough
Greetings to both of you!

@Mihai: if the 4 cases have independent folders and if you have enough RAM available, then the snappyHexMesh processes should not affect each other during serial execution!

Knowing the machine specifications you are using and the mesh resolutions you are using as well, it's possible to infer what the actual problem is.

For example: a simple cube-like hexahedral mesh generated with blockMesh to have 1 million cells, will need about 1GB of RAM. When running snappyHexMesh on top of this base mesh, with level 1 of additional resolution, you'll need at a maximum of (2^1)^3 times the mesh resolution, namely 8 million cells -> 8GB RAM. If using level 2 -> 4^3 -> 64 million cells!
Of course these values are only the maximum resolution, if and only if we used a resolution increase all over the mesh. When using localized resolution increments, the cell count will not increase this much.

If you are using mesh resolutions similar in all 4 cases, then after one of them is complete, run checkMesh. It will tell you the cell count and any issues that the mesh might have.

Best regards,
Bruno
wyldckat is online now   Reply With Quote

Old   April 5, 2012, 11:07
Default
  #10
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road
Quote:
Originally Posted by wyldckat View Post
Greetings to both of you!

@Mihai: if the 4 cases have independent folders and if you have enough RAM available, then the snappyHexMesh processes should not affect each other during serial execution!


Best regards,
Bruno
by serial you mean one starting after the previous ended?
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 5, 2012, 12:49
Default
  #11
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 7,511
Blog Entries: 33
Rep Power: 74
wyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the rough
Hi Mihai,

Let me quote from wikipedia
Quote:
Originally Posted by http://en.wikipedia.org/wiki/Parallel_computing#Background
Traditionally, computer software has been written for serial computation. To solve a problem, an algorithm is constructed and implemented as a serial stream of instructions. These instructions are executed on a central processing unit on one computer. Only one instruction may execute at a time—after that instruction is finished, the next is executed.[6]

Parallel computing, on the other hand, uses multiple processing elements simultaneously to solve a problem. This is accomplished by breaking the problem into independent parts so that each processing element can execute its part of the algorithm simultaneously with the others. The processing elements can be diverse and include resources such as a single computer with multiple processors, several networked computers, specialized hardware, or any combination of the above.[6]
Basically, taking lovecraft22's example, the serial execution of simultaneous snappyHexMesh processes would be this:
Code:
#!/bin/sh
cd path_to_first_case
snappyHexMesh &

cd path_to_second_case
snappyHexMesh &

cd path_to_third_case
snappyHexMesh &
The added ampersand (&) will cause the process to be launched (somewhat) independently of the shell.

For launching snappyHexMesh in a parallel meshing job, you can see the following tutorials on 2.1.x:
  • "incompressible/pisoFoam/les/motorBike" - this is a rather complex case, in the sense that it solves the case by using 2 cases! Similar to the "cavity graded" example from the user guide.
  • "incompressible/simpleFoam/turbineSiting" - this one shows how to generate with 2 cores and then redistributes the mesh to 4 cores for solving the case.
There is also another case familiar to me, which is a somewhat modified "motorBike" case, designed for doing some benchmarking: http://code.google.com/p/bluecfd-sin...untimes202_211


Now, as for the last detail: Can two or more simultaneous parallel runs interfere with one another?
The answer should be No, but I've seen some rare cases where the communications seemed to have gotten crossed over each other... but this should lead to simultaneous destruction, simply because both cases would get corrupted. This shouldn't happen, but either a bug in the MPI toolbox or some other issue could cause this.
But technically - aside from a possible bug and network issues - the only reason for two distinct parallel jobs to interfere with each other would be insufficient RAM for both to run... or a communication timeout, in a situation where one case is getting all of the CPU time, making the other one too late to answer a call...

Best regards,
Bruno
wyldckat is online now   Reply With Quote

Old   April 6, 2012, 09:57
Default
  #12
Senior Member
 
Mihai Pruna
Join Date: Apr 2010
Location: Boston
Posts: 179
Blog Entries: 1
Rep Power: 6
mihaipruna is on a distinguished road

thanks for the explanation. I'll have to figure something out.
Could also be a resource issue when they crash.
Have you tried running SHM in quick succession from multiple terminal windows?
That's what I'm having issue with. It lets only two run, if more appear it kills off the older ones, but not in the order in which they started.
Having simulations 1,2,3,4:
If I do that, only 2 and 3 run to the end.
__________________
Mihai Pruna's Bio
mihaipruna is offline   Reply With Quote

Old   April 6, 2012, 15:59
Default
  #13
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 7,511
Blog Entries: 33
Rep Power: 74
wyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the roughwyldckat is a jewel in the rough
Hi Mihai,

OK, here is a test case that needs about 1.5GB of RAM when running 4 separate cases and takes about 280s (on AMD 1055T 64bit x6 cores ) for each snappyHexMesh:
  1. Assuming we've already got our own copy of the "tutorials" folder at $FOAM_RUN:
    Code:
    cd $FOAM_RUN/mesh/snappyHexMesh
    cp -r flange flange1
  2. Edit the file "flange1/constant/polyMesh/blockMeshDict" and change the resolution to this line:
    Code:
    hex (0 1 2 3 4 5 6 7) (40 32 24) simpleGrading (1 1 1)
    This way the base mesh occupies about 40*32*24 = 30.7 kCells ~= 31 MB of RAM for each case.
  3. Let's clone the case 3 more times:
    Code:
    cp -r flange1 flange2
    cp -r flange1 flange3
    cp -r flange1 flange4
  4. Now lets launch them all simultaneously:
    Code:
    for a in 1 2 3 4; do cd flange$a; ./Allrun & cd ..; done
  5. Using top you can monitor the current status. Check the "log.snappyHexMesh" files on each "flange*" folder, in case one or more stop working.
Best regards,
Bruno
wyldckat is online now   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to set periodic boundary conditions Ganesh FLUENT 13 January 22, 2014 05:11
snappyHexMesh for porous - multiple regions without boundaries perjorgen OpenFOAM Native Meshers: snappyHexMesh and Others 0 April 2, 2012 05:01
OpenFOAM static build on Cray XT5 asaijo OpenFOAM Installation 9 April 6, 2011 13:21
Running multiple instances of solver using MPI and PyFoam bfa OpenFOAM Programming & Development 3 January 25, 2011 18:57
stitchMesh and snappyHexMesh gdbaldw OpenFOAM 0 December 23, 2009 03:09


All times are GMT -4. The time now is 09:37.