CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

Running in parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   January 21, 2013, 08:27
Default Running in parallel
  #1
Senior Member
 
Julien
Join Date: Jun 2012
Location: France
Posts: 153
Rep Power: 5
Djub is on a distinguished road
hi dear foamers,

I am dealing with running a large case in parallel.
Previously, I was doing different steps in this order :
Code:
blockmesh
snappyHexMesh
snappyHexMesh
decomposePar
mpirun -n 14 pimpleFoam -parallel
(I don't mention the displacement of dictionaries and boundary conditions)
And it worked quite well. Nevertheless, I think it was not 100% correct, because for example paraView is not able to read the decomposed case. I have to reconstruct it before to plot it.

Note: I am using twice sHM because I want to snap the mesh over one object, but not over another one. Thus, I use two STL and two different sHMDict.

By the way, I want to use renumberMesh, and to use sHM in parallel mode. Thus, my new steps are:
Code:
blockMesh
decomposePar
mpirun -n 14 snappyHexMesh -parallel -overwrite
mpirun -n 14 snappyHexMesh -parallel -overwrite
mpirun -n 14 renumberMesh -parallel -overwrite
mpirun -n 14 pimpleFoam -parallel
First question: I found these steps in tutorials/incompressible/pisoFoam/les/motorBike/motorBike/. In this tutorial, they are deleting all the *level* files:
Code:
find . -type f -iname "*level*" -exec rm {} \;
Why ? Is this necessary ?

Second question: When I run paraView on the case, using "decomposed case", I can see the separations between the 14 different processors. Is it normal?
When I try reconstructPar -zeroTime, it crashes:
Code:
--> FOAM FATAL ERROR: 
Size of maps does not correspond to size of mesh for processor 0
faceProcAddressing : 1023858 nFaces : 1028376
cellProcAddressing : 335957 nCell : 337705
boundaryProcAddressing : 10 nFaces : 11
When I try reconstructParMesh , it doesnot work on time 0, but need the -constant option. Then, reconstructPar -zeroTime works fine. And then, paraview works fine on the reconstructed case.
Can anyone explain me all this process ? Why working on constant or zerotime ?

Do you need anything more to help me?

Last edited by Djub; January 21, 2013 at 09:41. Reason: Stupid question has been erased (I solved it on my own)
Djub is offline   Reply With Quote

Old   January 23, 2013, 18:43
Default
  #2
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,251
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings Julien,

I don't have much time to go into details, so I'll be succinct:
  1. The command you showed on the first questions removes files that would just get in the way of decomposition and reconstruction.
  2. It's simpler if you take a look at the cases shared here: http://code.google.com/p/bluecfd-sin...untimes202_211
I know there is a thread on this forum that discusses more on this topic, but I can't find it right now


Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   January 24, 2013, 06:07
Default
  #3
Senior Member
 
Julien
Join Date: Jun 2012
Location: France
Posts: 153
Rep Power: 5
Djub is on a distinguished road
Thanks Bruno,
I think I've catched some more about how it works. In the BlueCFD tuto, you reconstruct the Mesh (only the mesh), and then decompose again. It forces the basic case to correspond with the decomposed case. Nice!
So I suppose I can copy my boundary condition just before to run the ultimate decomposition? Only once, instead of each processor?
Saying:
Code:
cp -r 0.org/* 0/
before final decomposition, instead of
Code:
ls -d processor* | xargs -i cp -r 0.org/* ./{}/0/
after final decomposition.

Thanks for this tuto!
Djub is offline   Reply With Quote

Old   January 24, 2013, 17:01
Default
  #4
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,251
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Julien,

Quote:
Originally Posted by Djub View Post
So I suppose I can copy my boundary condition just before to run the ultimate decomposition? Only once, instead of each processor?
Saying:
Code:
cp -r 0.org/* 0/
before final decomposition, instead of
Code:
ls -d processor* | xargs -i cp -r 0.org/* ./{}/0/
after final decomposition.
Just in case those two weren't rhetorical questions, the answers are:
  1. Yes...
  2. And yes.
But like I said, I know there is a thread somewhere that goes into more detail... mmm, in fact there are two:
  1. Parallel Computing decomposePar
  2. Parallel run: boundary condition missing on processorX/0
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
running OpenFoam in parallel vishwa OpenFOAM 20 November 7, 2012 16:33
Problems running in parallel - missing controlDict Argen OpenFOAM Running, Solving & CFD 4 June 7, 2012 03:50
error while running in parallel using openmpi on local mc 6 processors suryawanshi_nitin OpenFOAM 5 May 21, 2012 14:13
Running mapFields with Parallel Source and Parallel Target RDanks OpenFOAM Mesh Utilities 2 June 6, 2011 07:56
Running in parallel crashed zhajingjing OpenFOAM 4 September 15, 2010 07:12


All times are GMT -4. The time now is 11:53.