CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

Problem with parallel run

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By Alhasan

Reply
 
LinkBack Thread Tools Display Modes
Old   October 27, 2011, 13:26
Default Problem with parallel run
  #1
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 8
Hisham is on a distinguished road
Dear Foamers,

I am developing a solver. It runs OK as a serial job. I need to run it in parallel. I cannot apply decomposePar! I get this error for the last processor (regardless of number of processors). I have nothing in the code nor the input files that wants this "time-series" thing!!!

I tried the decomposeParDict with a tutorial and it runs.

Any ideas are appreciated!

Error:
Code:
Constructing processor meshes

Processor 0
    Number of cells = 22542
    Number of faces shared with processor 1 = 109
    Number of processor patches = 1
    Number of processor faces = 109
    Number of boundary faces = 45409

Processor 1
    Number of cells = 22542
    Number of faces shared with processor 0 = 109
    Number of faces shared with processor 2 = 95
    Number of processor patches = 2
    Number of processor faces = 204
    Number of boundary faces = 45332

Processor 2
    Number of cells = 22542
    Number of faces shared with processor 1 = 95
    Number of faces shared with processor 3 = 97
    Number of processor patches = 2
    Number of processor faces = 192
    Number of boundary faces = 45334

Processor 3
    Number of cells = 22542
    Number of faces shared with processor 2 = 97
    Number of processor patches = 1
    Number of processor faces = 97
    Number of boundary faces = 45407

Number of processor faces = 301
Max number of cells = 22542 (0% above average 22542)
Max number of processor patches = 2 (33.3333% above average 1.5)
Max number of faces between processors = 204 (35.5482% above average 150.5)



--> FOAM FATAL IO ERROR: 
file "/home/hisham/OpenFOAM/numubuntu-2.0.0/waveTankFoam/fineTutorial_waveTankFoam/time-series" does not exist

file: /home/hisham/OpenFOAM/numubuntu-2.0.0/waveTankFoam/fineTutorial_waveTankFoam/time-series at line 1.

    From function IFstream::operator()
    in file db/IOstreams/Fstreams/IFstream.C at line 178.

FOAM exiting
The Dictionary:
Code:
numberOfSubdomains 4;

method          simple;

simpleCoeffs
{
    n               ( 4 1 1 );
    delta           0.001;
}

hierarchicalCoeffs
{
    n               ( 1 1 1 );
    delta           0.001;
    order           xyz;
}

metisCoeffs
{
    processorWeights ( 1 1 1 1 );
}

manualCoeffs
{
    dataFile        "";
}

distributed     no;

roots           ( );
Hisham is offline   Reply With Quote

Old   October 27, 2011, 14:49
Default
  #2
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 8
Hisham is on a distinguished road
The same error message issues for foamToVTK !!
Hisham is offline   Reply With Quote

Old   October 28, 2011, 14:15
Default
  #3
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings Hisham,

Mmm, that's an odd error message...
My guess is that you don't have any time instance folder, such as "0". I can't test this right now, so may I ask you to post the tree list of your case folder:
Code:
tree -s
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   October 28, 2011, 19:02
Default
  #4
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 8
Hisham is on a distinguished road
Code:
.
├── [       4096]  0
│** ├── [       1309]  alpha
│** ├── [     182010]  alpha1
│** ├── [       1333]  alpha1~
│** ├── [       1464]  alpha1.org
│** ├── [       1464]  alpha1.org~
│** ├── [       1334]  B
│** ├── [       1631]  B~
│** ├── [       1324]  cellMotionUx~
│** ├── [       1383]  cellMotionUx.org
│** ├── [       1424]  k
│** ├── [       1889]  k~
│** ├── [       1211]  nuSgs
│** ├── [       1515]  nuSgs~
│** ├── [       1294]  nuTilda
│** ├── [       1624]  nuTilda~
│** ├── [       4096]  OldFilesForRANS
│** │** ├── [       1324]  cellMotionUx
│** │** ├── [       1923]  epsilon
│** │** ├── [       1843]  mut
│** │** ├── [       1832]  nut
│** │** └── [       2019]  R
│** ├── [       1327]  pointMotionUx~
│** ├── [       1386]  pointMotionUx.org
│** ├── [       1551]  p_rgh
│** ├── [       2001]  p_rgh~
│** ├── [       2588]  U
│** └── [       1420]  U~
├── [       4096]  constant
│** ├── [       4096]  boundaryData
│** │** └── [       4096]  waveMaker
│** │**     ├── [       4096]  0
│** │**     │** ├── [        912]  alpha1
│** │**     │** └── [        912]  alpha1~
│** │**     ├── [       1279]  points
│** │**     └── [       1156]  points~
│** ├── [       1048]  dynamicMeshDict
│** ├── [        939]  g
│** ├── [       1780]  LESProperties
│** ├── [       4096]  polyMesh
│** │** ├── [       8471]  blockMeshDict
│** │** ├── [       1573]  boundary
│** │** ├── [       1694]  boundary~
│** │** ├── [     531206]  cellZones
│** │** ├── [    7335351]  faces
│** │** ├── [        877]  faceZones
│** │** ├── [     801419]  neighbour
│** │** ├── [    1850003]  owner
│** │** ├── [    1952171]  points
│** │** ├── [        878]  pointZones
│** │** └── [       4096]  sets
│** │**     ├── [     498418]  flume
│** │**     ├── [      25549]  sponge1
│** │**     ├── [      25213]  sponge2
│** │**     ├── [      25213]  sponge3
│** │**     ├── [      25423]  sponge4
│** │**     └── [      25535]  sponge5
│** ├── [       1168]  porousZones
│** ├── [       2284]  transportProperties
│** └── [        910]  turbulenceProperties
├── [          0]  fineTutorial_waveTankFoam.OpenFOAM
├── [  445603053]  log
├── [       2214]  PyFoamHistory
├── [       4096]  system
│** ├── [       1317]  controlDict
│** ├── [       1319]  controlDict~
│** ├── [       1234]  decomposeParDict
│** ├── [       1234]  decomposeParDict~
│** ├── [       1553]  fvSchemes
│** ├── [       2399]  fvSolution
│** ├── [       2399]  fvSolution~
│** ├── [       1151]  setFieldsDict
│** ├── [       1151]  #setFieldsDict#
│** ├── [       1151]  setFieldsDict~
│** ├── [       3577]  waterWavesDict
│** └── [       3293]  waterWavesDict~
├── [       2053]  waveTank.geo
└── [   14650731]  waveTank.msh
Hisham is offline   Reply With Quote

Old   October 29, 2011, 04:50
Default
  #5
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Hisham,

You have many files that shouldn't be where they are. Allow me to explain:
  • A solver or utility usually knows which files it needs for a certain type of simulation or certain mesh structure. It will ignore completely any other files lying around.
  • decomposePar only knows that all files in the time folders and in the constant folder and system folder should be split and/or copied to each processor.
    Some corner cases might exists, such as the one you are getting, where I can only estimate that some types of zones or sets are expected to have time series associated to them.
Please keep in mind that I'm not familiar with any solver that requires the "constant/boundaryData" folder nor require the times series file, so I'm just going to suggest what I would do... So, I suggest that you:
  1. Make a backup of this case before you do the next steps.
  2. Move the folder "0" to "0.org".
  3. Create a new empty folder "0".
  4. Copy from "0.org" to "0" only the files that are strictly necessary for the simulation.
  5. Create a folder named "system_sandbox" (it's a random name).
  6. Move all files in "system" to "system_sandbox" that are not essential to the simulation.
  7. In the folder "constant", remove files that end with a tilde "~" in the end of their name. These are backup files probably created by a text editor. If you don't want to remove them, move them to another folder that OpenFOAM doesn't look into.
  8. Try running decomposePar once again.
  9. If it still complains, then create a folder named "constant_sandbox" and move "boundaryData" from "constant" to this new folder.
  10. Try running decomposePar once again.
  11. If it no longer complains, but if you need "boundaryData", then you'll have to check the original case you used as basis and check how exactly it works, since the time series file might be necessary for files in "boundaryData".
Keep in mind that each simulation case, although purely digital, should be kept neatly tidy as well as you would in a real life experiment of said case
You wouldn't leave a screw driver on the ground of a wind tunnel experiment and then turn on the fans, would you?

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   October 29, 2011, 06:02
Default
  #6
Senior Member
 
Hisham's Avatar
 
Hisham El Safti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 247
Blog Entries: 10
Rep Power: 8
Hisham is on a distinguished road
Hi Bruno,

As you've expected. It is the boundaryData that causes the problem. I do not need the timeVaryingMapped BC for now, so I guess your advice for keeping things tidy pays off.

I removed the boundaryData altogether and it decomposes now fine.

Thanks
Hisham
Hisham is offline   Reply With Quote

Old   March 7, 2012, 22:26
Default
  #7
Senior Member
 
Alhasan's Avatar
 
Hasan K.J.
Join Date: Dec 2011
Location: Bristol, United Kingdom
Posts: 181
Rep Power: 5
Alhasan is on a distinguished road
hey, i did this mesh in salome and ideasToUnv worked fine but after that when i decomposed it i had this error


Constructing processor meshes
#0 Foam::error:rintStack(Foam::Ostream&) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#1 Foam::sigSegv::sigHandler(int) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#2 in "/lib/x86_64-linux-gnu/libc.so.6"
#3
in "/opt/openfoam201/platforms/linux64GccDPOpt/bin/decomposePar"
#4
in "/opt/openfoam201/platforms/linux64GccDPOpt/bin/decomposePar"
#5
in "/opt/openfoam201/platforms/linux64GccDPOpt/bin/decomposePar"
#6 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
#7
in "/opt/openfoam201/platforms/linux64GccDPOpt/bin/decomposePar"
Segmentation fault

i have had no errors so far with my decomposePar ever... please help me I'm just stuck
Alhasan is offline   Reply With Quote

Old   March 8, 2012, 05:35
Default
  #8
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings Alhasan,

Did you check the mesh?
Code:
checkMesh
Also, consider upgrading to OpenFOAM 2.1.0, in case checkMesh gives you a good validation. This way it'll be easier to infer if the problem is due to a bug that has already been fixed or not.

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   March 8, 2012, 21:34
Default
  #9
Senior Member
 
Alhasan's Avatar
 
Hasan K.J.
Join Date: Dec 2011
Location: Bristol, United Kingdom
Posts: 181
Rep Power: 5
Alhasan is on a distinguished road
there was a problem with the mesh...!!!! sorry for the inconvinence... thanks ...

the problem was the there was one extra face on just one wall..!!!! and it couldn't segment the number of cells equally i guess...!!!

thanks.
wyldckat likes this.
Alhasan is offline   Reply With Quote

Old   March 13, 2012, 09:31
Default
  #10
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Quote:
Originally Posted by Alhasan View Post
the problem was the there was one extra face on just one wall..!!!! and it couldn't segment the number of cells equally i guess...!!!
Well, if you were using Scotch for decomposition, then apparently that might also have been a bug/limitation: http://www.openfoam.org/mantisbt/view.php?id=456
wyldckat is offline   Reply With Quote

Reply

Tags
decomposepar, parallel

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
area does not match neighbour by ... % -- possible face ordering problem St.Pacholak OpenFOAM 9 November 22, 2011 11:02
First Parallel Run - need some help Gian Maria OpenFOAM 3 June 17, 2011 12:08
nonNewtonianIcoFoam - problem with parallel run chris_sev OpenFOAM Bugs 4 April 1, 2009 09:13
Parallel run with engineFoam francesco OpenFOAM Bugs 1 November 25, 2008 08:06
Problem on Parallel Run Setup Hamidur Rahman CFX 0 September 23, 2007 17:11


All times are GMT -4. The time now is 03:28.