Kalabagh |
January 16, 2020 14:48 |
decomposePar ignores some boundary patches
1 Attachment(s)
Hi Foamers,
My simulation is about an industrial stirred mixer. The shaft is almost 10 m long with 5 turbines of 900 mm diam. That means the final snappy mesh has to be very, very large.
As my 4 core computer was not able to perform an accurate mesh when running in serial, I tried in parallel with 3 cores. No way, it's not powerful enough.
The problem I want to solve, before going ahead with 10000000000 cores' cloud servers, is that decomposePar literally IGNORES some patches of the boundary dicts, it only takes the ones that come from blockMesh and does not create all "processorBoundaryXtoX" for each file.
Here is the decomposeParDict, all OK:
Quote:
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 5.x |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 5.x-197d9d3bf20a
Exec : decomposePar
Date : Jan 15 2020
Time : 19:16:50
Host : "ordenador-Precision-T1600"
PID : 6092
I/O : uncollated
Case : /home/ordenador/OpenFOAM/ordenador-5.0/run/OXIRIS
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Decomposing mesh region0
Create mesh
Calculating distribution of cells
Selecting decompositionMethod scotch
Finished decomposition in 0.17 s
Calculating original mesh data
Distributing cells to processors
Distributing faces to processors
Distributing points to processors
Constructing processor meshes
Processor 0
Number of cells = 20073
Number of faces shared with processor 2 = 1275
Number of processor patches = 1
Number of processor faces = 1275
Number of boundary faces = 2543
Processor 1
Number of cells = 19900
Number of faces shared with processor 2 = 1325
Number of processor patches = 1
Number of processor faces = 1325
Number of boundary faces = 2523
Processor 2
Number of cells = 20027
Number of faces shared with processor 0 = 1275
Number of faces shared with processor 1 = 1325
Number of processor patches = 2
Number of processor faces = 2600
Number of boundary faces = 1334
Number of processor faces = 2600
Max number of cells = 20073 (0.365% above average 20000)
Max number of processor patches = 2 (50% above average 1.33333)
Max number of faces between processors = 2600 (50% above average 1733.33)
Time = 0
Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
End
|
Here is an example of the original boundary dict U:
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 5.x |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class volVectorField;
location "0";
object U;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
dimensions [0 1 -1 0 0 0 0];
internalField uniform (0 0 0);
boundaryField
{
walls
{
type noSlip;
}
inlet
{
type noSlip;
}
outlet
{
type noSlip;
}
TURBINA
{
type noSlip;
}
REACTOR
{
type noSlip;
}
ROTOR
{
type fixedValue;
value uniform (0 0 0);
}
ROTOR_slave
{
type fixedValue;
value uniform (0 0 0);
}
}
// ************************************************************************* //
And the U dictionary inside the folder "processor0":
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 5.x |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class volVectorField;
location "0";
object U;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
dimensions [0 1 -1 0 0 0 0];
internalField uniform (0 0 0);
boundaryField
{
walls
{
type noSlip;
}
inlet
{
type noSlip;
}
outlet
{
type noSlip;
}
procBoundary0to2
{
type processor;
value uniform (0 0 0);
}
}
// ************************************************************************* //
When it's about 3 cores it's easy to edit all the dictionaries in the processor* folders, but I want to try on an 8 core server...
Here are blockMeshDict and snappyHexMeshDict for more info:
blockMeshDict has inlet and outlet (I don't really need an inlet and outlet) because I'm using an improved cylinder-square background mesh and I don't want to modify the code.
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | |
| \\ / A nd | Copyright (C) 2016 Ehsan Madadi-Kandjani |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class dictionary;
object blockMeshDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
// General macros to create cylinder mesh
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
convertToMeters 1;
// Inner square side half
// Inner square side curvature
// cylinder radius
// Height of cylinder
// Base z
// Outlet z
// Number of cells at inner square
// Number of cells between inner square and circle
// Number of cells in the cylinder height
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
// 45 degree points angle
// Half of 45 degree points angle
// Inner square x and y position
// x
// y
// Circle x and y positions
// x
// y
// Inner square x and y position middle curvatures
// x
// y
// Circle x and y positions middle curvatures
// x
// y
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
vertices
(
(0.6 -0.6 -1.0) // Vertex s0b = 0
(-0.6 -0.6 -1.0) // Vertex s1b = 1
(-0.6 0.6 -1.0) // Vertex s2b = 2
(0.6 0.6 -1.0) // Vertex s3b = 3
(0.777817459305203 -0.777817459305202 -1.0) // Vertex r0b = 4
(-0.777817459305202 -0.777817459305203 -1.0) // Vertex r1b = 5
(-0.777817459305202 0.777817459305203 -1.0) // Vertex r2b = 6
(0.777817459305203 0.777817459305202 -1.0) // Vertex r3b = 7
(0.6 -0.6 4.3) // Vertex s0t = 8
(-0.6 -0.6 4.3) // Vertex s1t = 9
(-0.6 0.6 4.3) // Vertex s2t = 10
(0.6 0.6 4.3) // Vertex s3t = 11
(0.777817459305203 -0.777817459305202 4.3) // Vertex r0t = 12
(-0.777817459305202 -0.777817459305203 4.3) // Vertex r1t = 13
(-0.777817459305202 0.777817459305203 4.3) // Vertex r2t = 14
(0.777817459305203 0.777817459305202 4.3) // Vertex r3t = 15
);
blocks
(
//block0
hex (1 0 3 2 9 8 11 10)
square
(20 20 50)
simpleGrading (1 1 1)
//block1
hex (0 4 7 3 8 12 15 11)
innerCircle
(10 20 50)
simpleGrading (1 1 1)
//block2
hex (3 7 6 2 11 15 14 10)
innerCircle
(10 20 50)
simpleGrading (1 1 1)
//block3
hex (2 6 5 1 10 14 13 9)
innerCircle
(10 20 50)
simpleGrading (1 1 1)
//block4
hex (1 5 4 0 9 13 12 8)
innerCircle
(10 20 50)
simpleGrading (1 1 1)
);
edges
(
//Circle edges
arc 7 4 (1.1 0 -1.0)
arc 4 5 (6.73555739531045e-17 -1.1 -1.0)
arc 5 6 (-1.1 1.34711147906209e-16 -1.0)
arc 6 7 (6.73555739531045e-17 1.1 -1.0)
//Circle edges
arc 15 12 (1.1 0 4.3)
arc 12 13 (6.73555739531045e-17 -1.1 4.3)
arc 13 14 (-1.1 1.34711147906209e-16 4.3)
arc 14 15 (6.73555739531045e-17 1.1 4.3)
arc 3 0 (0.7 0 -1.0)
arc 0 1 (0 -0.7 -1.0)
arc 1 2 (-0.7 0 -1.0)
arc 2 3 (0 0.7 -1.0)
arc 11 8 (0.7 0 4.3)
arc 8 9 (0 -0.7 4.3)
arc 9 10 (-0.7 0 4.3)
arc 10 11 (0 0.7 4.3)
);
patches
(
wall walls
(
(4 7 15 12)
(5 4 12 13)
(6 5 13 14)
(7 6 14 15)
)
patch inlet
(
(3 0 1 2)
(3 7 4 0)
(2 6 7 3)
(1 5 6 2)
(0 4 5 1)
)
patch outlet
(
(11 10 9 8)
(11 8 12 15)
(10 11 15 14)
(9 10 14 13)
(8 9 13 12)
)
);
mergePatchPairs
(
);
Code:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 5 |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class dictionary;
object snappyHexMeshDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
// Which of the steps to run
castellatedMesh true;
snap true;
addLayers true;
geometry
{
TURBINA
{
type triSurfaceMesh;
file "TURBINA_SCALED.stl";
}
REACTOR
{
type triSurfaceMesh;
file "REACTOR_SCALED.stl";
}
ROTOR
{
type triSurfaceMesh;
file "ROTOR_SCALED.stl";
}
};
// Settings for the castellatedMesh generation.
castellatedMeshControls
{
// Refinement parameters
// ~~~~~~~~~~~~~~~~~~~~~
// If local number of cells is >= maxLocalCells on any processor
// switches from from refinement followed by balancing
// (current method) to (weighted) balancing before refinement.
maxLocalCells 1000000;
// Overall cell limit (approximately). Refinement will stop immediately
// upon reaching this number so a refinement level might not complete.
// Note that this is the number of cells before removing the part which
// is not 'visible' from the keepPoint. The final number of cells might
// actually be a lot less.
maxGlobalCells 4000000;
// The surface refinement loop might spend lots of iterations refining just a
// few cells. This setting will cause refinement to stop if <= minimumRefine
// are selected for refinement. Note: it will at least do one iteration
// (unless the number of cells to refine is 0)
minRefinementCells 10;
// Allow a certain level of imbalance during refining
// (since balancing is quite expensive)
// Expressed as fraction of perfect balance (= overall number of cells /
// nProcs). 0=balance always.
maxLoadUnbalance 0.10;
// Number of buffer layers between different levels.
// 1 means normal 2:1 refinement restriction, larger means slower
// refinement.
nCellsBetweenLevels 3;
// Explicit feature edge refinement
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Specifies a level for any cell intersected by its edges.
// This is a featureEdgeMesh, read from constant/triSurface for now.
features
(
{
file "TURBINA_SCALED.eMesh";
level 6;
}
{
file "REACTOR_SCALED.eMesh";
level 5;
}
{
file "ROTOR_SCALED.eMesh";
level 0;
}
);
// Surface based refinement
// ~~~~~~~~~~~~~~~~~~~~~~~~
// Specifies two levels for every surface. The first is the minimum level,
// every cell intersecting a surface gets refined up to the minimum level.
// The second level is the maximum level. Cells that 'see' multiple
// intersections where the intersections make an
// angle > resolveFeatureAngle get refined up to the maximum level.
refinementSurfaces
{
TURBINA
{
// Surface-wise min and max refinement level
level (3 4);
// Optional specification of patch type (default is wall). No
// constraint types (cyclic, symmetry) etc. are allowed.
patchInfo
{
type wall;
}
}
REACTOR
{
level (3 4);
patchInfo
{
type wall;
}
}
ROTOR
{
level (0 0);//1 2);
cellZone ROTOR;
faceZone ROTOR;
cellZoneInside inside;
}
}
// Resolve sharp angles
resolveFeatureAngle 1;
// Region-wise refinement
// ~~~~~~~~~~~~~~~~~~~~~~
// Specifies refinement level for cells in relation to a surface. One of
// three modes
// - distance. 'levels' specifies per distance to the surface the
// wanted refinement level. The distances need to be specified in
// descending order.
// - inside. 'levels' is only one entry and only the level is used. All
// cells inside the surface get refined up to the level. The surface
// needs to be closed for this to be possible.
// - outside. Same but cells outside.
refinementRegions
{
REACTOR
{
mode inside;
levels ((0 0));//1));
}
ROTOR
{
mode inside;
levels ((1 1));//1 1));
cellZone ROTOR;
faceZone ROTOR;
cellZoneInside inside;
}
}
// Mesh selection
// ~~~~~~~~~~~~~~
// After refinement patches get added for all refinementSurfaces and
// all cells intersecting the surfaces get put into these patches. The
// section reachable from the locationInMesh is kept.
// NOTE: This point should never be on a face, always inside a cell, even
// after refinement.
locationInMesh (0.05000000 0.0000000 0.0000000);
// Whether any faceZones (as specified in the refinementSurfaces)
// are only on the boundary of corresponding cellZones or also allow
// free-standing zone faces. Not used if there are no faceZones.
allowFreeStandingZoneFaces true;
}
// Settings for the snapping.
snapControls
{
//- Number of patch smoothing iterations before finding correspondence
// to surface
nSmoothPatch 4;
//- Relative distance for points to be attracted by surface feature point
// or edge. True distance is this factor times local
// maximum edge length.
tolerance 2.0;
//- Number of mesh displacement relaxation iterations.
nSolveIter 30;
//- Maximum number of snapping relaxation iterations. Should stop
// before upon reaching a correct mesh.
nRelaxIter 5;
// Feature snapping
//- Number of feature edge snapping iterations.
// Leave out altogether to disable.
nFeatureSnapIter 10;
//- Detect (geometric only) features by sampling the surface
// (default=false).
implicitFeatureSnap false;
//- Use castellatedMeshControls::features (default = true)
explicitFeatureSnap true;
//- Detect points on multiple surfaces (only for explicitFeatureSnap)
multiRegionFeatureSnap false;
}
// Settings for the layer addition.
addLayersControls
{
// Are the thickness parameters below relative to the undistorted
// size of the refined cell outside layer (true) or absolute sizes (false).
relativeSizes true;
// Per final patch (so not geometry!) the layer information
layers
{
TURBINA
{
nSurfaceLayers 3;
}
REACTOR
{
nSurfaceLayers 3;
}
}
// Expansion factor for layer mesh
expansionRatio 1.5;
// Wanted thickness of final added cell layer. If multiple layers
// is the thickness of the layer furthest away from the wall.
// Relative to undistorted size of cell outside layer.
// See relativeSizes parameter.
finalLayerThickness 0.5;
// Minimum thickness of cell layer. If for any reason layer
// cannot be above minThickness do not add layer.
// Relative to undistorted size of cell outside layer.
minThickness 0.1;
// If points get not extruded do nGrow layers of connected faces that are
// also not grown. This helps convergence of the layer addition process
// close to features.
// Note: changed(corrected) w.r.t 17x! (didn't do anything in 17x)
nGrow 0;
// Advanced settings
// When not to extrude surface. 0 is flat surface, 90 is when two faces
// are perpendicular
featureAngle 100;
// At non-patched sides allow mesh to slip if extrusion direction makes
// angle larger than slipFeatureAngle.
slipFeatureAngle 30;
// Maximum number of snapping relaxation iterations. Should stop
// before upon reaching a correct mesh.
nRelaxIter 3;
// Number of smoothing iterations of surface normals
nSmoothSurfaceNormals 1;
// Number of smoothing iterations of interior mesh movement direction
nSmoothNormals 3;
// Smooth layer thickness over surface patches
nSmoothThickness 10;
maxFaceThicknessRatio 0.5;
maxThicknessToMedialRatio 0.3;
minMedianAxisAngle 90;
nBufferCellsNoExtrude 0;
nLayerIter 50;
}
meshQualityControls
{
#include "meshQualityDict"
nSmoothScale 4;
errorReduction 0.75;
}
writeFlags
(
scalarLevels
layerSets
layerFields // write volScalarField for layer coverage
);
// ************************************************************************* //
Thanks.
|