CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Native Meshers: snappyHexMesh and Others

Parallel meshing with OP 2.0.0

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   August 5, 2011, 10:37
Default
  #21
Senior Member
 
Aurelien Thinat
Join Date: Jul 2010
Posts: 154
Rep Power: 7
Aurelien Thinat is on a distinguished road
Leong,

I have rebuilt my case folder from scratch :
- the parallel meshing is working but the feature edge handling option is not,
- the serial mode is working with the feature edge handling option.

I will check my case next week, but I think the feature edge handling is only for a serial use for now.

Thank you for your help.

Aurélien
Aurelien Thinat is offline   Reply With Quote

Old   August 16, 2011, 03:36
Default
  #22
New Member
 
Leong
Join Date: Mar 2009
Location: Malaysia
Posts: 20
Rep Power: 8
airfoil is on a distinguished road
Send a message via Skype™ to airfoil
Hi Aurélien,
I already manage to run snappyHexMesh in parallel with edges handling option on OpenFOAM-2.0.x. I not sure what could be your problem as I just follow the motorbike tutorial and applied the same concept on my case. The model that I am working on is cross flow fan. In my opinion, the geometry is quite complex.

Leong
airfoil is offline   Reply With Quote

Old   August 16, 2011, 03:46
Default
  #23
Senior Member
 
Aurelien Thinat
Join Date: Jul 2010
Posts: 154
Rep Power: 7
Aurelien Thinat is on a distinguished road
You are using the command :
"surfaceFeatureExtract -includeAngle 150 constant/triSurface/mygeometry.stl mygeometry"

???

When I check the mesh I get in parallel with OF 2.0.x, it doesn't seems to be the same than with the feature edge handling option.

Are you copying the folder extendedFeatureEdgeMesh in the processor* folder after the decomposition ?
Aurelien Thinat is offline   Reply With Quote

Old   August 16, 2011, 06:13
Default
  #24
New Member
 
Leong
Join Date: Mar 2009
Location: Malaysia
Posts: 20
Rep Power: 8
airfoil is on a distinguished road
Send a message via Skype™ to airfoil
Aurelien,
I am using below script.

#!/bin/sh
# Source tutorial run functions
. $WM_PROJECT_DIR/bin/tools/RunFunctions

runApplication blockMesh

runApplication surfaceFeatureExtract -includedAngle 150 -writeObj constant/triSurface/mygeometry.stl mygeometry
runApplication decomposePar
runParallel snappyHexMesh 12 -overwrite
runApplication reconstructParMesh -constant



You can view mygeometry file whether it has the point for edges or not. If not then, your the edge handling is not working. The file is located on the master folder.

I do not copy the folder extendedFeatureEdgeMesh in the processor*.

I am using hierarchical decomposition. Below is my decomposition file.

/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 2.0.0 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class dictionary;
object decomposeParDict;
}

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //


numberOfSubdomains 12;


method hierarchical;

// method ptscotch;

simpleCoeffs

{
n (4 1 1);
delta 0.001;
}

hierarchicalCoeffs

{
n (3 2 2);
delta 0.001;
order xyz;
}

manualCoeffs

{
dataFile "cellDecomposition";
}


// ************************************************** *********************** //



Do you have this option in your snappyHexMeshDict

features
(
{
file "mygeometry.eMesh";
level 3;
}
);



Hope this help.

Leong
airfoil is offline   Reply With Quote

Old   November 15, 2011, 06:34
Default
  #25
New Member
 
Alexander
Join Date: Dec 2009
Location: Saint-Petersburg
Posts: 29
Rep Power: 7
Svensson is on a distinguished road
Send a message via Skype™ to Svensson
I also had this problem and solved it by changing fvSchemes and fvSolution, as shown below:
Quote:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 2.0.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class dictionary;
location "system";
object fvSchemes;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

gradSchemes
{
}

divSchemes
{
}

laplacianSchemes
{
}

// ************************************************** *********************** //
and:
Quote:
/*--------------------------------*- C++ -*----------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 2.0.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
FoamFile
{
version 2.0;
format ascii;
class dictionary;
object fvSolution;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

solvers
{

}

// ************************************************** *********************** //
Svensson is offline   Reply With Quote

Old   November 15, 2011, 10:08
Default
  #26
New Member
 
Leong
Join Date: Mar 2009
Location: Malaysia
Posts: 20
Rep Power: 8
airfoil is on a distinguished road
Send a message via Skype™ to airfoil
Hi,
I had since move forward with SHM. It has nothing to do with other setting. From my experience, as long we got the correct setting then everything is fine. I am using pt-scotch and scotch to mesh now. I also manage to mesh complex components. I am using SHM to mesh air-condition indoor unit with cross-flow fan. My model consist of two regions which are stator and rotor. The setting for parallel and serial are the same. At least for my case.

Thanks,
Leong
airfoil is offline   Reply With Quote

Old   December 4, 2011, 22:15
Default Similar Problem
  #27
New Member
 
Saxwax
Join Date: Aug 2010
Posts: 17
Rep Power: 7
Saxwax is on a distinguished road
Hi Aurelien,

I seem to be having a similar issue when running snappyHexMesh (sHM) with feature edge handling in parallel on a cluster using OF 2.0.x. I noticed that you also are running on some variety of cluster (--hostfile machines).

On the cluster I am able to run the flange tutorial (in series) and the turbineSitting (in parallel) without any problems. When I try to run the flange tutorial in parallel on the cluster I experience the same problems that you have highlighted. I can run the flange tutorial in parallel without issues on my stand-alone PC. It is also interesting to note that if I run sHM with feature edge handling in parallel on just one of the PCs in my cluster I experience the same issues as you.

In summary,

CLUSTER:
-sHM NO feature edge handling series/parallel OK
-sHM feature edge handling series OK
-sHM feature edge handling parallel FAILS (same error as Aurelien)

Stand-alone PC:
-sHM NO feature edge handling series/parallel OK
-sHM feature edge handling series OK
-sHM feature edge handling parallel OK

Based on the above results I am under the impression that the architecture of the cluster may be in some way responsible for the error. I would be curious to hear what your setup consists of. My specs are as follows:

OS: Ubuntu 10.04 LTS
OF: 2.0.x
PCs/Nodes: 3x PCs with Two 4-core processors each. Gb Ethernet connection.
OF files in /opt/OpenFOAM; run directory in ~/OpenFOAM. Both of these directories are shared via NFS. The home directory uses software RAID5.

Do you have anything similar to this? I am concerned that something funny may be happening with the network sharing or the RAID setup (these are the only things that are different between my cluster and the stand-alone PC). I will try to change how I have my home and /opt/OpenFOAM directories mounted and see if that makes a difference to sHM in parallel with feature edge handling. At least then I will know that the install of OF is OK and my problem lies with the cluster setup in some way.

I'll keep you posted,

D.
Saxwax is offline   Reply With Quote

Old   December 5, 2011, 01:08
Default Doesn't seem to be a cluster related issue.
  #28
New Member
 
Saxwax
Join Date: Aug 2010
Posts: 17
Rep Power: 7
Saxwax is on a distinguished road
I got rid of the RAID5 and the NFS but that didn't help either. I got the same issue as before when running the flange case in parallel as a stand-alone PC. I might try to recompile OpenFOAM and see if that fixes the problem.

Regards,

D.
Saxwax is offline   Reply With Quote

Old   December 5, 2011, 04:26
Default
  #29
Senior Member
 
Aurelien Thinat
Join Date: Jul 2010
Posts: 154
Rep Power: 7
Aurelien Thinat is on a distinguished road
Hi,

To be honest I gave up about this problem.

I was running on OpenSuse system (stand alone PC), it was faster to mesh in serial than to fix the problem...
Anyway, I was able to mesh in serial with the feature edge handling option or in // without it. Even with the ".eMesh" files and the good SHMDict, the output was different in serial and in //.
I made my test on OpenSuse system (stand alone PC) and on a cluster. You will find the cluster specifications here http://www.cfd-numerics.com/IMG/pdf/...uise_Ships.pdf.

The tests were run in June or July. I didn't test it again since the last release of August.

Aurélien
Aurelien Thinat is offline   Reply With Quote

Old   December 5, 2011, 21:05
Default Switched to Ubuntu Package
  #30
New Member
 
Saxwax
Join Date: Aug 2010
Posts: 17
Rep Power: 7
Saxwax is on a distinguished road
Aureilen,

Thanks for the reply. I switched over to the Ubuntu OpenFOAM 2.0.1 package and had the same issues. I'm not sure what is going on - I'll probably just stick to meshing on the stand-alone PC.

D.
Saxwax is offline   Reply With Quote

Old   December 15, 2011, 08:58
Default
  #31
Member
 
Tibor Nyers
Join Date: Jul 2010
Location: Hungary
Posts: 91
Rep Power: 8
Toorop is on a distinguished road
Hi,

I would like to make a little snappyHexMesh sample case which can run in serial and parallel if the system has multiple cores. In serial mode the case reaches the solver phase (the goodness of the boundary / solver setup is another question).

The parallel mode gives the first error at the splitMeshRegion, somehow there are incorrect values in the processor folders. The boundary decomposition fails, it doesn't distribute the correct values per patch (how many faces / patch), instead writes 0 everywhere.

I made the swapping of the decomposition methods (ptscotch / scotch) and added changeDictionary as well. The blockMesh and processor patches are in the processor* folders but without the changeDictionary the surface generated by sMH wont show up. The turbineSiting case has the "procBoundary.*" as well in the changeDictionary, don't know why is that necessary - frankly, why all that is necessary?

Allmake script:
Code:
#!/bin/sh
cd ${0%/*} || exit 1    # run from this directory

# source tutorial run functions
. $WM_PROJECT_DIR/bin/tools/RunFunctions

nProc=$( grep -c processor /proc/cpuinfo )
if [ $nProc > 1 ]
then
    echo "$nProc processors available for parallel execution"
    execution="parallel"
else
    execution="serial"
fi
#uncomment line below to run in serial
#execution="serial"

# create background hex mesh
runApplication blockMesh

if [ $execution = "parallel" ]
then
    sed 's/$nProc/'$nProc'/; s/$ddm/scotch/' system/decomposeParDict.org > system/decomposeParDict
    runApplication decomposePar
    
    sed 's/$nProc/'$nProc'/; s/$ddm/ptscotch/' system/decomposeParDict.org > system/decomposeParDict
    runParallel snappyHexMesh $nProc -overwrite -parallel
    
    for i in $( ls processor* -d );
    do
        # rename sphere patch
        sed -i 's/sphere_region0/'"sphereSurf"'/g' $i/constant/polyMesh/boundary
        sed -i 's/sphere_region0/'"sphereSurf"'/g' $i/0/cellLevel
        sed -i 's/sphere_region0/'"sphereSurf"'/g' $i/0/pointLevel
    done
    
    runParallel changeDictionary $nProc -literalRE -enableFunctionEntries
    
    runParallel splitMeshRegions $nProc -cellZones -overwrite -parallel
else
    sed 's/$nProc/'$nProc'/; s/$ddm/ptscotch/' system/decomposeParDict.org > system/decomposeParDict
    runApplication snappyHexMesh -overwrite

    # rename sphere patch
    sed -i 's/sphere_region0/'"sphereSurf"'/g' constant/polyMesh/boundary
    sed -i 's/sphere_region0/'"sphereSurf"'/g' 0/cellLevel
    sed -i 's/sphere_region0/'"sphereSurf"'/g' 0/pointLevel
    
    runApplication splitMeshRegions -cellZones -overwrite
fi

# geometry & patch check
runApplication checkMesh -allGeometry -allTopology
runApplication patchSummary -constant

if [ $execution = "parallel" ]
then
    # run
    runParallel `getApplication` $nProc

    # reconstruct
    runApplication reconstructPar
else
    runApplication `getApplication`
fi

# ----------------------------------------------------------------- end-of-file
And the icing on the cake: with the exact same settings sMH didn't make any layers, in serial it's okay.

Thank you!
Attached Images
File Type: png parallel.png (69.1 KB, 46 views)
File Type: png serial.png (58.3 KB, 39 views)
Attached Files
File Type: gz sphere_sMH.tar.gz (7.4 KB, 6 views)
Toorop is offline   Reply With Quote

Old   December 16, 2011, 00:43
Default Baby steps
  #32
New Member
 
Saxwax
Join Date: Aug 2010
Posts: 17
Rep Power: 7
Saxwax is on a distinguished road
Hi Tibor,

I have always found it useful to start with something that works, i.e. a tutorial, and then change that 1 step at a time until you get what you are after. Why not try to change the windSimpleFoam tutorial case and understand what is required to run sHM in parallel.

It might be possible to run sHM in parallel without some of the utilities that are used in the windSimpleFoam case. However, a more elaborate way of running sHM may have been used to illustrate some of the features available in OpenFOAM. Or not - I'm just guessing

Good Luck,

D.
Saxwax is offline   Reply With Quote

Old   December 16, 2011, 03:48
Default
  #33
Member
 
Tibor Nyers
Join Date: Jul 2010
Location: Hungary
Posts: 91
Rep Power: 8
Toorop is on a distinguished road
Saxwax, thank you for the advise!

I always jump the gun ... I will take a closer look at the turbine case.

I bug report that produces a similar problem as in my case: snappyHexMesh bugs from running in parallel - 'boundary' files & 0 directory files
Toorop is offline   Reply With Quote

Old   December 23, 2011, 06:42
Default
  #34
New Member
 
Tim Gronarz
Join Date: Sep 2011
Location: Aachen
Posts: 4
Rep Power: 5
TimG is on a distinguished road
Hi Aurelian,

i was facing the same problem as you did. In my case, i was able to solve it by changing the internal name of the STL file to the same as the stl file itself.

geometry
{
Mesh.stl
{
type triSurfaceMesh;
name Mesh;
}
}

On a single core, i was able to compute the Mesh with different names, but for some reason in parallel that did not work.

Althoug you already quit trying, i wanted to share my findings

Cheers
Tim
TimG is offline   Reply With Quote

Old   March 27, 2012, 18:55
Default Ubuntu upgrade to fix issues
  #35
New Member
 
Saxwax
Join Date: Aug 2010
Posts: 17
Rep Power: 7
Saxwax is on a distinguished road
Turns out the issues I was having with parallel sHM and what not were solved by updating my Ubuntu 10.04(.3?) via the update manager. I had initially turned these updates off to keep the system more "stable" and prevent issues like XDMCP becoming unsupported due to a gdm package update etc. However, it appears that OpenFOAM has/had some issues relating to the version of Ubuntu I was using.
Saxwax is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Parallel meshing + simulating. need Help nicolarre OpenFOAM 2 April 11, 2011 11:04
Parallel meshing using XP64 with PVM in CFX Mesh Huw ANSYS Meshing & Geometry 4 July 12, 2010 10:24
OF15 Dynamic Adaptive Meshing in Parallel adona058 OpenFOAM Running, Solving & CFD 2 August 7, 2008 14:27
Meshing support parallel multiple computers! Kevin CD-adapco 1 July 26, 2007 19:27
CFX5.6 (Build) Parallel Meshing David Hargreaves CFX 2 January 12, 2005 13:39


All times are GMT -4. The time now is 15:22.