[Priya Somasundaran]

I am trying to run a model with 14596 cells with 3 node and one processor each using parellel computing. My parallel computing was tested and is working fine when I tested for the pitZDaily example from the tutorial.
But when I am trying to decompose the model with 14596 cells I get the following error
################################################
Decomposing mesh region0

Create mesh

Calculating distribution of cells
Selecting decompositionMethod hierarchical

Finished decomposition in 8.27 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes
new cannot satisfy memory request.
This does not necessarily mean you have run out of virtual memory.
It could be due to a stack violation caused by e.g. bad use of pointers or an out of date shared library
Aborted (core dumped)
################################################## #

I am using the same decomposeParDict which I used to decompose the pitZDaily case
###############################################
numberOfSubdomains 3;

method hierarchical;


simpleCoeffs
{
n (3 1 1);
delta 0.001;
}
hierarchicalCoeffs
{
n (3 1 1);
delta 0.001;
order xyz;
}

################################################## ###########

Any help would be much appreciated .

[fabian_roesler]

Hi

Is it a 2D case? Cause decomposePar doesn't like decomposing in the empty direction.
You could also try the decomposing method scotch.

numberOfSubdomains 3;

method scotch;

[Priya Somasundaran]

Hi,
Thanks fro responding. It is a 3D buildings mesh. And contrary to what I wrote there are 6331789 cells distributed in each of the processors. I have 3 nodes with one processor each with master having 32 GB of memory and the slaves 16 each.
I tried to use scotch distribution method also , but but a smiliar memory error. Now I am trying to run the distributionPar using MPIRUN so that i can distibute the mesh parallel. But failed.

/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 2.4.0 |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 2.4.0-f0842aea0e77
Exec : redistributePar -parallel
Date : Oct 26 2015
Time : 17:58:48
Host : "Geoscience-PC"
PID : 10202
Case : /quest_cluster/home/quest/OpenFOAM/quest-2.4.0/run/simulation
nProcs : 3
Slaves :
2
(
"Quest-2.14289"
"Quest-1.28844"
)

Pstream initialized with:
floatTransfer : 0
nProcsSimpleSum : 0
commsType : nonBlocking
polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
--> FOAM Warning :
From function redistributePar
in file redistributePar.C at line 444
Detected floating point exception trapping (FOAM_SIGFPE). This might give
problems when mapping fields. Switch it off in case of problems.
Create time

Using mesh subdirectory "polyMesh"

Found points in "/quest_cluster/home/quest/OpenFOAM/quest-2.4.0/run/simulation/processor0/constant/polyMesh"

Per processor mesh availability : 3{1}
Boundary definition OK.
Before distribution:

Processor 0
Number of cells = 6331789
Number of processor patches = 0
Number of processor faces = 0
Number of boundary faces = 1366151

Processor 1
Number of cells = 6331789
Number of processor patches = 0
Number of processor faces = 0
Number of boundary faces = 1366151

Processor 2
Number of cells = 6331789
Number of processor patches = 0
Number of processor faces = 0
Number of boundary faces = 1366151

Number of processor faces = 0
Max number of cells = 6331789 (0% above average 6.33179e+06)
Max number of processor patches = 0 (-100% above average 1)
Max number of faces between processors = 0 (-100% above average 1)

Selecting decompositionMethod hierarchical
Writing wanted cell distribution to volScalarField decomposition for postprocessing purposes.

Merge tolerance : 1e-06
Write tolerance : 1e-06
Overall meshes bounding box : (16700 34600 0) (18000 36000 500)
Relative tolerance : 1e-06
Absolute matching distance : 0.00197484

new cannot satisfy memory request.
This does not necessarily mean you have run out of virtual memory.
It could be due to a stack violation caused by e.g. bad use of pointers or an out of date shared library
[Quest-1:28844] *** Process received signal ***
[Quest-1:28844] Signal: Aborted (6)
[Quest-1:28844] Signal code: (-6)
[Quest-1:28844] *** End of error message ***
new cannot satisfy memory request.
This does not necessarily mean you have run out of virtual memory.
It could be due to a stack violation caused by e.g. bad use of pointers or an out of date shared library
[Quest-2:14289] *** Process received signal ***
[Quest-2:14289] Signal: Aborted (6)
[Quest-2:14289] Signal code: (-6)
[Quest-2:14289] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 28844 on node 192.168.1.107 exited on signal 6 (Aborted).


################################################## ######################

Any clues ? AM I still short of memory ?

[fabian_roesler]

Well, 7M Cells should be possible on 32 GB memory. Parallel decomposition is not possible. You have to decompose single core on your master node.
You could top your memory usage while decomposing.
Have you tried running single core on the master node?

Cheers

Fabian