CFD Online Discussion Forums - Segmentation fault when running in parallel

Hi everybody
I have a case I'm trying to solve. The case runs correctly when I launch on one core.
However when I launch it on multiple cores this is what i find in the standard output:

Code:

/*---------------------------------------------------------------------------*\

| =========                 |                                                 |

| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |

|  \\    /   O peration     | Version:  2.3.0                                 |

|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |

|    \\/     M anipulation  |                                                 |

\*---------------------------------------------------------------------------*/

Build  : 2.3.0-f5222ca19ce6

Exec   : pimpleFoam -parallel

Date   : Apr 01 2015

Time   : 13:27:21

Host   : "n08"

PID    : 14194

Case   : /home/amerio/Catherine_blocks_noSpires_narrow2

nProcs : 8

Slaves :

7

(

"n08.14195"

"n08.14196"

"n08.14197"

"n08.14198"

"n08.14199"

"n08.14200"

"n08.14201"

)



Pstream initialized with:

    floatTransfer      : 0

    nProcsSimpleSum    : 0

    commsType          : nonBlocking

    polling iterations : 0

sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).

fileModificationChecking : Monitoring run-time modified files using timeStampMaster

allowSystemOperations : Disallowing user-supplied system call operations



// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

Create time



Create mesh for time = 0

and this is what i find in the standard error:

Code:

[5] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ost

ream&)[1] #0  Foam::error::printStack(Foam::Ostream&)[4] #0  Foam::error::printStack(Foa

m::Ostream&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so"

[5] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6

4GccDPOpt/lib/libOpenFOAM.so"

[0] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6

4GccDPOpt/lib/libOpenFOAM.so"

[1] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6

4GccDPOpt/lib/libOpenFOAM.so"

[4] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6

4GccDPOpt/lib/libOpenFOAM.so"

[5] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[0] #2   in "/lib/x86_64-linux-gnu/libc.so.6"

[5] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/lib/x86_64-linu

x-gnu/libc.so.6"

[0] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/opt/OpenFOAM/Op

enFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[4] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[1] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[5] #4  Foam::polyBoundaryMesh::updateMesh() in "/lib/x86_64-linux-gnu/libc.so.6"

[4] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/lib/x86_64-linu

x-gnu/libc.so.6"

[1] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/opt/OpenFOAM/Op

enFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[0] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/

linux64GccDPOpt/lib/libOpenFOAM.so"

[5] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0

/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[4] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/

linux64GccDPOpt/lib/libOpenFOAM.so"

[1] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/

linux64GccDPOpt/lib/libOpenFOAM.so"

[0] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0

/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[4] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0

/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[5] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla

tforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[1] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0

/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[0] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla

tforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[4] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla

tforms/linux64GccDPOpt/lib/libOpenFOAM.so"

[1] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla

tforms/linux64GccDPOpt/lib/libfiniteVolume.so"

[5] #7  

 in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"

[0] #7   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.

so"

[4] #7  [5]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"

[5] #8  __libc_start_main



 in "/lib/x86_64-linux-gnu/libc.so.6"

[5] #9   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.

so"

[1] #7  [0]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"

[0] #8  __libc_start_main[4]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt

/bin/pimpleFoam"

[4] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"

[0] #9  



 in "/lib/x86_64-linux-gnu/libc.so.6"

[4] #9  

[1]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"

[1] #8  __libc_start_main[5]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt

/bin/pimpleFoam"

[n08:13738] *** Process received signal ***

[n08:13738] Signal: Segmentation fault (11)

[n08:13738] Signal code:  (-6)

[n08:13738] Failing at address: 0x3ed000035aa

[n08:13738] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f75652211e0]

[n08:13738] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f7565221165]

[n08:13738] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f75652211e0]

[n08:13738] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2b5) [0x7f75663cfc95

]

[n08:13738] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f75663d7729]

[n08:13738] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f7566426027]

[n08:13738] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol

ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f7567e54319]

[n08:13738] [ 7] pimpleFoam() [0x428154]

[n08:13738] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f756520dead

]

[n08:13738] [ 9] pimpleFoam() [0x42b66d]

[n08:13738] *** End of error message ***



[0]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"

[n08:13733] *** Process received signal ***

[n08:13733] Signal: Segmentation fault (11)

[n08:13733] Signal code:  (-6)

[n08:13733] Failing at address: 0x3ed000035a5

[n08:13733] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f325085f1e0]

[n08:13733] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f325085f165]

[n08:13733] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f325085f1e0]

[n08:13733] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2bd) [0x7f3251a0dc9d

]

[n08:13733] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f3251a15729]

[n08:13733] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f3251a64027]

[n08:13733] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol

ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f3253492319]

[n08:13733] [ 7] pimpleFoam() [0x428154]

[n08:13733] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f325084bead

]

[n08:13733] [ 9] pimpleFoam() [0x42b66d]

[n08:13733] *** End of error message ***

[4]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"

[n08:13737] *** Process received signal ***

[n08:13737] Signal: Segmentation fault (11)

[n08:13737] Signal code:  (-6)

[n08:13737] Failing at address: 0x3ed000035a9

 in "/lib/x86_64-linux-gnu/libc.so.6"

[1] #9  

[n08:13737] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f956738b1e0]

[n08:13737] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f956738b165]

[n08:13737] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f956738b1e0]

[n08:13737] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2bd) [0x7f9568539c9d

]

[n08:13737] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f9568541729]

[n08:13737] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.

so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f9568590027]

[n08:13737] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol

ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f9569fbe319]

[n08:13737] [ 7] pimpleFoam() [0x428154]

[n08:13737] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f9567377ead

]

[n08:13737] [ 9] pimpleFoam() [0x42b66d]

[n08:13737] *** End of error message ***

--------------------------------------------------------------------------

mpirun noticed that process rank 5 with PID 13738 on node n08 exited on signal 11 (Segme

ntation fault).

--------------------------------------------------------------------------

To run in parallel I do the common practice

Code:

renumberMesh -overwrite

snappyHexMesh -overwrite

potentialFoam -noFunctionObjects

decomposePar

mpirun  pimpleFoam  -parallel

I already run a case almost identical correctly. I just slightly modified the snappyHexMesh dictionary in order to expand a refinement region that was including part of the mesh to include within it the whole mesh.

CheckMesh returns no error, while checkMesh -allGeometry returns this failure:

Code:

***Concave cells (using face planes) found, number of cells: 35616

Before snappyHexMesh the mesh is COMPLETELY structured with a non ortogonality close to the machine error. snappy only refines some areas and only the castellatedMesh step is set to true.

What could be the cause of that segmentation fault?

The case is running on a unix cluster running SMP Debian 3.2.65-1+deb7u1 x86_64.