Segmentation fault when running in parallel

Pj. · April 1, 2015, 08:31

Hi everybody
I have a case I'm trying to solve. The case runs correctly when I launch on one core.
However when I launch it on multiple cores this is what i find in the standard output:

Code:

/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.0-f5222ca19ce6
Exec   : pimpleFoam -parallel
Date   : Apr 01 2015
Time   : 13:27:21
Host   : "n08"
PID    : 14194
Case   : /home/amerio/Catherine_blocks_noSpires_narrow2
nProcs : 8
Slaves :
7
(
"n08.14195"
"n08.14196"
"n08.14197"
"n08.14198"
"n08.14199"
"n08.14200"
"n08.14201"
)

Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

and this is what i find in the standard error:

Code:

[5] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ost
ream&)[1] #0  Foam::error::printStack(Foam::Ostream&)[4] #0  Foam::error::printStack(Foa
m::Ostream&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so"
[5] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6
4GccDPOpt/lib/libOpenFOAM.so"
[0] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6
4GccDPOpt/lib/libOpenFOAM.so"
[1] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6
4GccDPOpt/lib/libOpenFOAM.so"
[4] #1  Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux6
4GccDPOpt/lib/libOpenFOAM.so"
[5] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #2   in "/lib/x86_64-linux-gnu/libc.so.6"
[5] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/lib/x86_64-linu
x-gnu/libc.so.6"
[0] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/opt/OpenFOAM/Op
enFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #2   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #4  Foam::polyBoundaryMesh::updateMesh() in "/lib/x86_64-linux-gnu/libc.so.6"
[4] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/lib/x86_64-linu
x-gnu/libc.so.6"
[1] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/opt/OpenFOAM/Op
enFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/
linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0
/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/
linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #4  Foam::polyBoundaryMesh::updateMesh() in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/
linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0
/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0
/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla
tforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0
/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[0] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla
tforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla
tforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/opt/OpenFOAM/OpenFOAM-2.3.0/pla
tforms/linux64GccDPOpt/lib/libfiniteVolume.so"
[5] #7  
 in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
[0] #7   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.
so"
[4] #7  [5]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"
[5] #8  __libc_start_main

 in "/lib/x86_64-linux-gnu/libc.so.6"
[5] #9   in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.
so"
[1] #7  [0]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"
[0] #8  __libc_start_main[4]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt
/bin/pimpleFoam"
[4] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[0] #9  

 in "/lib/x86_64-linux-gnu/libc.so.6"
[4] #9  
[1]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"
[1] #8  __libc_start_main[5]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt
/bin/pimpleFoam"
[n08:13738] *** Process received signal ***
[n08:13738] Signal: Segmentation fault (11)
[n08:13738] Signal code:  (-6)
[n08:13738] Failing at address: 0x3ed000035aa
[n08:13738] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f75652211e0]
[n08:13738] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f7565221165]
[n08:13738] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f75652211e0]
[n08:13738] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2b5) [0x7f75663cfc95
]
[n08:13738] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f75663d7729]
[n08:13738] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f7566426027]
[n08:13738] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol
ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f7567e54319]
[n08:13738] [ 7] pimpleFoam() [0x428154]
[n08:13738] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f756520dead
]
[n08:13738] [ 9] pimpleFoam() [0x42b66d]
[n08:13738] *** End of error message ***

[0]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"
[n08:13733] *** Process received signal ***
[n08:13733] Signal: Segmentation fault (11)
[n08:13733] Signal code:  (-6)
[n08:13733] Failing at address: 0x3ed000035a5
[n08:13733] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f325085f1e0]
[n08:13733] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f325085f165]
[n08:13733] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f325085f1e0]
[n08:13733] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2bd) [0x7f3251a0dc9d
]
[n08:13733] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f3251a15729]
[n08:13733] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f3251a64027]
[n08:13733] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol
ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f3253492319]
[n08:13733] [ 7] pimpleFoam() [0x428154]
[n08:13733] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f325084bead
]
[n08:13733] [ 9] pimpleFoam() [0x42b66d]
[n08:13733] *** End of error message ***
[4]  in "/opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/bin/pimpleFoam"
[n08:13737] *** Process received signal ***
[n08:13737] Signal: Segmentation fault (11)
[n08:13737] Signal code:  (-6)
[n08:13737] Failing at address: 0x3ed000035a9
 in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #9  
[n08:13737] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f956738b1e0]
[n08:13737] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f956738b165]
[n08:13737] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f956738b1e0]
[n08:13737] [ 3] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2bd) [0x7f9568539c9d
]
[n08:13737] [ 4] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b9) [0x7f9568541729]
[n08:13737] [ 5] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.
so(_ZN4Foam8polyMeshC1ERKNS_8IOobjectE+0xd47) [0x7f9568590027]
[n08:13737] [ 6] /opt/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVol
ume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f9569fbe319]
[n08:13737] [ 7] pimpleFoam() [0x428154]
[n08:13737] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f9567377ead
]
[n08:13737] [ 9] pimpleFoam() [0x42b66d]
[n08:13737] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 5 with PID 13738 on node n08 exited on signal 11 (Segme
ntation fault).
--------------------------------------------------------------------------

To run in parallel I do the common practice

Code:

renumberMesh -overwrite
snappyHexMesh -overwrite
potentialFoam -noFunctionObjects
decomposePar
mpirun  pimpleFoam  -parallel

I already run a case almost identical correctly. I just slightly modified the snappyHexMesh dictionary in order to expand a refinement region that was including part of the mesh to include within it the whole mesh.

CheckMesh returns no error, while checkMesh -allGeometry returns this failure:

Code:

***Concave cells (using face planes) found, number of cells: 35616

Before snappyHexMesh the mesh is COMPLETELY structured with a non ortogonality close to the machine error. snappy only refines some areas and only the castellatedMesh step is set to true.

What could be the cause of that segmentation fault?

The case is running on a unix cluster running SMP Debian 3.2.65-1+deb7u1 x86_64.

jcespada · April 1, 2015, 09:32

Maybe I'm wrong, but I think you have to indicate the number of processors.
For example:

mpirun -np 8 pimpleFoam -parallel

Pj. · April 1, 2015, 09:48

Quote:

Originally Posted by jcespada

Maybe I'm wrong, but I think you have to indicate the number of processors.
For example:

mpirun -np 8 pimpleFoam -parallel

It works like this for this particular system. I actually don't know why it's different (i usually do like you say), the sys admin told me to do like this.

It's not the first time I use this system and it always worked without neither np nor the hostfile

Pj. · April 8, 2015, 09:12

Solved decomposing in a different number of processors.

In my opinion this sounds like a bug, but I wait someone more experienced than me to say so.

Any opinion?

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error running simpleFoam in parallel	Yuby	OpenFOAM Running, Solving & CFD	14	October 7, 2021 05:38
Fluent 14.0 file not running in parallel mode in cluster	tejakalva	FLUENT	0	February 4, 2015 08:02
Running CFX parallel distributed Under linux system with loadleveler queuing system	ahmadbakri	CFX	1	December 21, 2014 05:19
Segmentation Fault	Shawn_A	OpenFOAM Running, Solving & CFD	6	October 31, 2011 15:38
Fluent parallel: segmentation fault?	hp	FLUENT	2	September 6, 2001 15:18

April 1, 2015, 09:32		#2
jcespada New Member Jesus Contreras Espada Join Date: Aug 2011 Location: Switzerland Posts: 12 Rep Power: 16	Maybe I'm wrong, but I think you have to indicate the number of processors. For example: mpirun -np 8 pimpleFoam -parallel

April 8, 2015, 09:12		#4
Pj. Member Luca Join Date: Mar 2013 Posts: 68 Rep Power: 14	Solved decomposing in a different number of processors. In my opinion this sounds like a bug, but I wait someone more experienced than me to say so. Any opinion?