CFD Online Discussion Forums - MPI Aborts when using reactingFoam with cyclic BC in parallelization

Hi, all

I am simulating a section of a combustor with reactingFoam and cyclic BC for the side boundaries of a 60-degree section.

Starting the solver with mpirun, I got a error message which is transcripted below. However, without parallelization it runs fine.

The cluster used runs on openSUSE 11.3 32bits, with tons of memory (im not sure but maybe 16Gb per node) and OF is the latest OF1.7.x 32bits from the rpm.

I want to note that I tried this simulation with symmetryPlane BC instead of cyclic and it worked fine with parallelization. Also peculiar was that I ran the exact same sim on my computer (macbook pro core 2 duo 4GB 64 bits, OF 1.7.1 64bits) with and without parallelization and it worked fine.

Any thoughts on this?

Regards

Jose

jose@f22:~/CFD/OpenFOAM/jose-1.7.1/run/reactingFoamChemkinFcyclic> mpirun --hostfile machines.txt -np 24 reactingFoamIgnition -parallel
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 1.7.1 |
| \\ / A nd | Web: www.OpenFOAM.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 1.7.1-03e7e056c215
Exec : reactingFoamIgnition -parallel
Date : Feb 12 2011
Time : 19:32:38
Host : f15
PID : 32739
Case : /home/jose/CFD/OpenFOAM/jose-1.7.1/run/reactingFoamChemkinFcyclic
nProcs : 24
Slaves :
23
(
f15.32740
f15.32741
f15.32742
f15.32743
f15.32744
f15.32745
f15.32746
f22.27085
f22.27086
f22.27087
f22.27088
f22.27089
f22.27090
f22.27091
f22.27092
f14.2188
f14.2189
f14.2190
f14.2191
f14.2192
f14.2193
f14.2194
f14.2195
)

Pstream initialized with:
floatTransfer : 0
nProcsSimpleSum : 0
commsType : nonBlocking
SigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Reading chemistry properties

Reading combustion properties

[4] Found ignition cells:
[4] 1(22)
[1] Found ignition cells:
[1] 1(25)
[2] Found ignition cells:
[2] 1(46)
[3] Found ignition cells:
[3] 1(78)

Ignition switched off

Reading g

Reading thermophysicalProperties
Selecting psiChemistryModel ODEChemistryModel<gasThermoPhysics>
[5] Found ignition cells:
[5] 1(40)
Selecting thermodynamics package hsPsiMixtureThermo<reactingMixture<gasThermoPhysic s>>
Selecting chemistryReader chemkinReader
*** glibc detected *** reactingFoamIgnition: free(): invalid next size (fast): 0x08264098 ***
======= Backtrace: =========
/lib/libc.so.6(+0x6dffb)[0xb5bedffb]
/lib/libc.so.6(+0x6ed7b)[0xb5beed7b]
/lib/libc.so.6(cfree+0x6d)[0xb5bf2add]
/usr/lib/libstdc++.so.6(_ZdlPv+0x1f)[0xb5de598f]
/usr/lib/libstdc++.so.6(_ZdaPv+0x1b)[0xb5de59eb]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libfiniteVolume.so(_ZN4Foam12nearWallDist5doAllEv+ 0x2c9)[0xb65f9289]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libfiniteVolume.so(_ZN4Foam12nearWallDistC1ERKNS_6 fvMeshE+0x5ac)[0xb65f9abc]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleRASModels.so(_ZN4Foam12compressible 8RASModelC2ERKNS_4wordERKNS_14GeometricFieldIdNS_1 2fvPatchFieldENS_7volMeshEEERKNS5_INS_6VectorIdEES 6_S7_EERKNS5_IdNS_13fvsPatchFieldENS_11surfaceMesh EEERKNS_11basicThermoE+0x896)[0xb76040a6]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleRASModels.so(_ZN4Foam12compressible 9RASModels8kEpsilonC2ERKNS_14GeometricFieldIdNS_12 fvPatchFieldENS_7volMeshEEERKNS3_INS_6VectorIdEES4 _S5_EERKNS3_IdNS_13fvsPatchFieldENS_11surfaceMeshE EERKNS_11basicThermoE+0x48)[0xb7637d08]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleRASModels.so(_ZN4Foam12compressible 8RASModel31adddictionaryConstructorToTableINS0_9RA SModels8kEpsilonEE3NewERKNS_14GeometricFieldIdNS_1 2fvPatchFieldENS_7volMeshEEERKNS6_INS_6VectorIdEES 7_S8_EERKNS6_IdNS_13fvsPatchFieldENS_11surfaceMesh EEERKNS_11basicThermoE+0x4f)[0xb76566cf]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleRASModels.so(_ZN4Foam12compressible 8RASModel3NewERKNS_14GeometricFieldIdNS_12fvPatchF ieldENS_7volMeshEEERKNS2_INS_6VectorIdEES3_S4_EERK NS2_IdNS_13fvsPatchFieldENS_11surfaceMeshEEERKNS_1 1basicThermoE+0x226)[0xb7605b66]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleRASModels.so(_ZN4Foam12compressible 15turbulenceModel36addturbulenceModelConstructorTo TableINS0_8RASModelEE18NewturbulenceModelERKNS_14G eometricFieldIdNS_12fvPatchFieldENS_7volMeshEEERKN S5_INS_6VectorIdEES6_S7_EERKNS5_IdNS_13fvsPatchFie ldENS_11surfaceMeshEEERKNS_11basicThermoE+0x41)[0xb760b991]
/home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/lib/linuxGccDPOpt/libcompressibleTurbulenceModel.so(_ZN4Foam12compre ssible15turbulenceModel3NewERKNS_14GeometricFieldI dNS_12fvPatchFieldENS_7volMeshEEERKNS2_INS_6Vector IdEES3_S4_EERKNS2_IdNS_13fvsPatchFieldENS_11surfac eMeshEEERKNS_11basicThermoE+0x215)[0xb77cdce5]
reactingFoamIgnition[0x8060d66]
/lib/libc.so.6(__libc_start_main+0xfe)[0xb5b96c0e]
reactingFoamIgnition[0x805da61]
======= Memory map: ========
08048000-080b6000 r-xp 00000000 08:02 21626887 /home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/applications/bin/linuxGccDPOpt/reactingFoamIgnition
080b6000-080b7000 r--p 0006e000 08:02 21626887 /home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/applications/bin/linuxGccDPOpt/reactingFoamIgnition
080b7000-080b8000 rw-p 0006f000 08:02 21626887 /home/jose/CFD/OpenFOAM/OpenFOAM-1.7.1/applications/bin/linuxGccDPOpt/reactingFoamIgnition
080b8000-08291000 rw-p 00000000 00:00 0 [heap]
af500000-af521000 rw-p 00000000 00:00 0
af521000-af600000 ---p 00000000 00:00 0
af6a3000-b09b6000 rw-p 00000000 00:00 0
b09b6000-b49b8000 rw-s 00000000 08:01 1445562 /tmp/openmpi-sessions-jose@f22_0/46420/1/shared_mem_pool.f22
b49b8000-b4b3a000 rw-p 00000000 00:00 0
b4b3a000-b4b45000 r-xp 00000000 08:02 24645265 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_rdma.so
b4b45000-b4b46000 r--p 0000a000 08:02 24645265 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_rdma.so
b4b46000-b4b47000 rw-p 0000b000 08:02 24645265 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_rdma.so
b4b47000-b4b50000 r-xp 00000000 08:02 24645314 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_pt2pt.so
b4b50000-b4b51000 r--p 00008000 08:02 24645314 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_pt2pt.so
b4b51000-b4b52000 rw-p 00009000 08:02 24645314 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_osc_pt2pt.so
b4b52000-b4b69000 r-xp 00000000 08:02 24645296 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_tuned.so
b4b69000-b4b6a000 r--p 00016000 08:02 24645296 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_tuned.so
b4b6a000-b4b6b000 rw-p 00017000 08:02 24645296 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_tuned.so
b4b6b000-b4b6d000 r-xp 00000000 08:02 24645333 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sync.so
b4b6d000-b4b6e000 r--p 00001000 08:02 24645333 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sync.so
b4b6e000-b4b6f000 rw-p 00002000 08:02 24645333 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sync.so
b4b6f000-b4b74000 r-xp 00000000 08:02 24645254 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sm.so
b4b74000-b4b75000 r--p 00004000 08:02 24645254 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sm.so
b4b75000-b4b76000 rw-p 00005000 08:02 24645254 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_sm.so
b4b76000-b4b77000 r-xp 00000000 08:02 24645269 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_self.so
b4b77000-b4b78000 ---p 00001000 08:02 24645269 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_self.so
b4b78000-b4b79000 r--p 00001000 08:02 24645269 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_self.so
b4b79000-b4b7a000 rw-p 00002000 08:02 24645269 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_self.so
b4b7a000-b4b7c000 r-xp 00000000 08:02 24645285 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_inter.so
b4b7c000-b4b7d000 ---p 00002000 08:02 24645285 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_inter.so
b4b7d000-b4b7e000 r--p 00002000 08:02 24645285 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_inter.so
b4b7e000-b4b7f000 rw-p 00003000 08:02 24645285 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_inter.so
b4b7f000-b4b83000 r-xp 00000000 08:02 24645300 /home/jose/CFD/OpenFOAM/ThirdParty-1.7.1/platforms/linuxGcc/openmpi-1.4.1/lib/openmpi/mca_coll_hierarch.so[f22:27089] *** Process received signal ***
[f22:27089] Signal: Aborted (6)
[f22:27089] Signal code: (-6)
[f22:27089] [ 0] [0xffffe40c]
[f22:27089] [ 1] /lib/libc.so.6(+0x6dffb) [0xb5bedffb]
[f22:27089] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 12 with PID 27089 on node f22 exited on signal 6 (Aborted).
--------------------------------------------------------------------------
[1]+ Terminated reactingFoamIgnition > log
jose@f22:~/CFD/OpenFOAM/jose-1.7.1/run/reactingFoamChemkinFcyclic> ^C