Crushed with error "mpirun noticed that process rank 3 with PID 16913 on node"

lcraul · May 6, 2019, 12:50

Hello everyone,
I'm running a CFDEM (coupled by OpenFOAM and LIGGGHTS) case of particle movement in a wedge shape fracture. Unfortunately, I encountered a error when performing a mpi run. The case crushed at Time=0.0825 with error "mpirun noticed that process rank 1 with PID 37968 on node comp-ic-0015 exited on signal 11 (Segmentation fault)." This error is very strange because I run the similar case successfully before. However, when I change the massrate from 0.0135 to 0.003375, this error existed. In addition, I run another case with massrate 0.00675, the case crushed at Time=0.1023 too. The error is "mpirun noticed that process rank 3 with PID 16913 on node comp-ic-0015 exited on signal 6 (Aborted)." I searched the forums, and someone said it may be caused by the mpi. So I reduced the processor from 4 to 2. But it didn't work. Then some other said it may be caused by memory space. However, the massrate 0.0135 case requires a larger memory, but it didn't reported this error. I have no idea on this error, could anybody gives me some suggestions? Thank you very much. By the way, I use the latest version of the CFDEM.

Time = 0.0825

Courant Number mean: 0.00693989 max: 0.0134362

Coupling...
Starting up LIGGGHTS
Executing command: 'run 100 '
run 100
Setting up run at Fri May 3 19:32:54 2019

Memory usage per processor = 15.9705 Mbytes
Step Atoms KinEng 1 Volume
824901 5476 5.7715265e-05 8.262058e-06 8.991e-06
[1] #0 Foam::error:

rintStack(Foam::Ostream&)CFD Coupling established at step 825000
825000 5476 5.7768916e-05 8.2639657e-06 8.991e-06
insertion: proc 0 at 0 %
insertion: proc 0 at 10 %
insertion: proc 0 at 20 %
insertion: proc 0 at 30 %
insertion: proc 0 at 40 %
insertion: proc 0 at 50 %
insertion: proc 0 at 60 %
insertion: proc 0 at 70 %
insertion: proc 0 at 80 %
insertion: proc 0 at 90 %
INFO: Particle insertion ins2: inserted 220 particle templates (mass 1.117704e-05) at step 825001
- a total of 5698 particle templates (mass 2.894854e-04) inserted so far.
at ??:?
[1] #1 Foam::sigSegv::sigHandler(int) at ??:?
[1] #2 ? in "/lib/x86_64-linux-gnu/libpthread.so.0"
[1] #3 LAMMPS_NS::AtomVecSphere::copy(int, int, int) at ??:?
[1] #4 LAMMPS_NS::FixMassflowMesh:

re_exchange() at ??:?
[1] #5 LAMMPS_NS::Modify:

re_exchange() at ??:?
[1] #6 LAMMPS_NS::Verlet::run(int) at ??:?
[1] #7 LAMMPS_NS::Run::command(int, char**, long) at ??:?
[1] #8 void LAMMPS_NS::Input::command_creator(LAMMPS_NS::LAMMP S*, int, char**) at ??:?
[1] #9 LAMMPS_NS::Input::execute_command() at ??:?
[1] #10 LAMMPS_NS::Input:

ne(char const*) at ??:?
[1] #11 Foam::twoWayMPI::couple(int) const at ??:?
[1] #12 Foam::cfdemCloud::evolve(Foam::GeometricField&, Foam::GeometricField, Foam::fvPatchField, Foam::volMesh>&, Foam::GeometricField, Foam::fvPatchField, Foam::volMesh>&) at ??:?
[1] #13 ? at ??:?
[1] #14 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #15 ? at ??:?
[comp-ic-0015:37968] *** Process received signal ***
[comp-ic-0015:37968] Signal: Segmentation fault (11)
[comp-ic-0015:37968] Signal code: (-6)
[comp-ic-0015:37968] Failing at address: 0x430f00009450
[comp-ic-0015:37968] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f15b1c0e390]
[comp-ic-0015:37968] [ 1] /lib/x86_64-linux-gnu/libpthread.so.0(raise+0x29)[0x7f15b1c0e269]
[comp-ic-0015:37968] [ 2] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f15b1c0e390]
[comp-ic-0015:37968] [ 3] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7f15af2b01ad]
[comp-ic-0015:37968] [ 4] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7f15af26d836]
[comp-ic-0015:37968] [ 5] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7f15af1af46f]
[comp-ic-0015:37968] [ 6] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7f15af2c243f]
[comp-ic-0015:37968] [ 7] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7f15af2f99af]
[comp-ic-0015:37968] [ 8] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7f15aecd3a52]
[comp-ic-0015:37968] [ 9] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7f15aecd1353]
[comp-ic-0015:37968] [10] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7f15aecd2097]
[comp-ic-0015:37968] [11] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7f15b352021f]
[comp-ic-0015:37968] [12] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7f15b34a18b0]
[comp-ic-0015:37968] [13] cfdemSolverPiso[0x4261c7]
[comp-ic-0015:37968] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f15b1853830]
[comp-ic-0015:37968] [15] cfdemSolverPiso[0x42b009]
[comp-ic-0015:37968] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 37968 on node comp-ic-0015 exited on signal 11 (Segmentation fault).

Time = 0.1023

Courant Number mean: 0.00624899 max: 0.012053

Coupling...
Starting up LIGGGHTS
Executing command: 'run 100 '
run 100
Setting up run at Fri May 3 20:32:01 2019

Memory usage per processor = 15.9481 Mbytes
Step Atoms KinEng 1 Volume
1022901 13567 0.00014260139 5.2036481e-05 8.991e-06
CFD Coupling established at step 1023000
1023000 13567 0.00014263446 5.2099481e-05 8.991e-06
insertion: proc 0 at 0 %
insertion: proc 0 at 10 %
insertion: proc 0 at 20 %
insertion: proc 0 at 30 %
insertion: proc 0 at 40 %
insertion: proc 0 at 50 %
insertion: proc 0 at 60 %
insertion: proc 0 at 70 %
insertion: proc 0 at 80 %
insertion: proc 0 at 90 %
insertion: proc 0 at 100 %
INFO: Particle insertion ins2: inserted 438 particle templates (mass 2.225247e-05) at step 1023001
- a total of 14026 particle templates (mass 7.125872e-04) inserted so far.
[3] #0 Foam::error:

rintStack(Foam::Ostream&) at ??:?
[3] #1 Foam::sigSegv::sigHandler(int) at ??:?
[3] #2 ? in "/lib/x86_64-linux-gnu/libpthread.so.0"
[3] #3 LAMMPS_NS::AtomVecSphere::copy(int, int, int)*** Error in `cfdemSolverPiso': free(): invalid next size (normal): 0x000000000337a4e0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fe73c95b7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fe73c96437a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fe73c96853c]
/opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam22printSourceFileAndLineERN S_7OstreamERKNS_8fileNameEP7Dl_infoPv+0x321)[0x7fe73e0d6da1]
/opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error10printStackERNS_7Ost reamE+0x112)[0x7fe73e0d86f2]
/opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7sigSegv10sigHandlerEi+0x30 )[0x7fe73e0cb9c0]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7fe73a3611ad]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7fe73a31e836]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7fe73a26046f]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7fe73a37343f]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7fe73a3aa9af]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7fe739d84a52]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7fe739d82353]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7fe739d83097]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7fe73e5d121f]
/storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7fe73e5528b0]
cfdemSolverPiso[0x4261c7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe73c904830]
cfdemSolverPiso[0x42b009]
======= Memory map: ========
00400000-004a8000 r-xp 00000000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso
006a8000-006aa000 r--p 000a8000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso
006aa000-006ab000 rw-p 000aa000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso
010b7000-058b8000 rw-p 00000000 00:00 0 [heap]
7fe71f32c000-7fe72063f000 rw-p 00000000 00:00 0
7fe72063f000-7fe728640000 rw-s 00000000 fd:02 131139 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/shared_mem_pool.comp-ic-0015 (deleted)
7fe728640000-7fe728a41000 rw-s 00000000 fd:02 131146 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/2/vader_segment.comp-ic-0015.2
7fe728a41000-7fe728e42000 rw-s 00000000 fd:02 131135 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/1/vader_segment.comp-ic-0015.1
7fe728e42000-7fe729243000 rw-s 00000000 fd:02 131145 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/0/vader_segment.comp-ic-0015.0
7fe729243000-7fe729248000 r-xp 00000000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so
7fe729248000-7fe729447000 ---p 00005000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so
7fe729447000-7fe729448000 r--p 00004000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so
7fe729448000-7fe72944a000 rw-p 00005000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so
7fe72944a000-7fe729469000 r-xp 00000000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so
7fe729469000-7fe729668000 ---p 0001f000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so
7fe729668000-7fe729669000 r--p 0001e000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so
7fe729669000-7fe72966a000 rw-p 0001f000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so
7fe72966f000-7fe729670000 r-xp 00000000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so
7fe729670000-7fe72986f000 ---p 00001000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so
7fe72986f000-7fe729870000 r--p 00000000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so
7fe729870000-7fe729871000 rw-p 00001000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so
7fe729871000-7fe729873000 r-xp 00000000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so
7fe729873000-7fe729a72000 ---p 00002000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so
7fe729a72000-7fe729a73000 r--p 00001000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so
7fe729a73000-7fe729a74000 rw-p 00002000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so
7fe729a90000-7fe729aa5000 r-xp 00000000 07:00 59836 /usr/lib/openmpi/lib/openmpi/mca_osc_pt2pt.so[comp-ic-0015:16913] *** Process received signal ***
[comp-ic-0015:16913] Signal: Aborted (6)
[comp-ic-0015:16913] Signal code: (-6)
[comp-ic-0015:16913] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390]
[comp-ic-0015:16913] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fe73c919428]
[comp-ic-0015:16913] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fe73c91b02a]
[comp-ic-0015:16913] [ 3] /lib/x86_64-linux-gnu/libc.so.6(+0x777ea)[0x7fe73c95b7ea]
[comp-ic-0015:16913] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fe73c96437a]
[comp-ic-0015:16913] [ 5] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fe73c96853c]
[comp-ic-0015:16913] [ 6] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam22printSourceFileAndLineERN S_7OstreamERKNS_8fileNameEP7Dl_infoPv+0x321)[0x7fe73e0d6da1]
[comp-ic-0015:16913] [ 7] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error10printStackERNS_7Ost reamE+0x112)[0x7fe73e0d86f2]
[comp-ic-0015:16913] [ 8] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7sigSegv10sigHandlerEi+0x30 )[0x7fe73e0cb9c0]
[comp-ic-0015:16913] [ 9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390]
[comp-ic-0015:16913] [10] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7fe73a3611ad]
[comp-ic-0015:16913] [11] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7fe73a31e836]
[comp-ic-0015:16913] [12] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7fe73a26046f]
[comp-ic-0015:16913] [13] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7fe73a37343f]
[comp-ic-0015:16913] [14] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7fe73a3aa9af]
[comp-ic-0015:16913] [15] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7fe739d84a52]
[comp-ic-0015:16913] [16] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7fe739d82353]
[comp-ic-0015:16913] [17] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7fe739d83097]
[comp-ic-0015:16913] [18] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7fe73e5d121f]
[comp-ic-0015:16913] [19] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7fe73e5528b0]
[comp-ic-0015:16913] [20] cfdemSolverPiso[0x4261c7]
[comp-ic-0015:16913] [21] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe73c904830]
[comp-ic-0015:16913] [22] cfdemSolverPiso[0x42b009]
[comp-ic-0015:16913] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 3 with PID 16913 on node comp-ic-0015 exited on signal 6 (Aborted).

mehryaram · July 22, 2021, 21:34

Hi!

Can you fix this?

sadsid · December 10, 2021, 16:47

Quote:

Originally Posted by mehryaram

Hi!

Can you fix this?

did you find the solution?

atul1018 · November 17, 2023, 08:32

Hello community

Is there anyone who can help me fix this error. realloc(): invalid next size

I have installed the latest version of CFDEMCoupling_23.02 (couples LIGGGHTS-PFM with OF-6) in our university's Linux cluster. Compilation is done successfully without nay error.

Atm, I am running at the moment pure DEM simulation (LIGGGHTS) case in parallel wit 224 processor. The case starts successfully without any problem runs for a quite a while (1.5-1.6 sec) but all of a sudden (while insterting particles) it throws the error "realloc():invalid next size" and its getting killed. I am not able to understand what is this and how can I solve this issue. The error from output file looks like:

INFO: Particle insertion r_ins: inserted 133127 particle templates (mass 0.009986) at step 6000001
- a total of 798762 particle templates (mass 0.060041) inserted so far.
*** Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x0000000002c1dd90 ***
*** Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x00000000027d2ca0 ***
*** Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x0000000004307240 ***

================================================== =================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 140 PID 34722 RUNNING AT i22r04c02s01
= KILLED BY SIGNAL: 9 (Killed)
================================================== =================================

Please help if someone has faced this kind of behavior before. I could provide more information if required. I am in urgent and desperate need of help.

Best Regards
Atul

May 6, 2019, 12:50	Crushed with error "mpirun noticed that process rank 3 with PID 16913 on node"	#1
lcraul New Member Chong Lin Join Date: May 2019 Posts: 2 Rep Power: 0	Hello everyone, I'm running a CFDEM (coupled by OpenFOAM and LIGGGHTS) case of particle movement in a wedge shape fracture. Unfortunately, I encountered a error when performing a mpi run. The case crushed at Time=0.0825 with error "mpirun noticed that process rank 1 with PID 37968 on node comp-ic-0015 exited on signal 11 (Segmentation fault)." This error is very strange because I run the similar case successfully before. However, when I change the massrate from 0.0135 to 0.003375, this error existed. In addition, I run another case with massrate 0.00675, the case crushed at Time=0.1023 too. The error is "mpirun noticed that process rank 3 with PID 16913 on node comp-ic-0015 exited on signal 6 (Aborted)." I searched the forums, and someone said it may be caused by the mpi. So I reduced the processor from 4 to 2. But it didn't work. Then some other said it may be caused by memory space. However, the massrate 0.0135 case requires a larger memory, but it didn't reported this error. I have no idea on this error, could anybody gives me some suggestions? Thank you very much. By the way, I use the latest version of the CFDEM. Time = 0.0825 Courant Number mean: 0.00693989 max: 0.0134362 Coupling... Starting up LIGGGHTS Executing command: 'run 100 ' run 100 Setting up run at Fri May 3 19:32:54 2019 Memory usage per processor = 15.9705 Mbytes Step Atoms KinEng 1 Volume 824901 5476 5.7715265e-05 8.262058e-06 8.991e-06 [1] #0 Foam::error:rintStack(Foam::Ostream&)CFD Coupling established at step 825000 825000 5476 5.7768916e-05 8.2639657e-06 8.991e-06 insertion: proc 0 at 0 % insertion: proc 0 at 10 % insertion: proc 0 at 20 % insertion: proc 0 at 30 % insertion: proc 0 at 40 % insertion: proc 0 at 50 % insertion: proc 0 at 60 % insertion: proc 0 at 70 % insertion: proc 0 at 80 % insertion: proc 0 at 90 % INFO: Particle insertion ins2: inserted 220 particle templates (mass 1.117704e-05) at step 825001 - a total of 5698 particle templates (mass 2.894854e-04) inserted so far. at ??:? [1] #1 Foam::sigSegv::sigHandler(int) at ??:? [1] #2 ? in "/lib/x86_64-linux-gnu/libpthread.so.0" [1] #3 LAMMPS_NS::AtomVecSphere::copy(int, int, int) at ??:? [1] #4 LAMMPS_NS::FixMassflowMesh:re_exchange() at ??:? [1] #5 LAMMPS_NS::Modify:re_exchange() at ??:? [1] #6 LAMMPS_NS::Verlet::run(int) at ??:? [1] #7 LAMMPS_NS::Run::command(int, char*, long) at ??:? [1] #8 void LAMMPS_NS::Input::command_creator(LAMMPS_NS::LAMMP S, int, char*) at ??:? [1] #9 LAMMPS_NS::Input::execute_command() at ??:? [1] #10 LAMMPS_NS::Input:ne(char const) at ??:? [1] #11 Foam::twoWayMPI::couple(int) const at ??:? [1] #12 Foam::cfdemCloud::evolve(Foam::GeometricField&, Foam::GeometricField, Foam::fvPatchField, Foam::volMesh>&, Foam::GeometricField, Foam::fvPatchField, Foam::volMesh>&) at ??:? [1] #13 ? at ??:? [1] #14 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6" [1] #15 ? at ??:? [comp-ic-0015:37968] * Process received signal * [comp-ic-0015:37968] Signal: Segmentation fault (11) [comp-ic-0015:37968] Signal code: (-6) [comp-ic-0015:37968] Failing at address: 0x430f00009450 [comp-ic-0015:37968] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f15b1c0e390] [comp-ic-0015:37968] [ 1] /lib/x86_64-linux-gnu/libpthread.so.0(raise+0x29)[0x7f15b1c0e269] [comp-ic-0015:37968] [ 2] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f15b1c0e390] [comp-ic-0015:37968] [ 3] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7f15af2b01ad] [comp-ic-0015:37968] [ 4] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7f15af26d836] [comp-ic-0015:37968] [ 5] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7f15af1af46f] [comp-ic-0015:37968] [ 6] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7f15af2c243f] [comp-ic-0015:37968] [ 7] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7f15af2f99af] [comp-ic-0015:37968] [ 8] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7f15aecd3a52] [comp-ic-0015:37968] [ 9] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7f15aecd1353] [comp-ic-0015:37968] [10] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7f15aecd2097] [comp-ic-0015:37968] [11] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7f15b352021f] [comp-ic-0015:37968] [12] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7f15b34a18b0] [comp-ic-0015:37968] [13] cfdemSolverPiso[0x4261c7] [comp-ic-0015:37968] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f15b1853830] [comp-ic-0015:37968] [15] cfdemSolverPiso[0x42b009] [comp-ic-0015:37968] * End of error message * -------------------------------------------------------------------------- mpirun noticed that process rank 1 with PID 37968 on node comp-ic-0015 exited on signal 11 (Segmentation fault). Time = 0.1023 Courant Number mean: 0.00624899 max: 0.012053 Coupling... Starting up LIGGGHTS Executing command: 'run 100 ' run 100 Setting up run at Fri May 3 20:32:01 2019 Memory usage per processor = 15.9481 Mbytes Step Atoms KinEng 1 Volume 1022901 13567 0.00014260139 5.2036481e-05 8.991e-06 CFD Coupling established at step 1023000 1023000 13567 0.00014263446 5.2099481e-05 8.991e-06 insertion: proc 0 at 0 % insertion: proc 0 at 10 % insertion: proc 0 at 20 % insertion: proc 0 at 30 % insertion: proc 0 at 40 % insertion: proc 0 at 50 % insertion: proc 0 at 60 % insertion: proc 0 at 70 % insertion: proc 0 at 80 % insertion: proc 0 at 90 % insertion: proc 0 at 100 % INFO: Particle insertion ins2: inserted 438 particle templates (mass 2.225247e-05) at step 1023001 - a total of 14026 particle templates (mass 7.125872e-04) inserted so far. [3] #0 Foam::error:rintStack(Foam::Ostream&) at ??:? [3] #1 Foam::sigSegv::sigHandler(int) at ??:? [3] #2 ? in "/lib/x86_64-linux-gnu/libpthread.so.0" [3] #3 LAMMPS_NS::AtomVecSphere::copy(int, int, int)* Error in `cfdemSolverPiso': free(): invalid next size (normal): 0x000000000337a4e0 * ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fe73c95b7e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fe73c96437a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fe73c96853c] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam22printSourceFileAndLineERN S_7OstreamERKNS_8fileNameEP7Dl_infoPv+0x321)[0x7fe73e0d6da1] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error10printStackERNS_7Ost reamE+0x112)[0x7fe73e0d86f2] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7sigSegv10sigHandlerEi+0x30 )[0x7fe73e0cb9c0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7fe73a3611ad] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7fe73a31e836] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7fe73a26046f] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7fe73a37343f] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7fe73a3aa9af] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7fe739d84a52] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7fe739d82353] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7fe739d83097] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7fe73e5d121f] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7fe73e5528b0] cfdemSolverPiso[0x4261c7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe73c904830] cfdemSolverPiso[0x42b009] ======= Memory map: ======== 00400000-004a8000 r-xp 00000000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso 006a8000-006aa000 r--p 000a8000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso 006aa000-006ab000 rw-p 000aa000 00:24 169066347 /storage/work/c/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/bin/cfdemSolverPiso 010b7000-058b8000 rw-p 00000000 00:00 0 [heap] 7fe71f32c000-7fe72063f000 rw-p 00000000 00:00 0 7fe72063f000-7fe728640000 rw-s 00000000 fd:02 131139 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/shared_mem_pool.comp-ic-0015 (deleted) 7fe728640000-7fe728a41000 rw-s 00000000 fd:02 131146 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/2/vader_segment.comp-ic-0015.2 7fe728a41000-7fe728e42000 rw-s 00000000 fd:02 131135 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/1/vader_segment.comp-ic-0015.1 7fe728e42000-7fe729243000 rw-s 00000000 fd:02 131145 /tmp/openmpi-sessions-cxl1166@comp-ic-0015_0/9521/1/0/vader_segment.comp-ic-0015.0 7fe729243000-7fe729248000 r-xp 00000000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so 7fe729248000-7fe729447000 ---p 00005000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so 7fe729447000-7fe729448000 r--p 00004000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so 7fe729448000-7fe72944a000 rw-p 00005000 07:00 59837 /usr/lib/openmpi/lib/openmpi/mca_osc_sm.so 7fe72944a000-7fe729469000 r-xp 00000000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so 7fe729469000-7fe729668000 ---p 0001f000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so 7fe729668000-7fe729669000 r--p 0001e000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so 7fe729669000-7fe72966a000 rw-p 0001f000 07:00 59795 /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so 7fe72966f000-7fe729670000 r-xp 00000000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so 7fe729670000-7fe72986f000 ---p 00001000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so 7fe72986f000-7fe729870000 r--p 00000000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so 7fe729870000-7fe729871000 rw-p 00001000 07:00 59886 /usr/lib/openmpi/lib/openmpi/mca_topo_basic.so 7fe729871000-7fe729873000 r-xp 00000000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so 7fe729873000-7fe729a72000 ---p 00002000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so 7fe729a72000-7fe729a73000 r--p 00001000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so 7fe729a73000-7fe729a74000 rw-p 00002000 07:00 59793 /usr/lib/openmpi/lib/openmpi/mca_coll_self.so 7fe729a90000-7fe729aa5000 r-xp 00000000 07:00 59836 /usr/lib/openmpi/lib/openmpi/mca_osc_pt2pt.so[comp-ic-0015:16913] * Process received signal * [comp-ic-0015:16913] Signal: Aborted (6) [comp-ic-0015:16913] Signal code: (-6) [comp-ic-0015:16913] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390] [comp-ic-0015:16913] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fe73c919428] [comp-ic-0015:16913] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fe73c91b02a] [comp-ic-0015:16913] [ 3] /lib/x86_64-linux-gnu/libc.so.6(+0x777ea)[0x7fe73c95b7ea] [comp-ic-0015:16913] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fe73c96437a] [comp-ic-0015:16913] [ 5] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fe73c96853c] [comp-ic-0015:16913] [ 6] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam22printSourceFileAndLineERN S_7OstreamERKNS_8fileNameEP7Dl_infoPv+0x321)[0x7fe73e0d6da1] [comp-ic-0015:16913] [ 7] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error10printStackERNS_7Ost reamE+0x112)[0x7fe73e0d86f2] [comp-ic-0015:16913] [ 8] /opt/openfoam5/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7sigSegv10sigHandlerEi+0x30 )[0x7fe73e0cb9c0] [comp-ic-0015:16913] [ 9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe73ccbf390] [comp-ic-0015:16913] [10] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS13AtomVecSphere4copyEi ii+0x5d)[0x7fe73a3611ad] [comp-ic-0015:16913] [11] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS15FixMassflowMesh12pre _exchangeEv+0x126)[0x7fe73a31e836] [comp-ic-0015:16913] [12] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Modify12pre_exchangeE v+0x4f)[0x7fe73a26046f] [comp-ic-0015:16913] [13] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS6Verlet3runEi+0x53f)[0x7fe73a37343f] [comp-ic-0015:16913] [14] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS3Run7commandEiPPcl+0x8 1f)[0x7fe73a3aa9af] [comp-ic-0015:16913] [15] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15command_creato rINS_3RunEEEvPNS_6LAMMPSEiPPc+0x42)[0x7fe739d84a52] [comp-ic-0015:16913] [16] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input15execute_comman dEv+0x913)[0x7fe739d82353] [comp-ic-0015:16913] [17] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblmp_auto.so(_ZN9LAMMPS_NS5Input3oneEPKc+0x87)[0x7fe739d83097] [comp-ic-0015:16913] [18] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZNK4Foam9twoWayMPI6coupleEi+0x143f)[0x7fe73e5d121f] [comp-ic-0015:16913] [19] /storage/work/cxl1166/sw/CFDEM/CFDEMcoupling-PUBLIC-5.0/platforms/linux64GccDPInt32Opt/lib/liblagrangianCFDEM-PUBLIC-5.0.so(_ZN4Foam10cfdemCloud6evolveERNS_14Geometric FieldIdNS_12fvPatchFieldENS_7volMeshEEERNS1_INS_6V ectorIdEES2_S3_EES9_+0xe50)[0x7fe73e5528b0] [comp-ic-0015:16913] [20] cfdemSolverPiso[0x4261c7] [comp-ic-0015:16913] [21] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe73c904830] [comp-ic-0015:16913] [22] cfdemSolverPiso[0x42b009] [comp-ic-0015:16913] * End of error message * -------------------------------------------------------------------------- mpirun noticed that process rank 3 with PID 16913 on node comp-ic-0015 exited on signal 6 (Aborted).

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[snappyHexMesh] SHM mpirun rank 4, PID 0 exited on signal 9 (kill)	mwaqas	OpenFOAM Meshing & Mesh Conversion	1	February 29, 2024 09:25
process rank 12 with PID 25320 on node exited on signal 9 (killed).	Nitisha	CONVERGE	6	September 15, 2018 00:31
The fl process could not be started because of UDF	majid_kamyab	Fluent UDF and Scheme Programming	6	December 15, 2015 08:42
Linux Batch - Star-CCM+ .. running Sims in batchmode	eRzBeNgEl	STAR-CCM+	3	August 29, 2011 04:43
MPI_Recv process in local group is dead rank 44 MPI_COMM_WORLD	sampaio	OpenFOAM Running, Solving & CFD	0	January 9, 2006 09:43

July 22, 2021, 21:34		#2
mehryaram New Member mehryar Join Date: Mar 2019 Posts: 2 Rep Power: 0	Hi! Can you fix this?

November 17, 2023, 08:32	same error in pure LIGGGHTS case	#4
atul1018 Senior Member Join Date: Jun 2020 Posts: 100 Rep Power: 5	Hello community Is there anyone who can help me fix this error. realloc(): invalid next size I have installed the latest version of CFDEMCoupling_23.02 (couples LIGGGHTS-PFM with OF-6) in our university's Linux cluster. Compilation is done successfully without nay error. Atm, I am running at the moment pure DEM simulation (LIGGGHTS) case in parallel wit 224 processor. The case starts successfully without any problem runs for a quite a while (1.5-1.6 sec) but all of a sudden (while insterting particles) it throws the error "realloc():invalid next size" and its getting killed. I am not able to understand what is this and how can I solve this issue. The error from output file looks like: INFO: Particle insertion r_ins: inserted 133127 particle templates (mass 0.009986) at step 6000001 - a total of 798762 particle templates (mass 0.060041) inserted so far. * Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x0000000002c1dd90 * * Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x00000000027d2ca0 * * Error in `/dss/dsshome1/lxc03/ge72xes2/CFDEM/LIGGGHTS/src-build/liggghts': realloc(): invalid next size: 0x0000000004307240 * ================================================== ================================= = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = RANK 140 PID 34722 RUNNING AT i22r04c02s01 = KILLED BY SIGNAL: 9 (Killed) ================================================== ================================= Please help if someone has faced this kind of behavior before. I could provide more information if required. I am in urgent and desperate need of help. Best Regards Atul