CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

parallel calculation issues

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   December 20, 2018, 09:51
Question parallel calculation issues
  #1
New Member
 
Join Date: Oct 2018
Posts: 9
Rep Power: 6
tjuAeron is on a distinguished road
Hello everyone,

I'm very new to linux and OF, and currently I'm going with a very frustrating problem.

I've successfully set a case and run it to 5000 timesteps in serial without any mistake. So I'm now trying to run it in parallel. But it aborted at first timestep with the famous "float point exception". The log is as follows:

Code:
Time = 1

BiCGStab:  Solving for Ux, Initial residual = 1, Final residual = 0.00262781, No Iterations 1
BiCGStab:  Solving for Uy, Initial residual = 1, Final residual = 0.000321715, No Iterations 1
BiCGStab:  Solving for Uz, Initial residual = 1, Final residual = 0.00339929, No Iterations 2
BiCGStab:  Solving for p, Initial residual = 1, Final residual = 0.0440403, No Iterations 1
time step continuity errors : sum local = 0.0192303, global = -0.007747, cumulative = -0.007747
BiCGStab:  Solving for p, Initial residual = 0.861166, Final residual = 0.0152853, No Iterations 2
time step continuity errors : sum local = 0.068281, global = -0.0116883, cumulative = -0.0194353
BiCGStab:  Solving for i, Initial residual = 1, Final residual = 0.000584795, No Iterations 2
ExecutionTime = 3.99 s  ClockTime = 4 s

Cyclic GGI pair (perBlade1, perBlade2)
Area: 0.00523772 0.00523772 Diff = -5.96635e-13 or 1.13911e-08 % 
Flux: 155599 581.131 Diff = -155996 or 100.255 %
Cyclic GGI pair (perUp1, perUp2)
Area: 0.00967552 0.00967552 Diff = 3.23594e-13 or 3.34446e-09 % 
Flux: 46.6035 0.00235069 Diff = -46.6009 or 99.9944 %
Cyclic GGI pair (perDown1, perDown2)
Area: 0.00364872 0.00364872 Diff = -2.67976e-13 or 7.34439e-09 % 
Flux: 48.5907 124200 Diff = -123929 or 255047 %
Flux divergence min = 1.29895e-07 max = 8.40826e+06 average: 4.40663e+08
[admin:24845] *** Process received signal ***
[admin:24847] *** Process received signal ***
[admin:24847] Signal: Floating point exception (8)
[admin:24847] Signal code:  (-6)
[admin:24847] Failing at address: 0x4240000610f
[admin:24849] *** Process received signal ***
[admin:24849] Signal: Floating point exception (8)
[admin:24849] Signal code:  (-6)
[admin:24849] Failing at address: 0x42400006111
[admin:24845] Signal: Floating point exception (8)
[admin:24845] Signal code:  (-6)
[admin:24845] Failing at address: 0x4240000610d
[admin:24849] [ 0] [admin:24847] /lib64/libc.so.6(+0x35670)[0x2acb53b6a670]
[admin:24849] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2acb53b6a5f7]
[admin:24849] [ 2] /lib64/libc.so.6(+0x35670)[0x2acb53b6a670]
[admin:24849] [ 3] [admin:24850] *** Process received signal ***
[ 0] /lib64/libc.so.6(+0x35670)[0x2b0a72aea670]
[admin:24847] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2b0a72aea5f7]
[admin:24847] [ 2] [admin:24850] Signal: Floating point exception (8)
[admin:24850] Signal code:  (-6)
[admin:24850] Failing at address: 0x42400006112
[admin:24846] *** Process received signal ***
[admin:24845] [ 0] /lib64/libc.so.6(+0x35670)[0x2b0a72aea670]
[admin:24847] [ 3] [admin:24846] Signal: Floating point exception (8)
[admin:24846] Signal code:  (-6)
[admin:24846] Failing at address: 0x4240000610e
/lib64/libc.so.6(+0x35670)[0x2ac434a32670]
[admin:24845] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2ac434a325f7]
[admin:24845] [ 2] /lib64/libc.so.6(+0x35670)[0x2ac434a32670]
[admin:24845] [ 3] [admin:24846] [ 0] [admin:24851] *** Process received signal ***
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2acb527d4d48]
[admin:24849] [ 4] /lib64/libc.so.6(+0x35670)[0x2b9a99132670]
[admin:24846] [ 1] [admin:24850] [ 0] /lib64/libc.so.6(gsignal+0x37)[0x2b9a991325f7]
[admin:24846] [ 2] /lib64/libc.so.6(+0x35670)[0x2ad07bffc670]
[admin:24850] [ 1] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2b0a71754d48]
[admin:24847] [ 4] /lib64/libc.so.6(+0x35670)[0x2b9a99132670]
[admin:24846] [ 3] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2acb51c73a98]
[admin:24849] [ 5] /lib64/libc.so.6(gsignal+0x37)[0x2ad07bffc5f7]
[admin:24850] [ 2] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2acb6fb4ca93]
[admin:24849] [ 6] /lib64/libc.so.6(+0x35670)[0x2ad07bffc670]
[admin:24850] [ 3] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2b0a70bf3a98]
[admin:24847] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2b0a8ea70a93]
[admin:24847] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2ac43369cd48]
[admin:24845] [ 4] [admin:24851] Signal: Floating point exception (8)
[admin:24851] Signal code:  (-6)
[admin:24851] Failing at address: 0x42400006113
[admin:24851] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2acb52640f89]
[admin:24849] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2b0a715c0f89]
[admin:24847] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2b9a97d9cd48]
[admin:24846] [ 4] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2ad07ac66d48]
[admin:24850] [ 4] [ 0] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2b9a9723ba98]
[admin:24846] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2b9ab5030a93]
[admin:24846] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2ad07a105a98]
/lib64/libc.so.6(+0x35670)[0x2b0d70a5d670]
[admin:24851] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2b0d70a5d5f7]
[admin:24851] [ 2] /lib64/libc.so.6(+0x35670)[0x2b0d70a5d670]
[admin:24851] [ 3] [admin:24850] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2b0a715cdbda]
[admin:24847] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2acb5264dbda]
[admin:24849] [ 8] [admin:24848] *** Process received signal ***
[admin:24848] Signal: Floating point exception (8)
[admin:24848] Signal code:  (-6)
[admin:24848] Failing at address: 0x42400006110
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2b9a97c08f89]
[admin:24846] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2ad097f22a93]
[admin:24850] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2b0d6f6c7d48]
[admin:24848] [ 0] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2b0a715c5c64]
[admin:24847] [ 9] /lib64/libc.so.6(+0x35670)[0x2b221dbe8670]
[admin:24848] [ 1] steadyCompressibleMRFFoam[0x421670]
[admin:24847] [10] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2acb52645c64]
[admin:24849] [ 9] /lib64/libc.so.6(gsignal+0x37)[0x2b221dbe85f7]
[admin:24848] [ 2] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b0a72ad6b15]
[admin:24847] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24847] *** End of error message ***
/lib64/libc.so.6(+0x35670)[0x2b221dbe8670]
[admin:24848] [ 3] steadyCompressibleMRFFoam[0x421670]
[admin:24849] [10] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2acb53b56b15]
[admin:24849] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24849] *** End of error message ***
[admin:24851] [ 4] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2b0d6eb66a98]
[admin:24851] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2b0d8c9aca93]
[admin:24851] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2ad07aad2f89]
[admin:24850] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2b9a97c15bda]
[admin:24846] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4sqrtERNS_5FieldIdEERKNS_5UListIdEE+0x28)[0x2b221c852d48]
[admin:24848] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2b0d6f533f89]
[admin:24851] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2ad07aadfbda]
[admin:24850] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2b9a97c0dc64]
[admin:24846] [ 9] steadyCompressibleMRFFoam[0x421670]
[admin:24846] [10] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b9a9911eb15]
[admin:24846] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24846] *** End of error message ***
[ 4] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0x144)[0x2b221bcf1ae4]
[admin:24848] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2b2239b45a93]
[admin:24848] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2b0d6f540bda]
[admin:24851] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2ad07aad7c64]
[admin:24850] [ 9] steadyCompressibleMRFFoam[0x421670]
[admin:24850] [10] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2ad07bfe8b15]
[admin:24850] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24850] *** End of error message ***
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2b221c6bef89]
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2b0d6f538c64]
[admin:24851] [ 9] steadyCompressibleMRFFoam[0x421670]
[admin:24851] [10] [admin:24848] [ 7] Mach number min = 0.472644 max = 0.522382
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2b0d70a49b15]
[admin:24851] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24851] *** End of error message ***
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libcompressibleRASModels.so(_ZN4Foam4sqrtINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS8_+0xf8)[0x2ac432b3ba98]
[admin:24845] [ 5] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libutilityFunctionObjects.so(_ZN4Foam10MachNumber7executeEv+0x223)[0x2ac450999a93]
[admin:24845] [ 6] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2b221c6cbbda]
[admin:24848] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam18functionObjectList7executeEv+0x139)[0x2ac433508f89]
[admin:24845] [ 7] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2b221c6c3c64]
[admin:24848] [ 9] steadyCompressibleMRFFoam[0x421670]
[admin:24848] [10] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b221dbd4b15]
[admin:24848] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24848] *** End of error message ***
/public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4TimeppEv+0x7a)[0x2ac433515bda]
[admin:24845] [ 8] /public/home/linjw1/foam/foam-extend-4.0/lib/linux64GccDPOpt/libfoam.so(_ZN4Foam4Time4loopEv+0x24)[0x2ac43350dc64]
[admin:24845] [ 9] steadyCompressibleMRFFoam[0x421670]
[admin:24845] [10] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2ac434a1eb15]
[admin:24845] [11] steadyCompressibleMRFFoam[0x4241f9]
[admin:24845] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 0 on node admin exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
And it is my decomposeParDict:
Code:
numberOfSubdomains 8;

method    patchConstrained;

// All GGI interfaces should be listed here:
globalFaceZones
(
    perBlade1Zone
    perBlade2Zone
    perUp1Zone
    perUp2Zone
    perDown1Zone
    perDown2Zone
);

patchConstrainedCoeffs
{
    method            scotch;
    numberOfSubdomains    8;
    // You can force the faces of a patch to be on the same processor
    // this is currently needed for mixingPlane
    patchConstraints
    (

    );
}

distributed     no;
I considered the incompatibility of foam-extend-4.0 with system openmpi2.0.1. So I try to compile openmpi1.8.8 in Thirdparty. But when I try mpirun, the terminal told me there is something wrong with MCA. The log is as follows:

Code:
[admin:21477] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_posix" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
[admin:21475] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_posix" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
[admin:21477] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_mmap" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
[admin:21475] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_mmap" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
[admin:21477] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_sysv" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
[admin:21475] mca_base_component_repository_open: shmem "/public/home/linjw1/foam/foam-extend-4.0/ThirdParty/packages/openmpi-1.8.8/platforms/linux64GccDPOpt/lib/openmpi/mca_shmem_sysv" uses an MCA interface that is not recognized (component MCA v2.0.0 != supported MCA v2.1.0) -- ignored
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
I'm very confused and don't how to solve it. Could anybody help me?
tjuAeron is offline   Reply With Quote

Reply

Tags
foam-extend 4.0, parallel calculation

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
MPI error in parallel application usv001 OpenFOAM Programming & Development 2 September 14, 2017 12:30
chtMultiRegionSimpleFoam: crash on parallel run student666 OpenFOAM Running, Solving & CFD 3 April 20, 2017 12:05
simpleFoam parallel AndrewMortimer OpenFOAM Running, Solving & CFD 12 August 7, 2015 19:45
Problem with parallel calculation papteo ANSYS 1 January 15, 2015 11:45
Coal combustion, big node, slow calculation, parallel Whitebear CFX 1 April 8, 2014 04:27


All times are GMT -4. The time now is 16:57.