| jploz |
September 30, 2009 13:29 |
Moving mesh crashes in parallel (1.5-dev)
Hi Foamers,
I've created a fvDynamicMesh class for moving the mesh based on flow forces in 1.5-dev. When running the case on a single processor everything works as expected. However, if I run the case in parallel it does not work.
For testing purposes the points are set to the (fixed) original coordinates - but i cannot move the points in parallel.
Does not work means:
if I use tetFaceDecomposition the simulation runs (i.e. it does not crash) but never finishes the fvMesh::movePoints step-the processors run with 100% but nothing happens for a really long time (strange)
if I use fv based motion the first mesh update is done but the simulation crashes when the fvMesh::movePoints step is reached for the second time.
Here is the traceback:
Code:
Getting new points
PCG: Solving for cellMotionUx, Initial residual = 0, Final residual = 0, No Iterations 0
PCG: Solving for cellMotionUy, Initial residual = 0, Final residual = 0, No Iterations 0
PCG: Solving for cellMotionUz, Initial residual = 0.00115045894, Final residual = 9.72441498e-06, No Iterations 26
Getting new points (fix)
Calling for move points
[host:08332] *** Process received signal ***
[host:08333] *** Process received signal ***
[host:08334] *** Process received signal ***
[host:08334] Signal: Segmentation fault (11)
[host:08334] Signal code: (-6)
[host:08334] Failing at address: 0x1f80000208e
[host:08333] Signal: Segmentation fault (11)
[host:08333] Signal code: (-6)
[host:08333] Failing at address: 0x1f80000208d
[host:08332] Signal: Segmentation fault (11)
[host:08332] Signal code: (-6)
[host:08332] Failing at address: 0x1f80000208c
[host:08334] [ 0] /lib64/libc.so.6 [0x7f6d6a0c16e0]
[host:08334] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f6d6a0c1645]
[host:08334] [ 2] /lib64/libc.so.6 [0x7f6d6a0c16e0]
[host:08334] [ 3] /lib64/libc.so.6 [0x7f6d6a105bcb]
[host:08334] [ 4] /lib64/libc.so.6(__libc_malloc+0xa1) [0x7f6d6a107891]
[host:08334] [ 5] /usr/lib64/libstdc++.so.6(_Znwm+0x1d) [0x7f6d6a91cf4d]
[host:08334] [ 6] /usr/lib64/libstdc++.so.6(_Znam+0x9) [0x7f6d6a91d089]
[host:08334] [ 7] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam4ListIcE7setSizeEi+0x33) [0x7f6d6af27593]
[host:08332] [ 0] /lib64/libc.so.6 [0x7f2bffc546e0]
[host:08332] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f2bffc54645]
[host:08332] [ 2] /lib64/libc.so.6 [0x7f2bffc546e0]
[host:08332] [ 3] /lib64/libc.so.6 [0x7f2bffc98a9a]
[host:08332] [ 4] /lib64/libc.so.6(__libc_malloc+0xa1) [0x7f2bffc9a891]
[host:08332] [ 5] /usr/lib64/libstdc++.so.6(_Znwm+0x1d) [0x7f2c004aff4d]
[host:08332] [ 6] /usr/lib64/libstdc++.so.6(_Znam+0x9) [0x7f2c004b0089]
[host:08332] [ 7] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libinterfaceProperties.so(_ZN4Foam5FieldINS_6VectorIdEEEC1Ei+0x67) [0x7f2c036ee667]
[host:08332] [ 8] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam13primitiveMesh22calcCellCentresAndVolsEv+0xad) [0x7f2c00c0077d]
[host:08332] [ 9] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam13primitiveMesh11cellCentresEv+0x1d) [0x7f2c00c009bd]
[host:08332] [10] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam9polyPatch15faceCellCentresEv+0x58) [0x7f2c00b6a5a8]
[host:08332] [11] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam18processorPolyPatch12initGeometryEv+0xca) [0x7f2c00b8952a]
[host:08332] [12] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0xd1) [0x7f2c00bb53e1]
[host:08332] [13] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam8polyMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x262) [0x7f2c00be3c12]
[host:08332] [14] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libfiniteVolume.so(_ZN4Foam6fvMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x119) [0x7f2c0255da89]
[host:08332] [15] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libmyMotionFvMesh.so(_ZN4Foam17translation3DOFfv6updateEv+0x693) [0x7f2c01000fb3]
[host:08332] [16] myInterDyMFoam-1.5-dev [0x4226d9]
[host:08332] [17] /lib64/libc.so.6(__libc_start_main+0xe6) [0x7f2bffc40586]
[host:08332] [18] myInterDyMFoam-1.5-dev [0x41fba9]
[host:08332] *** End of error message ***
[host:08334] [ 8] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam8OPstreamC1ENS_7Pstream10commsTypesEiiNS_8IOstream12streamFormatENS3_13versionNumberE+0xcc) [0x7f6d6af2689c]
[host:08334] [ 9] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam18processorPolyPatch12initGeometryEv+0xbf) [0x7f6d6aff651f]
[host:08334] [10] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0xd1) [0x7f6d6b0223e1]
[host:08334] [11] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam8polyMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x262) [0x7f6d6b050c12]
[host:08334] [12] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libfiniteVolume.so(_ZN4Foam6fvMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x119) [0x7f6d6c9caa89]
[host:08334] [13] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libmyMotionFvMesh.so(_ZN4Foam17translation3DOFfv6updateEv+0x693) [0x7f6d6b46dfb3]
[host:08334] [14] myInterDyMFoam-1.5-dev [0x4226d9]
[host:08334] [15] /lib64/libc.so.6(__libc_start_main+0xe6) [0x7f6d6a0ad586]
[host:08334] [16] myInterDyMFoam-1.5-dev [0x41fba9]
[host:08334] *** End of error message ***
[host:08333] [ 0] /lib64/libc.so.6 [0x7f6cda8d66e0]
[host:08333] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f6cda8d6645]
[host:08333] [ 2] /lib64/libc.so.6 [0x7f6cda8d66e0]
[host:08333] [ 3] /lib64/libc.so.6 [0x7f6cda91aa9a]
[host:08333] [ 4] /lib64/libc.so.6(__libc_malloc+0xa1) [0x7f6cda91c891]
[host:08333] [ 5] /usr/lib64/libstdc++.so.6(_Znwm+0x1d) [0x7f6cdb131f4d]
[host:08333] [ 6] /usr/lib64/libstdc++.so.6(_Znam+0x9) [0x7f6cdb132089]
[host:08333] [ 7]
/home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libinterfaceProperties.so(_ZN4Foam5FieldINS_6VectorIdEEEC1Ei+0x67) [0x7f6cde370667]
[host:08333] [ 8] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZNK4Foam9polyPatch15faceCellCentresEv+0x2a) [0x7f6cdb7ec57a]
[host:08333] [ 9] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam18processorPolyPatch12initGeometryEv+0xca) [0x7f6cdb80b52a]
[host:08333] [10] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0xd1) [0x7f6cdb8373e1]
[host:08333] [11] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so(_ZN4Foam8polyMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x262) [0x7f6cdb865c12]
[host:08333] [12] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libfiniteVolume.so(_ZN4Foam6fvMesh10movePointsERKNS_5FieldINS_6VectorIdEEEE+0x119) [0x7f6cdd1dfa89]
[host:08333] [13] /home/acuser3/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libmyMotionFvMesh.so(_ZN4Foam17translation3DOFfv6updateEv+0x693) [0x7f6cdbc82fb3]
[host:08333] [14] myInterDyMFoam-1.5-dev [0x4226d9]
[host:08333] [15] /lib64/libc.so.6(__libc_start_main+0xe6) [0x7f6cda8c2586]
[host:08333] [16] myInterDyMFoam-1.5-dev [0x41fba9]
[host:08333] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 8332 on node host exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
Do I need special decomposition in order to run dynamic meshes in parallel or did I miss something else? I suppose dynamic meshes work in parallel, don't they?
Any help or suggestions are welcome!
Last but not least here is a code snippet from my dynamic mesh class using the fvMotion solver. Can I move the mesh in this way?
Code:
pointVectorField& pointMotionU =
const_cast<pointVectorField&>
(
this->objectRegistry::lookupObject<pointVectorField>("pointMotionU")
);
pointMotionU.boundaryField()[bodyPatchID_] == ode_.U().value();
Info<< "Translation: " << ode_.X().value().x() << tab
<< ode_.X().value().y() << tab
<< ode_.X().value().z() << endl;
Info << "Getting new points" << endl;
// pointField newPoints = motionPtr_->newPoints();
// this works
Info << "Getting new points (fix)" << endl;
pointField newPoints = stationaryPoints_ * 1.0;
Info << "Calling for move points" << endl;
movePoints(newPoints); // <-- this fails
Info << "Move points done." << endl;
volVectorField& U =
const_cast<volVectorField&>(lookupObject<volVectorField>("U"));
U.correctBoundaryConditions();
Thank you so much in advance.
Best regards.
Jean-Peer |
