CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

segmentation fault--parrallel problem?

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   March 20, 2015, 02:24
Default segmentation fault--parrallel problem?
  #1
New Member
 
LEE
Join Date: Feb 2010
Posts: 4
Rep Power: 14
donno is on a distinguished road
Can anyone tell me how to solve this problem?
it is ok for a serial run,but it stops with segentation fault when i try to run parrallel in a cluster. my openfoam version is 2.1.1.

Code:
Cloud size= 800
Courant Number mean: 0.555556 max: 0.555898
deltaT = 0.0634921
DILUPBiCG:  Solving for Ux, Initial residual = 0.363798, Final residual = 1.84647e-09, No Iterations 7
DILUPBiCG:  Solving for Uy, Initial residual = 0.4546, Final residual = 6.04328e-09, No Iterations 7
DILUPBiCG:  Solving for Uz, Initial residual = 0.441025, Final residual = 3.78583e-10, No Iterations 6
DICPCG:  Solving for p, Initial residual = 0.00655311, Final residual = 0.000447691, No Iterations 3
DICPCG:  Solving for p, Initial residual = 0.000444716, Final residual = 3.87554e-05, No Iterations 29
time step continuity errors : sum local = 1.2429e-09, global = -3.07084e-10, cumulative = -3.07334e-10
DICPCG:  Solving for p, Initial residual = 0.00138002, Final residual = 9.84069e-05, No Iterations 3
DICPCG:  Solving for p, Initial residual = 9.84319e-05, Final residual = 9.50433e-09, No Iterations 747
time step continuity errors : sum local = 3.0458e-13, global = 5.6887e-16, cumulative = -3.07334e-10
DILUPBiCG:  Solving for epsilon, Initial residual = 0.0288261, Final residual = 1.75545e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.66385, Final residual = 9.60895e-10, No Iterations 9
DILUPBiCG:  Solving for C, Initial residual = 0.0620355, Final residual = 2.92867e-09, No Iterations 8
DILUPBiCG:  Solving for T, Initial residual = 0.137049, Final residual = 1.90345e-10, No Iterations 6
Time = 0.119048

ExecutionTime = 277.13 s  ClockTime = 279 s

Moving particles.

[3] [5] #0  Foam::error::printStack(Foam::Ostream&)#0  Foam::error::printStack(Foam::Ostream&)[1] [4] #0  Foam::error::printStack(Foam::Ostream&)#0  Foam::error::printStack(Foam::Ostream&)[7] [2] #0  Foam::error::printStack(Foam::Ostream&)[6] #0  #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.  

The process that invoked fork was:

  Local host:          node65 (PID 29186)
  MPI_COMM_WORLD rank: 4

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/ in "/public2/home/nl/OpenFOAM in "/public2/home/nengyuan/limao8 in linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #1  Foam::sigSegv::sigHandler(int)"/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64Gcc in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/lin/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #1  9/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #1  DPOpt/lib/libOpenFOAM.so"
ux64GccDPOpt/lib/libOpenFOAM.so"Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int)[1] #1  
[5] #1  Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #1  Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[7] #1  Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[7] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #2   in "/lib64/libc.so.6"
[4] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[1] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[5] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[6] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[2] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[7] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[3] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #7  
[1]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #8  __libc_start_main in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #7  
[4]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #8  __libc_start_main in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #7  
 in "/lib64/libc.so.6"
[1] #9  
[5]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #8  __libc_start_main[2]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #8  __libc_start_main[6]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #8  __libc_start_main in "/lib64/libc.so.6"
[4] #9  
[7]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #8  __libc_start_main[3]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #8  __libc_start_main in "/lib64/libc.so.6"
[5] #9  
 in "/lib64/libc.so.6"
[2] #9  
[1]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[node65:29183] *** Process received signal ***
[node65:29183] Signal: Segmentation fault (11)
[node65:29183] Signal code:  (-6)
[node65:29183] Failing at address: 0x3f9000071ff
[node65:29183] [ 0] /lib64/libc.so.6(+0x32920) [0x7f3c81738920]
[node65:29183] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f3c817388a5]
[node65:29183] [ 2] /lib64/libc.so.6(+0x32920) [0x7f3c81738920]
[node65:29183] [ 3] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam8particle11trackToFaceINS_13solidParticle12trackingDataEEEdRKNS_6VectorIdEERT_+0x19e) [0x47dcce]
[node65:29183] [ 4] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam13solidParticle4moveERNS0_12trackingDataEd+0x14d) [0x477d9d]
[node65:29183] [ 5] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam5CloudINS_13solidParticleEE4moveINS1_12trackingDataEEEvRT_d+0x417) [0x565577]
[node65:29183] [ 6] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam18solidParticleCloud4moveERKNS_11dimensionedINS_6VectorIdEEEE+0x42b) [0x4afc0b]
[node65:29183] [ 7] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong() [0x42a0d7]
[node65:29183] [ 8] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f3c81724cdd]
[node65:29183] [ 9] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong() [0x428559]
[node65:29183] *** End of error message ***
 in "/lib64/libc.so.6"
[6] #9  
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 29183 on node node65 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
[node65:29181] 6 more processes have sent help message help-mpi-runtime.txt / mpi_init:warn-fork
[node65:29181] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
donno is offline   Reply With Quote

Old   March 20, 2015, 05:00
Default
  #2
Senior Member
 
M. Montero
Join Date: Mar 2009
Location: Madrid
Posts: 136
Rep Power: 15
be_inspired is on a distinguished road
Hi,

you could check several things:
  • If previously the simulation works correctly in parallel until some iteration or crash at first moving particle, it can tell you where to look for.
  • If you use AMI, be sure that patches are not divided between different processors.Change decomposition method according to that.
  • Check memory request when moving particles. Maybe some of the nodes is out of memory or swap memory is low of a memory module is damaged.
  • You say that it works in serial. Are you running an own solver that maybe is not correctly coded to be launched in parallel?
be_inspired is offline   Reply With Quote

Old   March 20, 2015, 21:23
Default
  #3
New Member
 
LEE
Join Date: Feb 2010
Posts: 4
Rep Power: 14
donno is on a distinguished road
It is ok now when i set a smaller deltaT 0.005.
donno is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
fluentError: received a fatal signal (Segmentation fault). thomaszhangjing Fluent UDF and Scheme Programming 11 January 13, 2021 09:37
[Salome] segmentation fault when trying to convert UNV file to OpenFOAM range_rover OpenFOAM Meshing & Mesh Conversion 2 March 30, 2015 04:14
[OpenFOAM] Segmentation Fault on start - pvserver using o/s rendering, remote on cluster chrisb2244 ParaView 2 June 8, 2014 21:26
Segmentation fault (core dumped) when changing pressure BC jptorol OpenFOAM Running, Solving & CFD 0 September 25, 2013 18:05
Segmentation Fault w/ compiled OF 2.2.0 - motorBike example sudo OpenFOAM Running, Solving & CFD 3 April 2, 2013 17:27


All times are GMT -4. The time now is 01:35.