| EternalSeekerX |
October 9, 2020 04:13 |
SU2 7.0.7 Built on CentOS 7, parallel computation pyscript mpi exit error?
Hello everyone,
So I was testing SU2 parallel on my CentOS 7 machine. For this test I was running the Unsteady NACA 0012 compressible case. I ran it using 6 processor cores.
I notice that it satisfies convergence criteria at restart_flow_00532. And that is when the script launches the SU2_SOL and this is where the error shows up.
Code:
----------------------------- Solver Exit -------------------------------
All windowed time-averaged convergence criteria are fullfilled.
-------------------------------------------------------------------------
+-----------------------------------------------------------------------+
| File Writing Summary | Filename |
+-----------------------------------------------------------------------+
|SU2 restart |restart_flow_00532.dat |
|Paraview |flow_00532.vtu |
|Paraview surface |surface_flow_00532.vtu |
+-----------------------------------------------------------------------+
------------------------- Solver Postprocessing -------------------------
Deleted CNumerics container.
Deleted CIntegration container.
Deleted CSolver container.
Deleted CIteration container.
Deleted CInterpolator container.
Deleted CInterface container.
Deleted CGeometry container.
Deleted CFreeFormDefBox class.
Deleted CSurfaceMovement class.
Deleted CVolumetricMovement class.
Deleted CConfig container.
Deleted nInst container.
Deleted COutput class.
-------------------------------------------------------------------------
------------------------- Exit Success (SU2_CFD) ------------------------
-------------------------------------------------------------------------
| ___ _ _ ___ |
| / __| | | |_ ) Release 7.0.6 "Blackbird" |
| \__ \ |_| |/ / |
| |___/\___//___| Suite (Solution Exporting Code) |
| |
-------------------------------------------------------------------------
| SU2 Project Website: https://su2code.github.io |
| |
| The SU2 Project is maintained by the SU2 Foundation |
| (http://su2foundation.org) |
-------------------------------------------------------------------------
| Copyright 2012-2020, SU2 Contributors |
| |
| SU2 is free software; you can redistribute it and/or |
| modify it under the terms of the GNU Lesser General Public |
| License as published by the Free Software Foundation; either |
| version 2.1 of the License, or (at your option) any later version. |
| |
| SU2 is distributed in the hope that it will be useful, |
| but WITHOUT ANY WARRANTY; without even the implied warranty of |
| MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
| Lesser General Public License for more details. |
| |
| You should have received a copy of the GNU Lesser General Public |
| License along with SU2. If not, see <http://www.gnu.org/licenses/>. |
-------------------------------------------------------------------------
----------------- Physical Case Definition ( Zone 0 ) -------------------
Input mesh file name: unsteady_naca0012_mesh.su2
-------------------- Output Information ( Zone 0 ) ----------------------
The tabular file format is Tecplot (.dat).
Flow variables file name: flow.
------------- Config File Boundary Information ( Zone 0 ) ---------------
+-----------------------------------------------------------------------+
| Marker Type| Marker Name|
+-----------------------------------------------------------------------+
| Far-field| farfield|
+-----------------------------------------------------------------------+
| Heat flux wall| airfoil|
+-----------------------------------------------------------------------+
Two dimensional problem.
14576 grid points before partitioning.
14336 volume elements before partitioning.
2 surface markers.
128 boundary elements in index 0 (Marker = airfoil).
352 boundary elements in index 1 (Marker = farfield).
Executing the partitioning functions.
Building the graph adjacency structure.
Calling ParMETIS... graph partitioning complete (465 edge cuts).
Distributing ParMETIS coloring.
Rebalancing vertices.
Rebalancing volume element connectivity.
Rebalancing markers and surface elements.
15510 vertices including ghost points.
14797 interior elements including halo cells.
14797 quadrilaterals.
Establishing MPI communication patterns.
Identify vertices.
Storing a mapping from global to local point index.
------------------------- Solution Postprocessing -----------------------
Volume output fields: COORDINATES, SOLUTION
Info: Ignoring the following screen output fields:
DRAG, LIFT, RMS_DENSITY, REL_RMS_DENSITY, CAUCHY_TAVG_DRAG, CAUCHY_TAVG_LIFT
Screen output fields: TIME_ITER, INNER_ITER
Info: Ignoring the following history output groups:
REL_RMS_RES, RMS_RES, AERO_COEFF, TAVG_AERO_COEFF, CAUCHY, D_TAVG_AERO_COEFF
History output group(s): ITER
Ignoring Convergence Field(s): REL_RMS_DENSITY
Warning: No (valid) fields chosen for convergence monitoring. Convergence monitoring inactive.
Ignoring Time Convergence Field(s): TAVG_DRAG TAVG_LIFT
Warning: No (valid) fields chosen for time convergence monitoring. Time convergence monitoring inactive.
Reading and storing the solution from restart_flow_00499.
Writing the volume solution for time step 499.
|Paraview |flow_00499.vtu |
|Paraview surface |surface_flow_00499.vtu |
Reading and storing the solution from restart_flow_00500.
Writing the volume solution for time step 500.
|Paraview |flow_00500.vtu |
|Paraview surface |surface_flow_00500.vtu |
Reading and storing the solution from restart_flow_00501.
Writing the volume solution for time step 501.
|Paraview |flow_00501.vtu |
|Paraview surface |surface_flow_00501.vtu |
Reading and storing the solution from restart_flow_00502.
Writing the volume solution for time step 502.
|Paraview |flow_00502.vtu |
|Paraview surface |surface_flow_00502.vtu |
Reading and storing the solution from restart_flow_00503.
Writing the volume solution for time step 503.
|Paraview |flow_00503.vtu |
|Paraview surface |surface_flow_00503.vtu |
Reading and storing the solution from restart_flow_00504.
Writing the volume solution for time step 504.
|Paraview |flow_00504.vtu |
|Paraview surface |surface_flow_00504.vtu |
Reading and storing the solution from restart_flow_00505.
Writing the volume solution for time step 505.
|Paraview |flow_00505.vtu |
|Paraview surface |surface_flow_00505.vtu |
Reading and storing the solution from restart_flow_00506.
Writing the volume solution for time step 506.
|Paraview |flow_00506.vtu |
|Paraview surface |surface_flow_00506.vtu |
Reading and storing the solution from restart_flow_00507.
Writing the volume solution for time step 507.
|Paraview |flow_00507.vtu |
|Paraview surface |surface_flow_00507.vtu |
Reading and storing the solution from restart_flow_00508.
Writing the volume solution for time step 508.
|Paraview |flow_00508.vtu |
|Paraview surface |surface_flow_00508.vtu |
Reading and storing the solution from restart_flow_00509.
Writing the volume solution for time step 509.
|Paraview |flow_00509.vtu |
|Paraview surface |surface_flow_00509.vtu |
Reading and storing the solution from restart_flow_00510.
Writing the volume solution for time step 510.
|Paraview |flow_00510.vtu |
|Paraview surface |surface_flow_00510.vtu |
Reading and storing the solution from restart_flow_00511.
Writing the volume solution for time step 511.
|Paraview |flow_00511.vtu |
|Paraview surface |surface_flow_00511.vtu |
Reading and storing the solution from restart_flow_00512.
Writing the volume solution for time step 512.
|Paraview |flow_00512.vtu |
|Paraview surface |surface_flow_00512.vtu |
Reading and storing the solution from restart_flow_00513.
Writing the volume solution for time step 513.
|Paraview |flow_00513.vtu |
|Paraview surface |surface_flow_00513.vtu |
Reading and storing the solution from restart_flow_00514.
Writing the volume solution for time step 514.
|Paraview |flow_00514.vtu |
|Paraview surface |surface_flow_00514.vtu |
Reading and storing the solution from restart_flow_00515.
Writing the volume solution for time step 515.
|Paraview |flow_00515.vtu |
|Paraview surface |surface_flow_00515.vtu |
Reading and storing the solution from restart_flow_00516.
Writing the volume solution for time step 516.
|Paraview |flow_00516.vtu |
|Paraview surface |surface_flow_00516.vtu |
Reading and storing the solution from restart_flow_00517.
Writing the volume solution for time step 517.
|Paraview |flow_00517.vtu |
|Paraview surface |surface_flow_00517.vtu |
Reading and storing the solution from restart_flow_00518.
Writing the volume solution for time step 518.
|Paraview |flow_00518.vtu |
|Paraview surface |surface_flow_00518.vtu |
Reading and storing the solution from restart_flow_00519.
Writing the volume solution for time step 519.
|Paraview |flow_00519.vtu |
|Paraview surface |surface_flow_00519.vtu |
Reading and storing the solution from restart_flow_00520.
Writing the volume solution for time step 520.
|Paraview |flow_00520.vtu |
|Paraview surface |surface_flow_00520.vtu |
Reading and storing the solution from restart_flow_00521.
Writing the volume solution for time step 521.
|Paraview |flow_00521.vtu |
|Paraview surface |surface_flow_00521.vtu |
Reading and storing the solution from restart_flow_00522.
Writing the volume solution for time step 522.
|Paraview |flow_00522.vtu |
|Paraview surface |surface_flow_00522.vtu |
Reading and storing the solution from restart_flow_00523.
Writing the volume solution for time step 523.
|Paraview |flow_00523.vtu |
|Paraview surface |surface_flow_00523.vtu |
Reading and storing the solution from restart_flow_00524.
Writing the volume solution for time step 524.
|Paraview |flow_00524.vtu |
|Paraview surface |surface_flow_00524.vtu |
Reading and storing the solution from restart_flow_00525.
Writing the volume solution for time step 525.
|Paraview |flow_00525.vtu |
|Paraview surface |surface_flow_00525.vtu |
Reading and storing the solution from restart_flow_00526.
Writing the volume solution for time step 526.
|Paraview |flow_00526.vtu |
|Paraview surface |surface_flow_00526.vtu |
Reading and storing the solution from restart_flow_00527.
Writing the volume solution for time step 527.
|Paraview |flow_00527.vtu |
|Paraview surface |surface_flow_00527.vtu |
Reading and storing the solution from restart_flow_00528.
Writing the volume solution for time step 528.
|Paraview |flow_00528.vtu |
|Paraview surface |surface_flow_00528.vtu |
Reading and storing the solution from restart_flow_00529.
Writing the volume solution for time step 529.
|Paraview |flow_00529.vtu |
|Paraview surface |surface_flow_00529.vtu |
Reading and storing the solution from restart_flow_00530.
Writing the volume solution for time step 530.
|Paraview |flow_00530.vtu |
|Paraview surface |surface_flow_00530.vtu |
Reading and storing the solution from restart_flow_00531.
Writing the volume solution for time step 531.
|Paraview |flow_00531.vtu |
|Paraview surface |surface_flow_00531.vtu |
Reading and storing the solution from restart_flow_00532.
Writing the volume solution for time step 532.
|Paraview |flow_00532.vtu |
|Paraview surface |surface_flow_00532.vtu |
Reading and storing the solution from restart_flow_00533.
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
Error in "void CSolver::Read_SU2_Restart_Binary(CGeometry*, CConfig*, std::string)":
-------------------------------------------------------------------------
Unable to open SU2 restart file restart_flow_00533.dat
------------------------------ Error Exit -------------------------------
Traceback (most recent call last):
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/parallel_computation.py", line 110, in <module>
main()
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/parallel_computation.py", line 57, in main
options.compute )
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/parallel_computation.py", line 96, in parallel_computation
info = SU2.run.merge(config)
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/SU2/run/merge.py", line 81, in merge
merge_solution(konfig)
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/SU2/run/merge.py", line 107, in merge_solution
SU2_SOL( config )
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/SU2/run/interface.py", line 229, in SOL
run_command( the_Command )
File "/home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/SU2/run/interface.py", line 292, in run_command
raise exception(message)
SU2.EvaluationFailure: Path = /home/CentOS-VM/Documents/SU2/Run/Compressible_Tutorial/Unsteady_NACA0012/,
Command = mpirun -n 6 /home/CentOS-VM/.root/opt/SU2/SU2-7.0.7/bin/SU2_SOL config_SOL.cfg
SU2 process returned error '1'
[localhost:04234] 5 more processes have sent help message help-mpi-api.txt / mpi-abort
[localhost:04234] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
It seems that when the script reaches the last data file, it throws this error, is this expected? Seems like mpi doesn't like being closed like this?
Thanks |
