Implicit supseronic flow simulations in flexi

vachan · March 30, 2022, 04:56

Dear all,

I am trying to perform a Mach 2 supersonic flow simulation in flexi (https://www.flexi-project.org/) using implicit time stepping.

Flexi user guide describes how to do implicit simulation for a Mach 0.2 RANS flat plate simulation. However, I am unable to obtain the correct solution with implicit time stepping in the supersonic case I am interested in.

The simulation I am doing is of an impinging shock-boundary layer interaction. I am pasting my settings here and attaching the mesh file (degrez_corrected_mesh.h5) as a part of the zipped attachment to this post.

Code:

!==================================================================================
! OUTPUT
!=============================================================================== !
ProjectName = degrez_corrected
Logging = F
ErrorFiles = T
OutputFormat = 0
doPrintStatusLine = T
ColoredOutput = F

!=========================================================================================
! Interpolation
!=========================================================================================
N = 2
NVisu = 2
NOut = 2

!=========================================================================================
! Mesh
!=========================================================================================
MeshFile = degrez_corrected_mesh.h5
useCurveds = F
crossProductMetrics = F

!=========================================================================================
! Equation
!=========================================================================================
IniExactFunc = 1
IniRefState = 3
kappa = 1.400
R = 286.689
Pr = 0.69
mu0 = 1.716e-5
! mu0 = 0.0
Ts = 111
Tref = 273
RefState = (/1,0,0,0,1718.81/) ! gives outflow pressure
RefState = (/0.0290566,519.41,-34.6182,0,1393.6/) ! post shock conditions
RefState = (/0.0248218,540,0,0,1117.3/) ! pre shock conditions (freestream)
BoundaryName = back
BoundaryType = (/9,0/)
BoundaryName = front
BoundaryType = (/9,0/)
BoundaryName = wall
BoundaryType = (/3,1/)
BoundaryName = outflow
BoundaryType = (/24,1/)
BoundaryName = post_shock_inflow
BoundaryType = (/2,2/)
BoundaryName = pre_shock_inflow
BoundaryType = (/2,3/)

!=========================================================================================
! Riemann
!=========================================================================================
Riemann = 33
RiemannBC = -1

! =============================================================================== !
! FV-Subcell
! =============================================================================== !
IndicatorType = Jameson
IndVar = 6
FV_LimiterType = 1
FV_IndUpperThreshold = 0.010 ! upper threshold (if IndValue above this value, switch to FV)
FV_IndLowerThreshold = 0.005 ! lower threshold (if IndValue below this value, switch to DG)
FV_toDG_indicator = T     
FV_toDG_limit = -5.5 
FV_IniSupersample = T

!=========================================================================================
! TimeDisc
!=========================================================================================
TimeDiscMethod = esdirk3-4
TEnd = 3e-3
CFLScale = 2.5
DFLScale = 2.5
! EpsNewton = 1e-2
adaptepsNewton = T
! nNewtonIter = 100
! FD_Order = 1
! EpsGMRES = 1e-1
! nRestarts = 50
! nKDim = 50
! PrecondType = 0
NoFillIn = T
SolveSystem = 1
! HyperbolicPrecond = T
! DoDisplayPrecond = T

!=========================================================================================
! Analyze
!=========================================================================================
Analyze_dt = 3e-6
nWriteData = 1
CalcResiduals = F
CalcErrorNorms = F

varName = Density
varName = Pressure
varName = VelocityX
varName = VelocityY
varName = Temperature

Even for a CFL value as small as 2.5, flexi reports the following:

Code:

Attention: Timestep failed, repeating with dt/2!

And then strangely, except for first few outputs, flexi reports zero GMRES and Newton iterations:

Code:

 Timestep   :    5.8680110E-07
#Timesteps  :    1.6000000E+01
#GMRES iter  :    0.0000000E+00
#Newton iter :    0.0000000E+00

I have also put the log file (flexi.log) generated with these specific settings in the zipped attachment. The result files show unphysical solution with negative pressure and extreme oscillations. I have also tried changing the preconditioner and tolerances. They seem to just delay/advance such behaviour, but I wasn't able to get a clean simulation.

My build settings:

Code:

CMAKE_BUILD_TYPE                 Release                                                                                                                             
 CMAKE_INSTALL_PREFIX             /home/vachan/Documents/Work/flexi-framework/flexi                                                                                   
 FLEXI_2D                         ON                                                                                                                                  
 FLEXI_EDDYVISCOSITY              OFF                                                                                                                                 
 FLEXI_EQNSYSNAME                 navierstokes                                                                                                                        
 FLEXI_FV                         ON                                                                                                                                  
 FLEXI_FV_RECONSTRUCTION          ON                                                                                                                                  
 FLEXI_LIFTING                    br2                                                                                                                                 
 FLEXI_NODETYPE                   GAUSS-LOBATTO                                                                                                                       
 FLEXI_PARABOLIC                  ON                                                                                                                                  
 FLEXI_PERFORMANCE                ON                                                                                                                                  
 FLEXI_POLYNOMIAL_DEGREE          N                                                                                                                                   
 FLEXI_SPLIT_DG                   ON                                                                                                                                  
 FLEXI_TESTCASE                   default                                                                                                                             
 FLEXI_VISCOSITY                  sutherland                                                                                                                          
 LIBS_BUILD_HDF5                  ON                                                                                                                                  
 LIBS_BUILD_MATH_LIB              OFF                                                                                                                                 
 LIBS_USE_MKL                     OFF                                                                                                                                 
 LIBS_USE_MPI                     ON                                                                                                                                  
 LIBS_USE_OPENMP                  OFF                                                                                                                                 
 LIBS_USE_PAPI                    OFF                                                                                                                                 
 POSTI                            ON                                                                                                                                  
 POSTI_AVG2D                      OFF                                                                                                                                 
 POSTI_CHANNEL_FFT                OFF                                                                                                                                 
 POSTI_DMD                        OFF                                                                                                                                 
 POSTI_MERGETIMEAVERAGES          OFF                                                                                                                                 
 POSTI_RP_EVALUATE                OFF                                                                                                                                 
 POSTI_RP_PREPARE                 OFF                                                                                                                                 
 POSTI_RP_VISUALIZE               OFF                                                                                                                                 
 POSTI_SWAPMESH                   OFF                                                                                                                                 
 POSTI_TO3D                       OFF                                                                                                                                 
 POSTI_VISU                       ON                                                                                                                                  
 POSTI_VISU_PARAVIEW              OFF                                                                                                                                 
 POSTI_WALLDISTANCE               OFF                                                                                                                                 
 USE_PGO                          ON

Am I missing some key setting, or do I have any setting wrong? Kindly help me in getting this simulation right.

Thanking in anticipation
Vachan

Eifoehn4 · March 31, 2022, 15:16

Dear vachan,

I will contact my FLEXI collegues. We're going to make this work!

Regards

vachan · April 1, 2022, 00:24

Thanks a lot, Eifoehn4! I am truly grateful for your quick response and support!

JonaZ · April 1, 2022, 07:04

Dear Vachan,

there was indeed a bug in the code that caused this behavior you observed. We fixed this and the updated version will be available on github after the weekend. So just pull the most recent version on Monday and you can run your simulation!

What you observed was the following: the bug caused not choosing the shock-capturing scheme property. This then resulted in instabilities, so FLEXI tried to repeat the timestep but it was never stable.

Though, I am not sure if implicit time discretization is the best choice for your setting. As the Mach number is quite high and the solution (especially in the beginning) changes rapidly, the implicit method is not able to use very large timesteps. So you maybe have to ramp your timestep to obtain a stable simulation.

Kind regards,

Jonas

vachan · April 1, 2022, 07:15

Dear Jonas,

Quote:

there was indeed a bug in the code that caused this behavior you observed. We fixed this and the updated version will be available on github after the weekend. So just pull the most recent version on Monday and you can run your simulation!

Great! Thanks very much for such a prompt response.

Quote:

What you observed was the following: the bug caused not choosing the shock-capturing scheme property. This then resulted in instabilities, so FLEXI tried to repeat the timestep but it was never stable.

I felt that the issue was due to some minor oversight. Glad it was debugged!

Quote:

Though, I am not sure if implicit time discretization is the best choice for your setting. As the Mach number is quite high and the solution (especially in the beginning) changes rapidly, the implicit method is not able to use very large timesteps. So you maybe have to ramp your timestep to obtain a stable simulation.

Will keep your suggestions in mind.

Thanks again for such a quick response!

vachan · April 4, 2022, 01:30

Dear Jonas,

As you said, I have noticed that a commit for the bug fix has been made to the github repo: https://github.com/flexi-framework/f...acaf6dd614ba21.

I have compiled the latest version of flexi now. However, I am afraid this still didn't improve the performance of implicit stepping.

This time, I started the implicit solution (with CFL 5 and the settings pasted below) with the solution at 1e-3 sec obtained through explicit stepping (standardrk3-3). By my estimate, the flow becomes steady in about 3e-3 sec. So at 1e-3 sec, most of the transients are gone, and it is only a matter of the separation region growing to its full size after this point.

Code:

!=========================================================================================
! TimeDisc
!=========================================================================================
TimeDiscMethod = esdirk3-4
TEnd = 3e-3
CFLScale = 5
DFLScale = 5
! EpsNewton = 1e-2
adaptepsNewton = T
nNewtonIter = 100
FD_Order = 1
EpsGMRES = 1e-1
nRestarts = 50
nKDim = 50
PrecondType = 1
NoFillIn = T
SolveSystem = 1
HyperbolicPrecond = T
DoDisplayPrecond = T

Even then, I still get this message:

Code:

Attention: Timestep failed, repeating with dt/2!

So is it possible that there is something still left to be resolved? Or, please let me know if I am setting something incorrectly.

Thanking again,
Vachan

JonaZ · April 6, 2022, 08:54

Dear Vachan,

I am not completely sure why this is happening in your case - there is something going wrong with the preconditioner. We probably will need some time to figure that out.

In the meanwile I would suggest to use either fully explicit methods or implicit methods without preconditioning and additionally change your settings slightly:

Code:

!=========================================================================================
! TimeDisc
!=========================================================================================
TimeDiscMethod = esdirk3-4
TEnd = 3e-3
CFLScale = 5
DFLScale = 5
EpsNewton = 1e-2
adaptepsNewton = F
nNewtonIter = 10
FD_Order = 1
EpsGMRES = 1e-1
nRestarts = 5
nKDim = 30
PrecondType = 0

I hope this helps you and we will continue working on making the use of a preconditioner for this setting to work.

Kind regards,
Jonas

vachan · April 7, 2022, 00:46

Dear Jonas,

Thanks for your response. It is indeed running fine without the preconditioner and your suggestions.

I wish and hope that the issue in preconditioner will be sorted out soon!

Thanking again,
Vachan

JonaZ · April 14, 2022, 05:28

Dear Vachan,

I had a closer look on your problem and I think the main issue that causes bad convergence behaviors is the non-linearity of the slope limiting procedure (you selected "minmod" limiting).
In this forum thread http://www.cfd-online.com/Forums/mai...d-limiter.html you find a related discussion and in the publications

W. Gropp, D. Keyes, L. C. McInnes, M. D. Tidriri: Globalized Newton-Krylow-Schwarz algorithms and software for parallel implicit CFD. The International Journal of High Performance Computing Applications 14(2), 102-136 (2000)

and

P. A.Burton, P. K. Sweby: A Dynamical Approach Study of Some Explicit and Implicit TVD Schemes and Their Convergences to Steady-state Solutions. Numerical Analysis Report 5/95, University of Reading, Department of Mathematics, 1995.

you find some investigations on this.

I therefore recommend you to use the first order finite volume scheme for shock capturing, i.e.

Code:

FV_LimiterType = 0

In a future release of FLEXI there might be the van Albada limiter. Even though, this limiter significantly improves the convergence properties compared to minmod limiting, it is still quite difficult to use very large timesteps.

Moreover, it is recommendable to change your preconditioner settings as I observed that the accuracy of the preconditioner with ILU(0) inversion of the blocks is not sufficient for larger timesteps for your setting. Maybe you can try the following settings:

Code:

PrecondType = 1
NoFillIn = F
SolveSystem = 0
HyperbolicPrecond = F
PrecondIter = 10

With these settings I could use rather large timesteps (dt>1e-4) without convergence issues. To reduce the required computational time you can play around with the PrecondIter value that specifies the amount of timesteps the preconditioner is kept frozen.

I hope that helps you!

Kind regards

Jonas

vachan · July 12, 2022, 01:24

Dear Jonas,

Sorry for such a late response!

I am really grateful to you for your continuing engagement in this topic. I will try the settings you suggested and get back with my experience shortly.

Thanks again!

vachan · July 13, 2022, 00:21

Dear Jonas,

As you mentioned, I confirm that I was able to use CFL=5 (which corresponds to dt ~ 1e-4) without any convergence issues.

Thanks!

March 31, 2022, 15:16		#2
Eifoehn4 Senior Member - Join Date: Jul 2012 Location: Germany Posts: 184 Rep Power: 13	Dear vachan, I will contact my FLEXI collegues. We're going to make this work! Regards flexi-project likes this. __________________ Check out my side project: A multiphysics discontinuous Galerkin framework: Youtube, Gitlab.

April 1, 2022, 00:24		#3
vachan New Member Vachan Potluri Join Date: Jul 2017 Posts: 29 Rep Power: 8	Thanks a lot, Eifoehn4! I am truly grateful for your quick response and support! flexi-project likes this.

April 1, 2022, 07:04		#4
JonaZ New Member Jonas Z. Join Date: Mar 2022 Posts: 3 Rep Power: 4	Dear Vachan, there was indeed a bug in the code that caused this behavior you observed. We fixed this and the updated version will be available on github after the weekend. So just pull the most recent version on Monday and you can run your simulation! What you observed was the following: the bug caused not choosing the shock-capturing scheme property. This then resulted in instabilities, so FLEXI tried to repeat the timestep but it was never stable. Though, I am not sure if implicit time discretization is the best choice for your setting. As the Mach number is quite high and the solution (especially in the beginning) changes rapidly, the implicit method is not able to use very large timesteps. So you maybe have to ramp your timestep to obtain a stable simulation. Kind regards, Jonas flexi-project likes this.

April 4, 2022, 01:30	Still not working?	#6
vachan New Member Vachan Potluri Join Date: Jul 2017 Posts: 29 Rep Power: 8	Dear Jonas, As you said, I have noticed that a commit for the bug fix has been made to the github repo: https://github.com/flexi-framework/f...acaf6dd614ba21. I have compiled the latest version of flexi now. However, I am afraid this still didn't improve the performance of implicit stepping. This time, I started the implicit solution (with CFL 5 and the settings pasted below) with the solution at 1e-3 sec obtained through explicit stepping (standardrk3-3). By my estimate, the flow becomes steady in about 3e-3 sec. So at 1e-3 sec, most of the transients are gone, and it is only a matter of the separation region growing to its full size after this point. Code: !========================================================================================= ! TimeDisc !========================================================================================= TimeDiscMethod = esdirk3-4 TEnd = 3e-3 CFLScale = 5 DFLScale = 5 ! EpsNewton = 1e-2 adaptepsNewton = T nNewtonIter = 100 FD_Order = 1 EpsGMRES = 1e-1 nRestarts = 50 nKDim = 50 PrecondType = 1 NoFillIn = T SolveSystem = 1 HyperbolicPrecond = T DoDisplayPrecond = T Even then, I still get this message: Code: Attention: Timestep failed, repeating with dt/2! So is it possible that there is something still left to be resolved? Or, please let me know if I am setting something incorrectly. Thanking again, Vachan

April 6, 2022, 08:54		#7
JonaZ New Member Jonas Z. Join Date: Mar 2022 Posts: 3 Rep Power: 4	Dear Vachan, I am not completely sure why this is happening in your case - there is something going wrong with the preconditioner. We probably will need some time to figure that out. In the meanwile I would suggest to use either fully explicit methods or implicit methods without preconditioning and additionally change your settings slightly: Code: !========================================================================================= ! TimeDisc !========================================================================================= TimeDiscMethod = esdirk3-4 TEnd = 3e-3 CFLScale = 5 DFLScale = 5 EpsNewton = 1e-2 adaptepsNewton = F nNewtonIter = 10 FD_Order = 1 EpsGMRES = 1e-1 nRestarts = 5 nKDim = 30 PrecondType = 0 I hope this helps you and we will continue working on making the use of a preconditioner for this setting to work. Kind regards, Jonas Eifoehn4 likes this.

April 7, 2022, 00:46		#8
vachan New Member Vachan Potluri Join Date: Jul 2017 Posts: 29 Rep Power: 8	Dear Jonas, Thanks for your response. It is indeed running fine without the preconditioner and your suggestions. I wish and hope that the issue in preconditioner will be sorted out soon! Thanking again, Vachan

April 14, 2022, 05:28		#9
JonaZ New Member Jonas Z. Join Date: Mar 2022 Posts: 3 Rep Power: 4	Dear Vachan, I had a closer look on your problem and I think the main issue that causes bad convergence behaviors is the non-linearity of the slope limiting procedure (you selected "minmod" limiting). In this forum thread http://www.cfd-online.com/Forums/mai...d-limiter.html you find a related discussion and in the publications W. Gropp, D. Keyes, L. C. McInnes, M. D. Tidriri: Globalized Newton-Krylow-Schwarz algorithms and software for parallel implicit CFD. The International Journal of High Performance Computing Applications 14(2), 102-136 (2000) and P. A.Burton, P. K. Sweby: A Dynamical Approach Study of Some Explicit and Implicit TVD Schemes and Their Convergences to Steady-state Solutions. Numerical Analysis Report 5/95, University of Reading, Department of Mathematics, 1995. you find some investigations on this. I therefore recommend you to use the first order finite volume scheme for shock capturing, i.e. Code: FV_LimiterType = 0 In a future release of FLEXI there might be the van Albada limiter. Even though, this limiter significantly improves the convergence properties compared to minmod limiting, it is still quite difficult to use very large timesteps. Moreover, it is recommendable to change your preconditioner settings as I observed that the accuracy of the preconditioner with ILU(0) inversion of the blocks is not sufficient for larger timesteps for your setting. Maybe you can try the following settings: Code: PrecondType = 1 NoFillIn = F SolveSystem = 0 HyperbolicPrecond = F PrecondIter = 10 With these settings I could use rather large timesteps (dt>1e-4) without convergence issues. To reduce the required computational time you can play around with the PrecondIter value that specifies the amount of timesteps the preconditioner is kept frozen. I hope that helps you! Kind regards Jonas

July 12, 2022, 01:24		#10
vachan New Member Vachan Potluri Join Date: Jul 2017 Posts: 29 Rep Power: 8	Dear Jonas, Sorry for such a late response! I am really grateful to you for your continuing engagement in this topic. I will try the settings you suggested and get back with my experience shortly. Thanks again!

July 13, 2022, 00:21		#11
vachan New Member Vachan Potluri Join Date: Jul 2017 Posts: 29 Rep Power: 8	Dear Jonas, As you mentioned, I confirm that I was able to use CFL=5 (which corresponds to dt ~ 1e-4) without any convergence issues. Thanks!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Issues on the simulation of high-speed compressible flow within turbomachinery	dowlee	OpenFOAM Running, Solving & CFD	11	August 6, 2021 06:40
Match Pressure Inlet/Outlet Boundary Condition Mass Flow Rate	MSchneid	Fluent UDF and Scheme Programming	3	February 23, 2019 06:00
Implementing AUSM for FVM and Fully Implicit subsonic compressible flow	Krishan.aero	Main CFD Forum	6	December 23, 2018 22:13
Inviscid flow simulations in cfx	mahdidelfan	CFX	1	June 29, 2014 06:14
How to do simulations for pressible flow	Gaurav	FLUENT	3	October 9, 2003 16:20