CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Main CFD Forum

How do CFD engineers make sure their simulations give correct results?

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree23Likes
  • 3 Post By sbaffini
  • 3 Post By aerosayan
  • 4 Post By Simbelmynė
  • 2 Post By sbaffini
  • 2 Post By Eifoehn4
  • 1 Post By JBeilke
  • 1 Post By sbaffini
  • 2 Post By LuckyTran
  • 2 Post By piu58
  • 3 Post By JBeilke

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   July 1, 2021, 04:22
Default How do CFD engineers make sure their simulations give correct results?
  #1
New Member
 
Join Date: Jul 2021
Posts: 12
Rep Power: 5
coffee_break is on a distinguished road
I am involved in CFD as a part of my postgraduate thesis study. Since I have experimental results I am able to understand my CFD simulation is correct or not. Through this process, I have also learned that turbulence models or many other parameters affect the results tremendously (especially heat transfer). So, I wonder how CFD engineers in industry are sure that their simulation for complex systems is correct?
coffee_break is offline   Reply With Quote

Old   July 1, 2021, 06:09
Default
  #2
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
Just to be complete, let me mention that, in general, there are actually two aspects in this. They are typically summarised with the following two questions:

1) Am I solving the right equations? Which is your original question. We refer to this as validation.

2) Am I solving the equations right? Which is the other part of the story, but relevant as well. We refer to this as verification.

For what concerns the second question (I'll get back to the first in a moment), once you have picked up the models/equations you think are needed in your case, you can do two things to be sure that the software correctly solves the equations as promised.

The more general one is using the so called "Method of Manufactured Solution" (MMS). You might or not have an analytical solution for your set of equations, but you can put whatever function into the variables of your equations, work out all the derivatives, etc. If it was an actual analytical solution, then your equations will be identically satisfied (i.e., return 0 = 0), otherwise some source term S will be needed to make it happen.

In both cases now you have some functions (possibly with a source term S to insert into them) that must solve your equations within the promised accuracy of the solver. You then just solve the given problem with the given source term S (if any at all) and check that the error reduces as you expect when you reduce time or grid steps. The very nice thing of the MMS is that you can apply it to whatever geometry, grid and model equations, no matter how complex. So, if you are rigorous, a first step in solving a problem with a black box software would be to use the MMS on the exact geometry and set of equations you plan to solve. Note that, as you know the solution everywhere, you also know the b.c. everywhere, and you can convert them at wish between dirichlet or flux type.

I have to admit that, in practice, this is rarely done, if ever at all. However, I wanted to highlight this aspect because there is a lot of confusion on the topic in relation to open vs closed source codes. It is a scientifical bullshit that open source is better. What you need from a software is to know exactly what it solves and how it does it and methods to apply the MMS (i.e., user routines for source terms, initializations and boundary conditions). That is, even if you coded the software yourself, scientifically, you should apply the MMS on each different set of geometry/grid/bc/equations/settings.

Now let's say you already did MMS. This tells you nothing on your real problem (your actual boundary conditions and the resulting solution). The ideal is that you perform the computation with different time and grid steps and, if certain conditions are met, extrapolate the solution with Richardson. If such conditons are not met, but MMS worked great with the same grid and time steps, it means your actual problem requires finer ones. In this case you should go finer then. I have to admit again that, while more common than MMS, this is rarely done as well. People just pick up the best they can for the given computational resources and, if ever needed, just improve that in some place by compromising the solution somewhere else.

If everything worked as expected, you can be sure that you are solving the equations correctly. But, as you asked, were they the correct equations in the first place? Your only chance here is validation. You typically pick up a problem that has been experimentally tested (and data is available, of course) and is as close as possible to your original problem. Then, using the methods of the verification (it is indeed more common to use them for validation cases), you solve such problem with a set of equations that should, heuristically, give the best answer for such problem. If they don't, you either switch one or more models, you develop one (or more) yourself, or you just don't give an F if you can't do nothing. There are also a lot of peer reviewed articles on validation, so you typically don't do that at all, just pick up the same code and models of the article (probably just models if you can't switch code). If no realistic validation is possible at all, you just use heuristic and experience. Again, in reality, your mileage might vary here as well.
sbaffini is offline   Reply With Quote

Old   July 1, 2021, 06:19
Default
  #3
Senior Member
 
Sayan Bhattacharjee
Join Date: Mar 2020
Posts: 495
Rep Power: 8
aerosayan is on a distinguished road
We don't. CFD is mostly numerical. Analytical and Experimental solutions are king.


In the most simplest terms, there are two sources of errors:
  1. Solver is garbage, and the developers didn't validate it properly. But the salesman is good enough to get your university or employer to pay for it, and now you have to use it even though it gives wrong results.
  2. User is not using the solver correctly, and using wrong models, discretization, turbulent schemes, timesteps, etc. This is most likely to happen.



While developing solvers, you validate them by comparing the results with analytical or experimental data. For example : I'm developing and validating my homebrewed supersonic-hypersonic compressible euler solver by running simulations on compression corners, and prandtl-meyer expansion corners meshes. Those cases have analytical solutions, and thus, I use them more than anything. If you validate your solver on multiple different analytical and experimental cases, then you can say that your solver is validated *but only for those cases*.


However, the user might be using the solver incorrectly, and thus they might get wrong results even though the solver is validated by the developers. This is the most realistic error that could occur. Thus, it's extremely important to use small scale experiments to validate your solutions, and progress step by step towards your final product. For example : You will do CFD on a small scale model of an aeroplane, and then do an experiment in wind tunnel, then go back to see if your CFD results are validated or not. If not, you fix your CFD model, then redo experiments until correct.


Experimental data is what matters at the end.
aerosayan is offline   Reply With Quote

Old   July 1, 2021, 06:37
Default
  #4
Senior Member
 
Simbelmynė's Avatar
 
Join Date: May 2012
Posts: 551
Rep Power: 16
Simbelmynė is on a distinguished road
@aerosayan


I have a different view, in that there is no difference in hierarchy between a physical experiment and a numerical experiment. Both can be good and both can be garbage. In my research field it is impossible to conduct most physical experiments since there is no equipment that can manage it. Yet, it is possible to estimate behavior of the system to a reasonable degree of accuracy, using numerical experiments.


I think there are many analogs here. In most cases you purchase your experimental equipment (a microscope or a numerical code) which means that they are mostly black boxes. Poor settings in the numerical code will lead to poor results, as will poor settings in the experimental equipment and sample preparation (which in a sense is equal to setting a poor boundary condition in the numerical code).


The verification may be better with experimental equipment though (but not always the validation). Here I define verification as finding bugs in the code or in the equipment. Similar to what @sbaffini said, the verification part of any code/equipment requires a lot of testing and this points towards using the most popular codes rather than developing something (general purpose) yourself.


It is fully possible to use numerical experiments as validation for physical experiments, although it is not as common as the other way around.
Simbelmynė is offline   Reply With Quote

Old   July 1, 2021, 06:48
Default
  #5
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
Quote:
Originally Posted by Simbelmynė View Post
@aerosayan


I have a different view, in that there is no difference in hierarchy between a physical experiment and a numerical experiment. Both can be good and both can be garbage. In my research field it is impossible to conduct most physical experiments since there is no equipment that can manage it. Yet, it is possible to estimate behavior of the system to a reasonable degree of accuracy, using numerical experiments.


I think there are many analogs here. In most cases you purchase your experimental equipment (a microscope or a numerical code) which means that they are mostly black boxes. Poor settings in the numerical code will lead to poor results, as will poor settings in the experimental equipment and sample preparation (which in a sense is equal to setting a poor boundary condition in the numerical code).


The verification may be better with experimental equipment though (but not always the validation). Here I define verification as finding bugs in the code or in the equipment. Similar to what @sbaffini said, the verification part of any code/equipment requires a lot of testing and this points towards using the most popular codes rather than developing something (general purpose) yourself.


It is fully possible to use numerical experiments as validation for physical experiments, although it is not as common as the other way around.
Let me add that, when it comes to validation, either of a code with respect to an experiment or, as you mention, of an experiment with respect to a simulation, there is, typically, little scrutiny of the source of validation from the other side.

It is an experiment, it must be correct, thinks the numerical guy. It is a DNS, it must be correct, thinks the experimental guy. Hopefully they both are, but we tend to forget that they could not.
Simbelmynė and aero_head like this.
sbaffini is offline   Reply With Quote

Old   July 1, 2021, 07:46
Default
  #6
Senior Member
 
Eifoehn4's Avatar
 
-
Join Date: Jul 2012
Location: Germany
Posts: 184
Rep Power: 14
Eifoehn4 is on a distinguished road
I would never trust a code framework, which does not at least has implemented:
  • Code verification via MMS
  • Regressiontests for each new feature
  • Unittests for the most relevant routines
  • A good documentation with various tutorials/examples
aerosayan and aero_head like this.
__________________
Check out my side project:

A multiphysics discontinuous Galerkin framework: Youtube, Gitlab.
Eifoehn4 is offline   Reply With Quote

Old   July 1, 2021, 09:01
Default
  #7
Senior Member
 
Joern Beilke
Join Date: Mar 2009
Location: Dresden
Posts: 516
Rep Power: 20
JBeilke is on a distinguished road
Most of the answers completely miss the original question:
"how would a CFD engineer in industry ..."
coffee_break likes this.
JBeilke is offline   Reply With Quote

Old   July 1, 2021, 09:10
Default
  #8
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
Quote:
Originally Posted by JBeilke View Post
Most of the answers completely miss the original question:
"how would a CFD engineer in industry ..."
Just summarizing my previous post:

Quote:
Originally Posted by sbaffini View Post
...
The more general one is using the so called "Method of Manufactured Solution" (MMS) ...

I have to admit that, in practice, this is rarely done, if ever at all ...

The ideal is that you perform the computation with different time and grid steps and, if certain conditions are met, extrapolate the solution with Richardson ...

I have to admit again that, while more common than MMS, this is rarely done as well. People just pick up the best they can for the given computational resources and, if ever needed, just improve that in some place by compromising the solution somewhere else ...

There are ... peer reviewed articles on validation, so you typically ... just pick up the same code and models of the article (probably just models if you can't switch code). If no realistic validation is possible at all, you just use heuristic and experience. Again, in reality, your mileage might vary here as well.
Maybe we can add that the CFD engineer in industry might end up with:

1) Whatever his company already uses or

2) Whatever the CFD code support team tells him to use

which, nonetheless, would still be based on points above (even if just heuristic and experience)
aero_head likes this.
sbaffini is offline   Reply With Quote

Old   July 1, 2021, 11:38
Default
  #9
Senior Member
 
Lucky
Join Date: Apr 2011
Location: Orlando, FL USA
Posts: 5,746
Rep Power: 66
LuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura aboutLuckyTran has a spectacular aura about
Quote:
Originally Posted by sbaffini View Post
Just summarizing my previous post:



Maybe we can add that the CFD engineer in industry might end up with:

1) Whatever his company already uses or

2) Whatever the CFD code support team tells him to use

which, nonetheless, would still be based on points above (even if just heuristic and experience)

Can confirm that 1) and 2) occurs.


Also we should take a step back and remember that the vast majority of engineering analysis is still done with low order tools (0D and 1D models) on excel sheets. A flow network model is already very high tech.

Whatever the fancy 3D CFD fails to do, it's certainly still better at it than whatever crap I was using before.

That is to say, we might as well be discussing how engineers are sure of anything ever.
sbaffini and aero_head like this.
LuckyTran is offline   Reply With Quote

Old   July 2, 2021, 02:12
Default
  #10
Senior Member
 
piu58's Avatar
 
Uwe Pilz
Join Date: Feb 2017
Location: Leipzig, Germany
Posts: 744
Rep Power: 15
piu58 is on a distinguished road
I made a bunch of calculations for industrial application. I knew in advance that my results would be use to change the process, mostly without additional experimental validation. So I had to ensure that the solution is correct within the given error level.

My advice:
- Make the simulation as simple as possible. I often simulate only "regions of interest" without meshing all of the area / technical installation
- Make the physics as simple as possible. Describe the effect of emitted variation (e.q. temperature dependency) in the discussion part of the result paper.
- Make a real good mesh. The physics dictates the mesh, not the geometry. Increase or decrease node density smoothly.
- If paramater variation is necessary (mostly it is) write a dedicated mesh generator to ensure the quality of the mesh and avoiding breaks in the node density.
Try to connect the simulation results with experimental data. That means calibrate the results against measuring available. Mostly they have something or can get something with less effort.
- Avoid 3D simulations. You get more accuracy from 2d results, perhaps with a correction term for 3d in comparison to an imperfect 3d solution.
- If possible, calculate with abstract (or rather without) coordinates. So the solution is valid for a whole class of problems.

All this makes only sense, if you know your tools. Simulation tools have pecularities, their usage need to be trained. There is no short cut here.
aero_head and coffee_break like this.
__________________
Uwe Pilz
--
Die der Hauptbewegung überlagerte Schwankungsbewegung ist in ihren Einzelheiten so hoffnungslos kompliziert, daß ihre theoretische Berechnung aussichtslos erscheint. (Hermann Schlichting, 1950)
piu58 is offline   Reply With Quote

Old   July 2, 2021, 04:45
Default
  #11
Senior Member
 
Joern Beilke
Join Date: Mar 2009
Location: Dresden
Posts: 516
Rep Power: 20
JBeilke is on a distinguished road
Quote:
Originally Posted by piu58 View Post
- Avoid 3D simulations. You get more accuracy from 2d results, perhaps with a correction term for 3d in comparison to an imperfect 3d solution.

The first thing I learned in the world of industrial CFD was, that even a simplfied 3d geometry is much better than a 2d simulation for most of the cases.

One of the worst ongoing jokes in CFD is, that people do 2d steady state calculations and blame the turbulence models for the crappy results.
JBeilke is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Moving mesh Niklas Wikstrom (Wikstrom) OpenFOAM Running, Solving & CFD 122 June 15, 2014 07:20
Generate a CFD domain from FEA results Dave442 Main CFD Forum 0 August 3, 2011 13:42
Installation OF1.5-dev ttdtud OpenFOAM Installation 46 May 5, 2009 03:32
OpenFOAM with IBM AIX matthias OpenFOAM Installation 20 March 25, 2008 03:36
CFD - Trends and Perspectives Jonas Larsson Main CFD Forum 16 August 7, 1998 17:27


All times are GMT -4. The time now is 19:27.