[nash85]

Forum members,

I wasn't sure where was best to post as it covers a couple of codes but I wanted to share a piece of work to compare OpenFOAM and STAR-CCM+ (both compressible) for a complete aircraft as part of the 3rd High-Lift Prediction Workshop ( https://hiliftpw.larc.nasa.gov).

I haven't seen this done properly before and I thought it might be of interest for people using either code and looking to complete complex flows. It's not entirely comprehensive but I think it does a fairly decent job of being fair and objective.

Main conclusions were:
1) Wall-Distance is not 'correct' in OpenFOAM i.e not compared to STAR-CCM+ and the majority of other commercial and government codes which means for highly skewed grids you won't get the same Spalart-Allmaras solution as other codes.
2) OpenFOAM and STAR-CCM+ agree surprisedly well for both the complex aircraft cases for the forces and flow fields (using the same grids).
3) SA RANS model does well for low angles of attack but at high angles of attack cannot predict the correct flow physics
4) OpenFOAM is not as robust or computational efficient as STAR-CCM+ requiring more iterations and babysitting to get it to converge. Likely due to the lack of a coupled solver.

Happy to answer any questions. Just google 'validation and verification of OpenFOAM for high-lift aircraft flows' to find versions of the paper.

Neil (neil.ashton@eng.ox.ac.uk)

[LuckyTran]

Originally Posted by nash85

4) OpenFOAM is not as robust or computational efficient as STAR-CCM+ requiring more iterations and babysitting to get it to converge. Likely due to the lack of a coupled solver.

Statements like these are generally unwelcome (to me). They are unsusceptingly subjective and not as objective as it might seem. If you consider segregated vs coupled solvers on a spectrum (and yes it is a spectrum), more strongly coupled solvers will undoubtedly converge in fewer iterations than less-coupled solvers in situations when this coupling is limiting the convergence.... which it undoubtedly is, otherwise the problem would converge in a few iterations to begin with. That's a no-brainer yet it is not obvious.

User errors (in my opinion) shouldn't be included in a comparison of technical capabilities.

As an example.... Let's say one of the software implements features that makes it more easy for people with color-blindness to use the software. Will you then say Star-CCM/OF is better than the other because it needs less babysitting?

It is a nice feature to have but it's not the fault of either software that their user might be handicapped (e.g. braindead). And while it is nice to take these into consideration as part of good software design, it's non-technical. There's plenty more relevant comparisons that can be made than to focus on the less relevant ones. Otherwise, you might as well say:

5) Star-CCM+ is better because it has a GUI and OpenFOAM doesn't. Star-CCM+ is more user friendly.

[FMDenaro]

This type of workshops are always interesting but one should have care in drawing the correct conclusions. There are some users-dependent choices and some fixed parameters but, at the end, it is hard to do an effective comparison that addresses a winner ...
We can only clearly assess the fact the commercial codes are user-friendly...

[nash85]

Thanks for your comments. I should stress again that this isn't about saying which code is better but illustrating the pros and cons and discussing some of the V&V of turbulence models in OpenFOAM i.e. the wall-distance calculation.

I would encourage you to read the paper as I don't believe user-friendliness comes into it or anything to do with GUI's etc.

With regards to coupled and segregated - the point is that OpenFOAM does not have a compressible coupled solver in it's standard release and this test-case in our opinion has shown essentially the need for one. If you run these cases with a segregated solver in STAR-CCM+ they also show worse convergence so we were hoping to provide some motivation for the OpenFOAM teams to look into a compressible coupled solver.

[FMDenaro]

Is the evaluation of the convergence rate performed using exactly same grid, same discretization schemes and same turbulence modelling?


However, high lift configuration is a challenging flow problem, it should be also defined clearly what is the converged solution.

[nash85]

Yes this is all described in the paper, which as I said in the previous post you can find by googling 'validation and verification of openfoam for high-lift aircraft'.

[FMDenaro]

Originally Posted by nash85

Yes this is all described in the paper, which as I said in the previous post you can find by googling 'validation and verification of openfoam for high-lift aircraft'.

The paper is here for general interest of a reader:

https://www.researchgate.net/publica...Aircraft_Flows


From the figures it seems that OF requires further grid refinement to produce a more flatted slope but we cannot say if STAR-CCM+ provides a better result. Maybe, some variables that are not integral quantities should be also supported.

If a time dependent formulation is solved until a steady state is reached, what about the time integration scheme and what about the criterion for the converged steady state?


However, this is a flow problem where RANS fail to provide accurate solutions

[wyldckat]

Greetings to all!

I'm sort of passing by quickly and tried to skim through the article ASAP...

It's a pain in the neck that a V&V article doesn't have a clear link to the cases ready to be re-tested... and they are using OpenFOAM 4.1 from years ago, even though the article is tagged as being release April 2019... And I can't find a clear indication of which specific Star-CCM+ version was used... I guess we're lucky if they don't mistake 1ft with 10cm

And I can't find any clear references to the fact that OpenFOAM has at least 2 algorithms for wall distance calculation... so which one did they use? Because the default in the tutorials is the fastest algorithm, not the most accurate which is more CPU intensive... I'm not entirely familiar with it, but after a quick search, here are two different documentations, depending on the version:

• OpenFOAM 4.x (OpenFOAM.org): https://cpp.openfoam.org/v4/classFoa...e.html#details
• OpenFOAM v1812 (OpenFOAM.com): https://www.openfoam.com/documentati...-meshwave.html - link may change with version...

I didn't have time to read the whole article, but given that "reproducibility" wasn't mentioned in the article, it's almost only as good as far as we can throw it

Best regards,
Bruno

[nash85]

Thanks for your comments Bruno.

The meshes in both STAR-CCM+ and OpenFOAM format are available at https://hiliftpw.larc.nasa.gov for the high-lift cases so please feel free to run these cases yourself. The other cases mainly used the meshes from https://turbmodels.larc.nasa.gov so again I encourage you and others to run these cases and see what results you get in whatever version of OpenFOAM or STAR-CCM+ you wish.

It states in the paper that we use STAR-CCM+ v11.6 and OpenFOAM 4.1.

You can thank the slowness of journal publications for why it is April 2019! Even though the work was done in 2016/2017 hence the older version.

All the wall-distance methods were tested even in 1812 and none of them give the correct answer. There is a forum thread on this back in 2015 and I don't believe there has been any newer versions since.
https://bugs.openfoam.org/view.php?id=968

I believe this has been raised with ESI but it just needs some funding to implement a fully parallel version of the suggest fix in the paper link below.

Kareem, A., Spence, S. M., and Wei, D., “Turbulence Model [28] Verification and Validation in an Open Source Environment,” Progress
in Computational Fluid Dynamics: An International Journal, Vol. 1,
No. 1, 2016, Paper 1.
doi:10.1504/PCFD.2016.10001448

Thanks again

Neil

[wyldckat]

Hi Neil,

My apologies for my previous post... I was reviewing in my head earlier today and realized that I had missed several points (e.g. the effort put into the article and the V&V websites) due to writing while being very sleepy and in a hurry.... nor did I notice that you were one of the authors of the article, so I had construed that this was a complaint thread and not a constructive analysis thread...

Furthermore, I'm more traumatized by past papers and thesis that had wrong units, than I had noticed... because those had made things a lot harder to figure out where the problems were when we were trying to reproduce cases, that said trauma effects surfaced while I was skimming through things...

I've read the whole thread now at a slower pace, but haven't read the full paper yet.
However, I did go search for where the Star-CCM+ version was written in it and it was 9 pages away from the indication of which OpenFOAM version was used... which... wasn't the best place... I believe they should have been disclaimed right in the first paragraph of the chapter II "Verification and Validation", when they were first mentioned. A lot can change between versions, so it's as critical as pointing out in which units things are being done...

Many thanks for providing the link to the bug report where details were addressed regarding the wall distance calculations... it wasn't clear to me that the cases that were simulated were using this feature.

It took me several minutes to find the cases that could (perhaps) be used with OpenFOAM... the simplest to try and reproduce things seemed to be this one: https://turbmodels.larc.nasa.gov/nac...erics_val.html - but I don't any more time today to try and look into this.

However, I would like to point out a few details, from what I've gathered over the years on this topic and OpenFOAM:

1. FreeCASE - https://home.aero.polimi.it/freecase...Aster:Overview - has an implementation of a solver that builds with OpenFOAM that is allegedly more well suited for compressible flow. See the images given in section "Aerodynamic solver".
2. Relaxation steps: last year I learned that for at least academic studies of airfoil simulations, they used both explicit and implicit relaxation factors, namely relaxed both the equations and the fields, when solving them. It would take a ton more (outer) iterations, but it would eventually reach the same results as the validation/experimental data. And orthogonal discretization had to be used, because the cells were extremely flat, so any skewness correction operations would only make things worse...
3. foam-extend (one of the major forks of OpenFOAM) has block-coupled solvers that can indeed solve in a lot fewer iterations. I don't remember if they have a compressible implementation.
4. Tetrahedral cells are a pain in the neck when it comes to OpenFOAM... using "nonOrthogonalCorrectors" is a must and sometimes a special discretization scheme is needed for using cell vertexes in the calculations, akin to FEM, in order for things to improve.
   • Took me a long while to re-find this, but here it is: Too diffusive result on Delaunay type mesh for pipe - the "pointLinear" and "pointCellsLeastSquares" discretization schemes.

Best regards,
Bruno

[nash85]

No problem at all Bruno! This publication certainly does not have all the answers and raises more questions than it answers, but I hope this can at least give a push to those OpenFOAM users to try out their improved compressible solvers for these challenging cases.

I'm on the organising committee of the next High-Lift Prediction Workshop and it would be fantastic to see more OpenFOAM contributors (as well as other CFD codes) as whilst OpenFOAM has really been adopted by the automotive and motorsport sectors I don't see the same happening with the aerospace sector at present. Perhaps this is because these improved compressible solvers are not in the general release yet?

Thanks for your thoughts and happy to answer any other questions.

[FMDenaro]

Originally Posted by nash85

No problem at all Bruno! This publication certainly does not have all the answers and raises more questions than it answers, but I hope this can at least give a push to those OpenFOAM users to try out their improved compressible solvers for these challenging cases.

I'm on the organising committee of the next High-Lift Prediction Workshop and it would be fantastic to see more OpenFOAM contributors (as well as other CFD codes) as whilst OpenFOAM has really been adopted by the automotive and motorsport sectors I don't see the same happening with the aerospace sector at present. Perhaps this is because these improved compressible solvers are not in the general release yet?

Thanks for your thoughts and happy to answer any other questions.

I think that one of the reasons is that in many aerospace projects the CFD code must be certificated and, often, the agencies and research centres work using only shared codes. On the other hand, High-Lift prediction is such a flow problem for which only very specific formulations such as DES/LES makes sense to be used. Compressible extension of such formulations are also something at the frontier of the basic research.