Hi Guys, I'm about to start writing a pretty big CFD code to solve transonic flow about a moving airfoil. I have lots of experience in Fortran 90 and have dabbled a little in C++.

Which language would you suggest I use for a computationally expensive process? I have been told that Fortran usually leads to faster executables (i.e. less computation time for a given system) but some colleagues dislike Fortran and opt for C++.

I have no feelings either way - I would like to choose whichever program will give the fastest executable.

Could someone please offer some advice?

my PERSONAL choice is c++. If you ask me why, my answer would be felxibility.

Fortran = FORmula TRANslation

It was designed for, and continues to be optimized for, computationally intensive algorithms.

Go with what you know - use the FORTRAN.

This is an entirely religious debate, fought over many years with no hint of logic or evidence.

For ultimate speed, forget high level languages and use assembly.

Steve has put it rather well. Which is the "best" programming language for CFD has been argued ad nauseum on this forum. You can search the forum for many past discussions on this matter.

You'll be spending a lot of time coding and debugging - use the language in which you are most proficient.

... or look if it is possible to use an existing library like OpenFOAM to solve your problem much faster.

as others, use what is comfortable for you, for a good programmer, OOP etc (which are sound after cpp) are more concept, such that for an expert it's possible to implement it within fortran as well,

If the execution time is the only matter you concern, there is no difference between Fortran,C and C++. the difference of cpu time in different compilers of fortran (or c or c++) is much more that difference between c++ and fortran compiler of the same company. you can find many benchmarks on the internet. Hadian

This depends on what features you use. A straight fortran code compared to a C++ code exploiting OOP (i.e. when the code is really C++ and not C) then the speed difference can be considerable (fortran can be > 2 times faster).

You can do the same thing to fortran (using the same compiler); e.g. compare two codes that do exactly the samething but write one in straight fortran and the other using derived types and operator overloading. Another way of putting this is that the choice of programming method is more significant than the choice of compiler; eg. g77 can actually be faster than intel ifort if you exploit derived types and operator overloading in ifort (however on the same fortran code ifort will be faster).

This is mostly FUD. Whether a code is object oriented or not is mostly orthogonal to performance. It is easy to write C++ code which is inefficient or for which it is difficult to reason about performance, but that does not mean that OO is necessarily slow. Having extremely fine-grained objects is often inefficient, because it may put more stress on memory bandwidth or because of function-call overhead.

As an example of excellent performance from modern C++, see Eigen2. It may be very hard to write a library like this or to reason about it's performance because the compiler is doing so much manipulation, but it does use all the C++ features you claim are slow.

Assuming you are familiar with all languages involved, my advice is to write in C or C++ and use objects extensively for high-level components, but not use objects for the fine-grained stuff like individual elements of a mesh.

"As an example of excellent performance from modern C++, see Eigen2. It may be very hard to write a library like this or to reason about it's performance because the compiler is doing so much manipulation, but it does use all the C++ features you claim are slow."

So where's the equivalent optimized FORTRAN in those benchmarks? Also how well does the same code perform on a non-intel machine such as an NEC or IBM without any changes?

You're giving compilers too much credit for being able to fix/optimize the code. My example of comparing vanilla FORTRAN with a (also FORTRAN) program using derived types and operator overloading (two aspects of OOP) demonstrates that the compiler cannot rearrange the code to be optimal. The same thing happens when you compare C and C++. This has nothing to do with "badly" written code - it's an aspect of the compiler not been able to figure out what the best rearrangement of the code is in the more complicated case.

So where's the equivalent optimized FORTRAN in those benchmarks?

It's comparing favorably to vendor-optimized assembly. Ary you claiming that Intel's MKL could be sped up by rewriting it in vanilla Fortran?

using derived types and operator overloading (two aspects of OOP)

Indirect function calls (virtual functions) add less than 10 cycle overhead on modern hardware. If you use very fine-grained derived types and address them through the generic interface (i.e. you don't put the extra effort to resolve the calls statically) then this overhead may be significant, but using objects for your matrix, vector, preconditioner, Krylov accelerator, Newton solver, and mesh types does not incur any measurable performance overhead. You do several (like 8 or more) orders of magnitude more work each time you make these calls. Also, a preconditioner application or matrix-multiply usually calls MPI and every MPI implementation uses such polymorphism (they are written in C using object oriented design) which is still negligible compared to hitting the network or performing a system call.

If you use fine-grained objects, such as for the components of velocity at a point, or for each cell/face/vertex of a mesh then you'll have to be sure that all function calls are resolved statically (at compile time) and preferably inlined, otherwise performance will suffer. Even so, storing a mesh in this form will generally use more space, hence slow the application for memory bandwidth limited operations.

Also note that operator overloading has absolutely nothing to do with OOP and does not incur any overhead compared to the equivalent named functions (though I dislike it for other reasons). It is equivalent to function overloading which (in C++) just writes the types of the arguments into the name of the function (look at the symbols in the object file). Such functions are still statically resolved and incur zero overhead (except with shared libraries in which case the loader needs to resolve longer symbol names, but that only happens once and is irrelevant for scientific codes).

Operator overloading has a bad rap because it would appear to prevent rearranging computations so that lots of intermediate objects are not created. For instance, consider the expression <code>r = (a+b)(c+d)</code> where <code>a,b,c,d,r</code> are vectors and the operation is to be performed poinwise. Naive operator overloading will create as many as three temporary vectors, putting obscene pressure on the memory bus when the vectors are large, hence performing very poorly. Eigen2 uses a technique called expression templates which forces the compiler to perform the necessary transformations and produce the tight loop with no temporary vectors. Expression templates are reasonably advanced use of the C++ type system and can be tricky to debug (the error messages are hideous) but they are a prime counterexample to your claim.

"It's comparing favorably to vendor-optimized assembly. Ary you claiming that Intel's MKL could be sped up by rewriting it in vanilla Fortran?"

The Intel MKL are not in assembly language they're written in C as I recall with compiler directive inserted and some hand optimization (loop unrolling etc) + "trimming" of the resulting assembly language. None of the routines are written in pure machine code! And actually, for most of the operations they are benchmarking, the intel fortran compiler would produce code as fast (or faster) in just a few lines. Try it! Most of these operations are relatively simple to write in straight FORTRAN

If all of these features are so efficient and compilers are so good why do people (myself included) spend time adjusting/optimizing very large FORTRAN/C programs every few years when we purchase a new supercomputer. Running relatively small codes on PC's is clouding you're judgement - you need to look at portability across different processor architectures.

If large CFD codes in FORTRAN need hand optimizing to help the compiler on different platforms how much work do you think C++ code with all the bells and whistles would take?

And if you want a "proper" OOP language you should use Ada and not C++

they're written in C as I recall with compiler directive inserted and some hand optimization

SSE intrinsics are almost the same as assembly. Such code is definitely optimized by reading the produced assembly, it's just often easier to write and maintain such assembly by having the compiler produce it from C+intrinsics.

the intel fortran compiler would produce code as fast (or faster) in just a few lines.

Yes, the BLAS is easy to implement in Fortran (or just download it from netlib), but people still use vendor BLAS because it's a hell of a lot faster than compiling the reference BLAS (F77), even with good compilers. After the usual loop transformations and tuning your block/tile sizes, you'll likely end up with something similar to ATLAS, which isn't that good. Without SSE-style intrinsics or other compiler extensions, you don't have a way to exploit alignment and vector instructions beyond what the compiler can do automatically.

Note: BLAS beyond level 1 is irrelevant for most CFD applications since we don't have large dense matrices (if you do then you fail the scalability test). Level 1 is memory bandwidth limited for large vector sizes so your choice of BLAS has little impact on performance. Eigen2 is just an example of modern C++ delivering excellent performance (possibly better than any existing Fortran implementation).

Running relatively small codes on PC's is clouding you're judgement

My experience is limited to sub-4000 CPU machines with x86, x86_64, and POWER4/5. While it's not all possible architectures, they are the most widely used today. But, the architecture has nothing to do with language choice or OOP, provided reasonable compilers are available (they are, on every viable HPC platform for each language we are discussing). Optimizing for such architectures has everything to do with the memory architecture, which you deal with in the same way from each language. Furthermore, parallel scalability is all about the algorithm, hence the language is doubly irrelevant.

If large CFD codes in FORTRAN need hand optimizing to help the compiler on different platforms how much work do you think C++ code with all the bells and whistles would take?

The most relevant optimizations concern the memory hierarchy and are orthogonal to which language or design principles are being used. Good software design will make it easier to identify the hotspots and maintain hand-tuned versions for different architectures and compilers. This can extend arbitrarily deeply, for example you may have a structure (e.g. matrix, mesh, something needed by the preconditioner) which needs to be stored differently to exploit the memory architecture on the BG/P versus the XT5. With appropriate use of OOP, it is not difficult to keep both versions around and verify their correctness as the code evolves. If you are not using OO design, then many parts of your code may depend on the storage format in which case you essentially fork the project when optimizing for each architecture.

I agree with you that a weakness of "modern" C++ relative to C and Fortran is that it's more difficult to reason about performance, but this is not because of compiler inadequacies.

And if you want a "proper" OOP language you should use Ada and not C++

I'm not a fan of C++ or it's object system, but it's not because of performance.

My final comments on this:-

"SSE intrinsics are almost the same as assembly. Such code is definitely optimized by reading the produced assembly, it's just often easier to write and maintain such assembly by having the compiler produce it from C+intrinsics."

But you don't need to add SSE intrinsics yourself the FORTRAN compiler will do it for you. A lot of compiler optimizations are actually just hints to the compiler (i.e. compiler directives) which look like comment lines (bit like openmp). Nobody should need to write assembly language to do there scientific computing (especially with FORTRAN.

Also the use of BLAS etc can actually cause code to run slower; i.e. if you write the code to use the BLAS library excessively you will miss many optimizations for your specific problem.

My experience is limited to sub-4000 CPU machines with x86, x86_64, and POWER4/5

Why bring the Ghz rating into this? it's only a meaningful number if you are talking about the same processor family! I've used a 500Mz machine whose CPU's are more than twice as fast as a 2Ghz x86 - in real terms not benchmarks. The numbers not even meaningful when comparing a x86 to a P5.

"The most relevant optimizations conc ..."

yes memory optimizations are important but your example is misleading - we usually know the optimal order a priori in FORTRAN and so it rarely changes between platforms. What usually happens is we either waste some temporary memory so that the loops can be rearranged for pipe-lining or try to reduce it for caching. In many cases the FORTRAN compiler will try to do some of this for you.

I agree with you that a weakness of "modern" C++ relative to C and Fortran is that it's more difficult to reason about performance, but this is not because of compiler inadequacies.

I didn't say it was an inadequacy of the compiler I said you were expecting too much from it - each language has it's strengths and weaknesses.

It is amusing though that you need to download all these classes and libraries to make C++ as fast a pure, out the tin, FORTRAN for scientific computing though.

But you don't need to add SSE intrinsics yourself the FORTRAN compiler will do it for you.

Such intrinsics can exploit knowledge that is not available from Fortran, such as that a particular array will always be 16-byte aligned. You can also give application-aware prefetch instructions.

Also the use of BLAS etc can actually cause code to run slower

It's well known that BLAS is terrible for small matrices and vectors, but if your objects are large and the operation you are doing is exactly a BLAS operation (not a variant or combination), then the BLAS implementation is broken if it doesn't perform at least as well as your code.

Why bring the Ghz rating into this?

I didn't, that was a processor count.

we usually know the optimal order a priori in FORTRAN

Which order is better has nothing to do with it being Fortran, C, or otherwise. A C compiler has all the same opportunity for loop optimizations as the Fortran compiler has, in many cases, the intermediate representation that the optimizer works with is identical.

each language has it's strengths and weaknesses

Yes, but in this case the differences are mostly orthogonal to performance, they have to do with maintainability (this includes who knows the language), library support, etc.

Considering that the intermediate representation is typically the same, Fortran's historic edge comes from explicitly prohibiting aliasing while C and C++ allow it. C99 adds the <code>restrict</code> qualifier which nullifies this advantage. Although <code>restrict</code> is not in the C++ standard, many compilers support it.

Fortran is faster than c++???

hoho,

how do you know???

from words of mouth???

how did they know???

did they show you solid comparison of speed??

or they also learned this from word of mouth???

come on,

how can c++, invented 1970 by the brilliant computerist, be a lot slower than fortran, which was invented in 1950s

and, just use your common sense --since you have lots of programming experience, how can a computer code which largely formed by loops be slower than the same code which have the same loops but differ in a couple of time-negligible operations???

100 times more importantly,

fortran promotes ugly programming style;

in contrast, c++ promotes beautiful programming;

anyone, if she/he tells me she/he is using fortran and knows little c++, my first response is this is a poor person in scientific computing or this person was ill-educated or ill-informed by some old researchers who know nothing about CFD and programming.

10% improvement in speed means nothing;

even 200% improvment in speed means NOTHING

because

1. codes in scientific computer are not intended for real-time situations

2. by selcting a better numerical method, your code could be 10 times faster; for example, multigrid is a lot faster, have you used it? spectral is a lot faster, have you used it???

3. parallelization can improve the speed of your code by 10 times, 100 times -- as long as you have resources and proper algorithms/implementations

4. how to save your time during development, maintetance of codes, is lot more significant. you have to understand, most of codes we developed eventually will be thrown into trash cans. so code developing is a temparary thing and focus on how to save your own time, not for the computer.

since you have a large project to do, and you have many years experience , fortran is the first choise.

since your code is for the computation, both language will do the same thing.

you can find a lot of code written in fortran to do the cfd