[simrego]

Hi!


I'm just curious if anyone can provide some data about the performance profile of any OpenFOAM solver? I mean the time spent for the different functions. Now I'm interested in the equation formulation. So as i know for example let's assume some useless equation:

solve(fvm::laplacian(T)+fvc::div(U)+fvc::div(k)+fvc::div(someVariable))

If I'm right, it will create a matrix from fvm::laplacian(T) which will contains 3 vectors (ldu matrix and source), THEN it will create a vector from fvc::div(U) and loop through it and add to the matrix. Then a new vector for div(k), add it to the matrix with a loop, then new vector for div(someVariable) add it to the matrix. So I hope you already know the question.
Is it really feasible to create the equation system this way?
I just made a really dummy code in c++ to check the difference between four loop and add one by one and between a single loop but add four times. As i expected the results are clear. One loop, four addition wins. The second method is 2x faster. (In the code the clock starts after the vectors are initialized with some data.)


So are there any tests about it? I know this is a "small part" of the code but the difference is huge.

[HPE]

The performance questions are tricky. The outcome from a performance analysis is difficult to generalise, as you have attempted to deduce some insight from another piece of code (which is good by itself).

The steps you have elaborated in your post are mostly discarded by the compiler, which operates in a more clever way than a human can do.

For OpenFOAM, or any other CFD software, the bottleneck is not CPU, not the number of floating-point opers. You can confidently assume that the floating-point oper cost for OpenFOAM is free. The bottleneck is memory and caching.

But the most important challenge is the maintenance, for which the code must be maintainable, for which it must be expressive and intuitive for human readers.

My suggestion is that trust compiler and do not worry about the performance, which is good for OpenFOAM. I personally saw a 1.5 billion cell two-phase simulation for the Japan earthquake, and the weak scaling is virtually resembling to the ideal curve.

At least these are my opinions.

[simrego]

You call the constructor of fvc::grad so it will create this object. Then add it to the previous one, loop through the vectors, etc. I'm not an expert in programming but correct me if I'm wrong.
I know that the memory access for the CPU is problematic in these kind of software, but looping through the same vector multiple times and access its data multiple times while you could use it once, I think it is the same what you are talking about. Memory...
That's why I'm interested if are there any tests about that.

[HPE]

>> You call the constructor of fvc::grad so it will create this object. Then add it to the previous one, loop through the vectors.

That's what is coded, but that's not what the machine instructions that compiler and linker assemble. Even if you would code it differently, the compiler would produce the same instructions. I had raised this question once in stackexchange, and the top answer was telling me like "don't worry, compilers are smarter than you, a lot smarter".

So those are my opinions. Somebody else may provide more tangible insight.

[simrego]

<Deleted post>