[bastil]

Hi all,

because of this:
http://www.hpcwire.com/features/17885244.html?viewAll=y

it might make sense to generate a mixed sp/dp solver. I found all mesh operations to be very critical for meshes with short edge length but I want to do all calculations in single precision. How can this be done? As far as I can see this needs some quite basic changes in OpenFOAM?

Thanks and regards

Bastian

[alberto]

Hi,

what the article is saying is that they keep double precision accuracy, using single precision operations on the machine. Clearly I don't know the details of their implementation, however what they do seems to manipulate a double type as a set of floats, which, as a concept is not new. Similar ideas are used, with a different goal, in arbitrary precision libraries (GNU MP http://gmplib.org/ for example).

Best,

[bastil]

Originally Posted by alberto

Hi,
what the article is saying is that they keep double precision accuracy, using single precision operations on the machine

Exactly. And that is what I want to do in simpleFoam. I don't even mind double precision accuracy since I have small differences between single and double. However, I need to have mesh (saving, reading,...) in double since I have short edge lengths. Has anybody done something similar before?

Regards Bastian

[alberto]

Not that I know. In your case you actually care of double precision accuracy. If you need a mesh in double precision (can't you simply scale it? or is the aspect ratio extreme?), you need, for example, to compute differential operators in double precision too for example, or you end up in large errors.

Best,

[bastil]

Originally Posted by alberto

Not that I know. In your case you actually care of double precision accuracy. If you need a mesh in double precision (can't you simply scale it? or is the aspect ratio extreme?), you need, for example, to compute differential operators in double precision too for example, or you end up in large errors.

Alberto,

I don't really understand how you want to scale the mesh? Yes, I have high aspect ratios. I have the following observations:

• My high aspect ratio meshes run fine in simpleFOAM in DP
• The same meshes run fine in FLUENT in both SP and DP without remarkable result differences between SP and DP.
• FLUENT saves the mesh in DP regardless if you run SP or DP
• I see a speedup of nearly 50% when running in SP for both codes

So my aim is to run my model in OF using SP maths (to get the 50% speedup). That is why I came across this. Hints?

Regards Bastian

[eugene]

Hi Bastian,

Unfortunately, what you want is not possible without a hell of a lot of effort. Every single number in foam is templated on the scalar type, which can be compiled for either float or double, but not a mix of both (as you probably know). To have only the mesh work in double precision, you would have to redefine every single scalar or scalar derived variable used in the mesh. This would mean duplicating a massive number of foam data structures. This is not something you can do on the weekend.

However, arbitrary variable precision would be a really useful thing to have. I know that the code Xflow uses 10 bit precision as part of its acceleration strategy. It would be fantastic if we could get say 40% speed-up by doing the same kind of thing.

Eugene

[alberto]

Originally Posted by bastil

Alberto,

I don't really understand how you want to scale the mesh? Yes, I have high aspect ratios. I have the following observations:

If you have high aspect ratios in the geometry (not mesh) that are so extreme to require double precision to represent the geometry, you really have no choice than solving in double precision the whole problem, or you will obtain a solution seriously affected by the truncation error when switching from double precision (mesh) to single precision (your data). I have already given you an example where this happens.

One doubt/suggestion. What happens if you increase the accuracy used to write the mesh in OpenFOAM? Check in controlDict the writePrecision option. The default is 6 digits if you use ASCII format. As an alternative, try the binary format.

• The same meshes run fine in FLUENT in both SP and DP without remarkable result differences between SP and DP.

This seems contradictory with your statement that you need to represent your mesh in double precision, since the run in SP will be affected by the truncation error.

• FLUENT saves the mesh in DP regardless if you run SP or DP

Yes, but it will perform a conversion when working on SP data. There you will lose information. The question is: is this conversion performed while reading the mesh? FLUENT warns you when you load a 64-bit case (not mesh, whole case) in the 32-bit solver...

• I see a speedup of nearly 50% when running in SP for both codes

So my aim is to run my model in OF using SP maths (to get the 50% speedup). That is why I came across this. Hints?

The speedup in SP is normal. However, as Eugene said, there is no easy way. You will not only need to re-implement OpenFOAM structures, but you will have to code specific algorithms that take care your accuracy is not reduced excessively by the truncation (of course if you want to do it properly).

Alberto

[eugene]

A more viable option might be to translate only some specific matrix solution components to reduced precision and see how that works out. Interestingly, GMP is already included with the foam distribution as a Gcc dependency. Seems criminal not to take advantage of it!

You could also check whether it is possible to use gmp to template the scalars in foam onto something like a 48bit number. This obviously won't give you quite the speed-up of a 32bit float or the accuracy of a 64bit double, but the compromise might be acceptable and it would be much easier than re-writing large chunks of code.

Eugene

[deepsterblue]

Did it already. Not sure how useful this would be, but I guess it doesn't hurt to post it:

http://www.ecs.umass.edu/~smenon/tarballs/multiPrecision.tgz

Basically wraps around the MPFR library to provide arbitrary precision arithmetic. You can replace the scalar class with this.

[eugene]

Originally Posted by deepsterblue

Did it already. Not sure how useful this would be, but I guess it doesn't hurt to post it:

http://www.ecs.umass.edu/~smenon/tarballs/multiPrecision.tgz

Basically wraps around the MPFR library to provide arbitrary precision arithmetic. You can replace the scalar class with this.

Interesting. So you haven't tried replacing scalar with this yourself yet?

[deepsterblue]

Well, at least not at the linear-solvers level. The performance would go through the floor if I tried that, but I don't see why not.

I was mainly using it for geometric intersection calculations, since those tend to be finicky about round-off. Clearly, this is intended for numerical robustness, and not performance.

[eugene]

I will put it on my list of things to try. Thanks for the wrapper code Sandeep.