CFD Online Discussion Forums

CFD Online Discussion Forums (
-   Main CFD Forum (
-   -   Smallest binary file format to save large data (

Zonexo June 2, 2008 05:29

Smallest binary file format to save large data

I need to save the u,v and p data output from my CFD code. I only 've limited space on my school's server so I need to conserve disk space.

Which data format allows me to read/write data with min size?

I'm currently using fortran binary. I hope the file size can go smaller. I'm also using tecplot to visualize. Its file size is smaller, but I can't read out the data (only write).

Is CGNS better? Btw, I'm using fortran and working in linux.

Thank you!

Ananda Himansu June 2, 2008 19:50

Re: Smallest binary file format to save large data
The fortran binary format (form='unformatted') you use already saves you much space over ASCII. The only further savings could come from compression algorithms. General binary data will not shrink much if compressed, unless your flowfield has contiguous regions of identical values (an unlikely possibility except for extensive freestream regions, or extensive no-slip surfaces, and even there you must store the velocity as block and pressure as a block, not cell-wise). The only savings in going from fortran binary (form='unformatted') to C binary (form='binary') is the removal of the end-of-record markers. This should normally be negligible savings.

I am guessing that the reason that the Tecplot files are smaller is that Tecplot may be saving the data in single precision versus the binary precision that you supply. Certainly, if your future use of the data does not require that you retain the full precision of your calculations so far, you can convert your data from binary double precision (64 bit?) to binary single-precision (32-bit?). This is a lossy conversion. You could alternatively shrink the data set by a more controllable lossy compression algorithm that gives you control over the precision of the output. Note that Tecplot supplies (in the tech support part of their website) the C code that will take your dataset in memory and write it out into the format that their Preplot utility creates. By studying the C code you can learn how to read from the format, and not just write in that format.

Zonexo June 2, 2008 20:25

Re: Smallest binary file format to save large data
Oh thanks Ananda!

I now understand why my tecplot file is smaller. I don't think it is necessary to store the values in double precision so guess I'll store my real(8) data in real(4). Then maybe I'll use 7z or rar to compress them.

All times are GMT -4. The time now is 10:13.