[flotus1]

I am in the process of deciding which file format for flow results we will use in the future for out in-house CFD codes. Currently, I only see one candidate that ticks most of the boxes. But I would be happy to hear more opinions on this.
Up until now, we have been using Ensight gold format. But with the switch from OpenMP to MPI, I can only come up with what appear to be ugly hacks in order to keep that file format.

Here are the requirements:

• Post-processing will be done in ParaView. So the file format should be well supported.
• Solver runs MPI parallel with domain decomposition. So a scalable MPI implementation for I/O is a must.
• Solutions are always transient with several (up to ~2000) time steps needed in the results. Plus a few time-averaged quantities
• Cell count: should be able to handle in the order of 10^9 cells. Results are cell-centered, no boundary values required.
• The mesh exclusively consists of Cartesian cubes (Lattice Boltzmann solver). A mesh can consist of several "levels", where each level has 2x the cube size of the last level. The data layout is unstructured/sparse.
• Support for moving meshes (rotation + translation) would be nice to have in the future. No deformation required
• Solvers are written in Fortran, so at least a Fortran interface should be available.
• Result types are scalars and 3-dimensional vectors.

So the question is: which file format ticks all the boxes, perhaps without over-encumbering me with a plethora of options that I will never use

[arjun]

You will be surprised to know that the answer is None.

This one thing is bothering me and I am intending to create an open source simple format that solvers shall be able to write easily.


1. Ensight Gold is most suitable in the sense that it supports polyhedral and is binary but also creates many files for each variable. Managing these files is pain in A. Also format is not easy to be written.
2. Fieldview - Not easy to be written by a solver and propriety format.
3. Tecplot with poly support, same issue as Fieldview.
4. VTK with xml. God knows how to write it. Too complicated.

5. HDF5 only 5 people know how to use it. Overly complicated.

Originally Posted by flotus1

I am in the process of deciding which file format for flow results we will use in the future for out in-house CFD codes. Currently, I only see one candidate that ticks most of the boxes. But I would be glad to happy more opinions on this.
Up until now, we have been using Ensight gold format. But with the switch from OpenMP to MPI, I can only come up with what appear to be ugly hacks in order to keep that file format.

Here are the requirements:

• Post-processing will be done in ParaView. So the file format should be well supported.
• Solver runs MPI parallel with domain decomposition. So a scalable MPI implementation for I/O is a must.
• Solutions are always transient with several (up to ~2000) time steps needed in the results. Plus a few time-averaged quantities
• Cell count: should be able to handle in the order of 10^9 cells. Results are cell-centered, no boundary values required.
• The mesh exclusively consists of Cartesian cubes (Lattice Boltzmann solver). A mesh can consist of several "levels", where each level has 2x the cube size of the last level.
• Support for moving meshes (rotation + translation) would be nice to have in the future. No deformation required
• Solvers are written in Fortran, so at least a Fortran interface should be available.
• Result types are scalars and 3-dimensional vectors.

So the question is: which file format ticks all the boxes, perhaps without over-encumbering me with a plethora of options that I will never use

[flotus1]

So far, CGNS+HDF5 was my top contender.
With caveats: cgns, in its attempt to be as general as possible, seems to be overly complicated and convoluted for my requirements. And you already mentioned the steep learning curve, especially when combining it with HDF5.

[arjun]

Originally Posted by flotus1

So far, CGNS+HDF5 was my top contender.
With caveats: cgns, in its attempt to be as general as possible, seems to be overly complicated and convoluted for my requirements. And you already mentioned the steep learning curve, especially when combining it with HDF5.

Just yesterday, I downloaded cgns from github, compiled it and wanted to read cgns exported from Salome.

I used this https://github.com/CGNS/CGNS/blob/ma...ad_grid_unst.c
This is only example I could find after long search.

Guess what it says that the file is invalid. So I checked the cgns by Salome is cgns (hdf5). So the cgns downloaded from github from people who develop it can not load the cgns files. What chances do you think we mere mortals have.

As I said only 5 people know how to use it. It is a format created to be so painful to use that it stays with 5 people who know it.

Like you I am also searching for such format. More and more I am coming to conclusion that I shall create my own simple to use format for CFD and should write a plugin to load into polyflow.

The format shall be easy enough to be loaded by simple C or C++ code that also should be provided with the format.

[Gerry Kan]

Hi Flotus:

If your results are presented on a very large uniformly spaced Cartesian grid, you might want to consider using netCDF. Paraview has a netCDF reader that will import the file, and the new version of netCDF is in fact a HDF5 wrapper so you don't need to know the HDF API to write to netCDF.

If you are adventurous, you can use parallel netCDF, with the drawback that you need to know how many timsteps to write at the beginning since it does not support unlimited dimension (e.g., time) as it does in the serial version. I have never used parallel netCDF myself, however.

The only thing you have to worry about is the row and column order as, like HDF5 and CGNS, netCDF is written in C. So netCDf is ijk, and Fortran is kji ... you get the idea.

Hope that helps, Gerry.

P.S. - The Fotran API for netCDF is actually quite straightforward but still reasonably flexible if what you do is to dump an array into a postprocessing (i.e., netCDF) file.

[flotus1]

If your results are presented on a very large uniformly spaced Cartesian grid

The only thing you have to worry about is the row and column order as, like HDF5 and CGNS, netCDF is written in C. So netCDf is ijk, and Fortran is kji ... you get the idea.

I was probably not precise enough about that part. The cells themselves are cartesian cubes, meaning that they are axis-aligned cubes, with equal spacing within a "level". But there will be more than one of these levels, meaning different cell sizes within one mesh.
And the data is not structured as in "ijk-coordinates". It is stored in a 1D sparse format to allow for arbitrary geometries, where each cell has its physical xyz-coordinates and knows its 26 direct neighbors.

Like you I am also searching for such format. More and more I am coming to conclusion that I shall create my own simple to use format for CFD and should write a plugin to load into polyflow.

I have been in similar traps already. Not fully content with what is available, I decided to do my own thing. How hard can it be... turns out pretty hard once you get into the details.
So this time, I want to give existing solutions a try at least. But it is good to know that I am not the only one who is intimidated of the poor documentation and general obscurity of readily available solutions for data export.

[Gerry Kan]

Originally Posted by flotus1

I was probably not precise enough about that part. The cells themselves are cartesian cubes, meaning that they are axis-aligned cubes, with equal spacing within a "level". But there will be more than one of these levels, meaning different cell sizes within one mesh.
And the data is not structured as in "ijk-coordinates". It is stored in a 1D sparse format to allow for arbitrary geometries, where each cell has its physical xyz-coordinates and knows its 26 direct neighbors.

bummer that there is refinement in your LB mesh makes everything more difficult ... at the very worst there is always point cloud (I jest, of course).

Gerry.

[flotus1]

Originally Posted by Gerry Kan

bummer that there is refinement in your LB mesh makes everything more difficult

Indeed. More difficult on so many levels . The MPI communication scheme with overlapping grid interfaces almost broke me.
But without grid refinement, most real-world applications would be out of reach.

[arjun]

Around year 2010 I was working with LES , DNS and we had our inhouse code (that i developed). That time our calculations were of sizes between 1Billion to 3 Billion cells.

Ours was cartesian mesh solver so we used Plot3D data format. Still Paraview needed lots of memory for iso surface generation and it was tough to get animation out from such large meshes.
So the solution that I did was that during the simulation I would use the fact that cartesian cell is normal cube, I would find the cut surface (iso surface) and store it as polygons into a binary file.
After the simulation was done, these files would be read by a simple program (C++) and i would then create a bmp file from it. The program was written such a way that it did not allocate any memory and it was processing as it was reading the polygons from the file.

These bmp files would be then processed into an animation file.
In your case such post processor could work.


--------------------------


As far as post processing solution goes, in short term we have no good choice. But slowly if few people come together then this could be solved.
Even if someone point me to anything that shows how to create mesh reader plug in for paraview, that could be an start. So far to me even this has been confusing.

[Gerry Kan]

Originally Posted by flotus1

=
But without grid refinement, most real-world applications would be out of reach.

Ausgerechnet, daß Du (wenn ich so darf) mit Lattice-Boltzmann beschäftigt bist. Bist Du zufällig mit der Entwicklung von Palabos zuständig?

Gerry.

[flotus1]

No, I have no affiliation with the Palabos team. Or any other LB solver you might have heard of. Let's hope that the second part of that statement will change in the future.

In your case such post processor could work.

I could probably get that to work. But for now it seems to lack the generality of a fully-fledged post-processor like ParaView. What if I don't know in advance which iso-surface I need. Or if I need something other than iso-surfaces.

[arjun]

Originally Posted by flotus1

No, I have no affiliation with the Palabos team. Or any other LB solver you might have heard of. Let's hope that the second part of that statement will change in the future.


I could probably get that to work. But for now it seems to lack the generality of a fully-fledged post-processor like ParaView. What if I don't know in advance which iso-surface I need. Or if I need something other than iso-surfaces.

Yes, Indeed. This is a problem needs to be solved.

Please do share what you end up doing since I (and many more) are in same boat.

In coming days I intend to take the mesh conversion tools out from Wildkatze solver and put it out since the mesh format is very easy to read and write from solvers.
Next step for me would be to see if a paraview plugin could be created as reader. If that happens then this issue is mostly solved.
(then hoping that more like minded people who favour simplicity over complex solutions join and use and develop it).

PS: Currently it can convert from starccm, openfoam and fluent mesh and can write fluent mesh as output that can be used with most solvers out there).

[JBeilke]

The people around "Flexi" https://www.flexi-project.org/ use HDF5. So there is at least someone in germany who knows, how to use it :-)

[flexi-project]

Yes, but we are far from the only ones. Many high performance computing CFD codes do indeed use HDF5 for their IO. To my knowledge, there is nothing else that works on large file systems. For medium to smaller ones, of course there are other options!

[arjun]

Originally Posted by JBeilke

The people around "Flexi" https://www.flexi-project.org/ use HDF5. So there is at least someone in germany who knows, how to use it :-)

Good for them, they have lots of patience. It would be nice if they shared their knowledge as to how they made it work.



Even in starccm I know we can't load hdf5 well so I believe flexi people hold some esoteric knowledge that is hard to acquire.

[arjun]

Originally Posted by flexi-project

Yes, but we are far from the only ones. Many high performance computing CFD codes do indeed use HDF5 for their IO. To my knowledge, there is nothing else that works on large file systems. For medium to smaller ones, of course there are other options!

BTW on your page the golf ball iso surface, is it for non rotating golf ball??

[flexi-project]

Originally Posted by arjun

BTW on your page the golf ball iso surface, is it for non rotating golf ball??

IIRC: yes, it is stationary.

To topic: I do not think that there is any esoteric knowledge required. The code is open source and available on github, feel free to have a look. All hdf5 routines are in /src/hdf5

[arjun]

Originally Posted by flexi-project

IIRC: yes, it is stationary.

I worked on golf balls simulations in past so was curious. We did balls with spin too.

Originally Posted by flexi-project

To topic: I do not think that there is any esoteric knowledge required. The code is open source and available on github, feel free to have a look. All hdf5 routines are in /src/hdf5

I will check, I want reader for cgns (hdf5) and so far no clear information. I hope src has something about how to read them. I will check, thank you.

[flexi-project]

if you are interested in reading cgns, check out our preprocessor at hopr-project.org

[arjun]

Originally Posted by flexi-project

if you are interested in reading cgns, check out our preprocessor at hopr-project.org

Thank you. This is what I am interested in. I want to create Salome to Fluent converter. They export in cgns format, that the cgns lib from their github ( https://github.com/CGNS/CGNS ) does not recognize.