Which result file format to choose for in-house CFD code

flotus1 · August 1, 2020, 03:59

SCNR

Simbelmynë · August 1, 2020, 05:05

Quote:

Originally Posted by arjun

I want to create Salome to Fluent converter.

How about the OpenFOAM route? (Salome->OF->Fluent)

arjun · August 1, 2020, 05:56

Quote:

Originally Posted by fjois

This one thing is bothering me and I am intending to create an open source simple format that solvers shall be able to write easily.

It is very much needed.

pcosta · September 21, 2020, 19:00

My data layouts are simple, so I typically write field data into a raw binary format onto a single binary field using MPI I/O, which works like a charm for at least 10^10 grid points. Then I use a xdmf (http://www.xdmf.org) metadata file written with the aid of a python script to visualize the binary data in paraview. If you are interested in how I do it I can write more techical details.

In your case I would go for HDF5 or raw binary data and load it in paraview with xdmf.

flotus1 · September 22, 2020, 03:32

Thanks for all your valuable input. Here is what I implemented for now, ugly hacks for the win I guess

The solver writes the minimum amount of values (basically just pressure and velocity) into raw binary files. For post-processing, I have a different program that can stitch this database together into a single Ensight gold database.
While that may sound like a totally unnecessary step, it has some advantages to me:

minimal disk space usage until I decide to run the post-routine
minimum possible slowdown of the solver due to writing results
if necessary, I can only transform a subset of all time steps into PV-readable format. Saves disk space and time
it gives me free reign over any derived quantities I want to evaluate, like vorticity, lambda2, Q-criterion, density gradients... Not a big fan of letting ParaView compute these. And since the solver is outside of my area of responsibility, letting it write this kind of derived quantities would have been tedious to say the least.
I can switch between cell data and point data without a PV filter. I prefer point data, because most of the animations I create use some sort of iso-surface. My colleague wants cell data because he does not trust the interpolation for cell to point data...

I know, I know, some of these advantages may sound silly to some of you. But it works for me, at least for now. Maybe I will regret this decision in a few months or years. But that's a problem for future me.

flotus1 · March 10, 2021, 14:40

Future me here, sooner than past me was hoping for

Turns out ensight gold has some hidden limitations, at least when combined with ParaView. I could probably patch around them, but don't really feel like it. Adding even more workarounds to an already ugly solution won't make things easier in the long run. Or is that just how programming works?

So I guess it's time for CGNS+HDF5 now. Combing through the documentation for about an hour did not exactly boost my confidence levels. Wish me luck.

arjun · March 10, 2021, 15:27

While you are going through it, could you please make notes about how to use it (in simple manner). Could be then used by others.

Also do share the experiences, its learning for others.

flotus1 · March 12, 2021, 09:00

Quote:

Originally Posted by arjun

could you please make notes about how to use it (in simple manner)

Simple is all I can do anyway

I just love how they found it necessary to include detailed installation instructions for Windows in their FAQ section, but not for Linux. Or at least the section for Linux is hidden so well that I haven't found it yet.

Forgot to mention what kind of limitation I ran into with ParaView ensight gold reader: it fails as soon as (number of elements x number of nodes per element) is larger than 2^31 in a single part. Even if the actual number of nodes is lower because adjacent elements share the same nodes.
In my case with hexa8 elements, that threshold is around 268 million elements.

arjun · March 13, 2021, 12:36

I wonder if it is possible to create our own importer. I tried to find on it but could not find much on to it.

aerosayan · March 13, 2021, 16:50

I use ASCII VTK format for my personal projects as that works for Paraview.
I don't know if ASCII would be suitable for 10^9 cells. Binary format would be best.

Regarding the multiple level requirement, I plan to write the cells from the highest layer first, and then as we go down, write only cells that are not hidden by the higher layers.

It will finally look like a quadtree grid.

August 1, 2020, 03:59		#21
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,399 Rep Power: 46	SCNR sbaffini, arjun, Simbelmynë and 1 others like this.

September 22, 2020, 03:32		#25
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,399 Rep Power: 46	Thanks for all your valuable input. Here is what I implemented for now, ugly hacks for the win I guess The solver writes the minimum amount of values (basically just pressure and velocity) into raw binary files. For post-processing, I have a different program that can stitch this database together into a single Ensight gold database. While that may sound like a totally unnecessary step, it has some advantages to me: minimal disk space usage until I decide to run the post-routine minimum possible slowdown of the solver due to writing results if necessary, I can only transform a subset of all time steps into PV-readable format. Saves disk space and time it gives me free reign over any derived quantities I want to evaluate, like vorticity, lambda2, Q-criterion, density gradients... Not a big fan of letting ParaView compute these. And since the solver is outside of my area of responsibility, letting it write this kind of derived quantities would have been tedious to say the least. I can switch between cell data and point data without a PV filter. I prefer point data, because most of the animations I create use some sort of iso-surface. My colleague wants cell data because he does not trust the interpolation for cell to point data... I know, I know, some of these advantages may sound silly to some of you. But it works for me, at least for now. Maybe I will regret this decision in a few months or years. But that's a problem for future me.

March 10, 2021, 14:40		#26
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,399 Rep Power: 46	Future me here, sooner than past me was hoping for Turns out ensight gold has some hidden limitations, at least when combined with ParaView. I could probably patch around them, but don't really feel like it. Adding even more workarounds to an already ugly solution won't make things easier in the long run. Or is that just how programming works? So I guess it's time for CGNS+HDF5 now. Combing through the documentation for about an hour did not exactly boost my confidence levels. Wish me luck. arjun likes this.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Custom Thermophysical Properties	wsmith02	OpenFOAM	4	June 1, 2023 14:30
[swak4Foam] groovyBC in openFOAM-2.0 for parabolic velocity bc	ofslcm	OpenFOAM Community Contributions	25	March 6, 2017 10:03
[Other] Adding solvers from DensityBasedTurbo to foam-extend 3.0	Seroga	OpenFOAM Community Contributions	9	June 12, 2015 17:18
friction forces icoFoam	ofslcm	OpenFOAM	3	April 7, 2012 10:57
"parabolicVelocity" in OpenFoam 2.1.0 ?	sawyer86	OpenFOAM Running, Solving & CFD	21	February 7, 2012 11:44

September 21, 2020, 19:00		#24
pcosta New Member Pedro Costa Join Date: Jul 2017 Posts: 9 Rep Power: 8	My data layouts are simple, so I typically write field data into a raw binary format onto a single binary field using MPI I/O, which works like a charm for at least 10^10 grid points. Then I use a xdmf (http://www.xdmf.org) metadata file written with the aid of a python script to visualize the binary data in paraview. If you are interested in how I do it I can write more techical details. In your case I would go for HDF5 or raw binary data and load it in paraview with xdmf.

March 10, 2021, 15:27		#27
arjun Senior Member Arjun Join Date: Mar 2009 Location: Nurenberg, Germany Posts: 1,272 Rep Power: 34	While you are going through it, could you please make notes about how to use it (in simple manner). Could be then used by others. Also do share the experiences, its learning for others.

March 13, 2021, 12:36		#29
arjun Senior Member Arjun Join Date: Mar 2009 Location: Nurenberg, Germany Posts: 1,272 Rep Power: 34	I wonder if it is possible to create our own importer. I tried to find on it but could not find much on to it.

March 13, 2021, 16:50		#30
aerosayan Senior Member Sayan Bhattacharjee Join Date: Mar 2020 Posts: 495 Rep Power: 8	I use ASCII VTK format for my personal projects as that works for Paraview. I don't know if ASCII would be suitable for 10^9 cells. Binary format would be best. Regarding the multiple level requirement, I plan to write the cells from the highest layer first, and then as we go down, write only cells that are not hidden by the higher layers. It will finally look like a quadtree grid.