[flotus1]

Anyone has or knows where to find some better examples how to write files using CGNS library? Preferably Fortran.

I got some of the examples from the documentation to work. Problem is: they are weird af, and leave out crucial bits of information. An example:
The only example for a transient write saves all data and metadata in memory, and then writes it all at the end. That's not what I would consider a practical approach, and I have trouble rewriting the example to continuously add to the database each time step. And thus have a readable file after each time step. The documentation and examples just leave too much to the imagination.
Another example: for writing a 3D vector result like velocity, I figured out you can write the 3 components individually, naming them e.g. VeloX, VeloY and VeloZ respectively. Paraview correctly parses that into a vector when reading the file, but I don't know if that's really how it should be done.

[aerosayan]

Originally Posted by flotus1

The only example for a transient write saves all data and metadata in memory, and then writes it all at the end. That's not what I would consider a practical approach.

I don't use CGNS, and don't have an exact answer for your query, but what you're describing seems like a buffered write. As per my understanding from your description, I'm assuming CGNS writes out the whole file at once, only in the end. The output is first stored in memory, then written to disk after the buffer gets full. These kinds of libraries generally have a method to flush out the buffer, i.e write everything in memory, to disk. For, standard c++ library writer std::cout, it's called std::flush.

Maybe there is such a functionality in CGNS, to flush the output to file?

[flotus1]

Not quite. It is pretty clear to me what the example does, and it has little to do with buffering. There are several distinct write commands for the data of every single time step. You could easily deallocate those parts of memory after writing the data for one time step.
My main issue is that they write the metadata (e.g. time values, pointers to individual time steps etc.) at the end in one go.

Edit: here is the interesting part of the code

    ! WRITE FLOW SOLUTION TO EXISTING CGNS FILE
    ! open CGNS file for modify
    call cg_open_f('grid.cgns',CG_MODE_MODIFY,index_file,ier)
    if (ier .ne. CG_OK) call cg_error_exit_f
    ! we know there is only one base (real working code would check!)
    index_base=1
    ! we know there is only one zone (real working code would check!)
    index_zone=1
    ! define 3 different solution names (user can give any names)
    solname(1) = 'FlowSolution1'
    solname(2) = 'FlowSolution2'
    solname(3) = 'FlowSolution3'
    ! do loop for the 3 solutions:
    do n=1,3
        ! create flow solution node
        call cg_sol_write_f(index_file,index_base,index_zone,solname(n), CellCenter,index_flow,ier)
        ! write flow solution (user must use SIDS-standard names here)
        if (n .eq. 1) then
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Density',r1,index_field,ier)
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Pressure',p1,index_field,ier)
        else if (n .eq. 2) then
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Density',r2,index_field,ier)
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Pressure',p2,index_field,ier)
        else
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Density',r3,index_field,ier)
            call cg_field_write_f(index_file,index_base,index_zone,index_flow, RealDouble,'Pressure',p3,index_field,ier)
        end if
    end do
    ! create BaseIterativeData
    nsteps=3
    call cg_biter_write_f(index_file,index_base,'TimeIterValues',nsteps,ier)
    ! go to BaseIterativeData level and write time values
    call cg_goto_f(index_file,index_base,ier,'BaseIterativeData_t',1,'end')
    nuse=3
    call cg_array_write_f('TimeValues',RealDouble,1,nuse,time,ier)
    ! create ZoneIterativeData
    call cg_ziter_write_f(index_file,index_base,index_zone, 'ZoneIterativeData',ier)
    ! go to ZoneIterativeData level and give info telling which
    ! flow solution corresponds with which time (solname(1) corresponds
    ! with time(1), solname(2) with time(2), and solname(3) with time(3))
    call cg_goto_f(index_file,index_base,ier,'Zone_t', index_zone,'ZoneIterativeData_t',1,'end')
    idata(1)=32
    idata(2)=3
    call cg_array_write_f('FlowSolutionPointers',Character,2,idata, solname,ier)
    ! add SimulationType
    call cg_simulation_type_write_f(index_file,index_base,TimeAccurate,ier)
    ! close CGNS file
    call cg_close_f(index_file,ier)

Sure, I could quickly rework that so it overwrites the whole metadata each time step. But that doesn't feel quite right, there has to be a way to append the info instead of keeping arrays of all the metadata up to this time step in memory. Or allocating them each time a new time step is written.

[flotus1]

By the way... am I even reading the right documentation here?
https://cgns.github.io/CGNS_docs_current/index.html

For pretty much all code examples in there, you can find various slightly more modern versions within the source files themselves under src/ptestrs, src/Test_UserGuideCode and src/tests.
The Fortran parallel example is the worst offender, because the version printed in the documentation can't even be compiled with a current version of CGNS. And what even is that example? It requires each process to write the cell-to-vertex connectivity in a global ordering scheme, which largely defeats the purpose as far as I am concerned.

[flotus1]

cgns_timing.png
I'm not quite sure what I am looking at here. Writing transient solutions, the more solutions have already been written to disk, the longer it takes for writing the next iteration.
In this case, the metadata was written in advance, timing only reflects writing of the raw data. On an NVMe-SSD capable of well over 1000 MiB/s write speed.
That can't be how this is supposed to work
Also, the more timesteps I write in total (i.e. more metadata written at the start) the longer it takes starting with the first time step. E.g. for 1000 time steps total on a 64³ grid, the first iteration already takes 3 seconds. While the CPU core this is running at is pegged at 100%. Presumably doing lots of string compares, what else could take this long?

Edit: for whatever reason, not closing the file in between writes gets rid of this performance issue entirely. I know that for regular files, closing them can force buffers being flushed to disk. But that's not enough to explain what is happening here.
Then again, all of the data remains in flight if I don't close the file. Which causes it to be unreadable if I abort the program.