Owls post-processing library for ipython notebook
 

Hi all,

I've just discovered Owls which is a python library for rapid exploration of OpenFOAM data https://github.com/greole/owls.git. (see announcement here)

First of all I would like to thank the author of this very promising tool: what a great addon to OpenFOAM post-processing :)

Gregor (the author) said that "The scope is to generate line and scatter plots from foam data in a simple way". But it's not clear for me if the goal of this library is although to generate this data directly with the library (outside the standard OpenFOAM utilities like sample, etc ...).

So Gregor, if you are readings those lines, can you tell us something on this ?
Thanks ! :)

Francois

Hi Francois,

Ok let me try to clarify things a bit more. The main motivation for me to start Owls was that I find it very cumbersome to plot OpenFOAM sets data with gnuplot and alike. For example every time you decide to sample a new field it probably breaks your gnuplot script since file names (because of the convention to have the sampled field names as file name) and the order/column numbers have changed. Owls aims to import all available sets (or eulerian, lagrangian data) into a pandas dataframe. Check out the 10 minutes intro on pandas http://pandas.pydata.org/.
Having all the data in memory as a DataFrame you can build post-processing scripts around that data. Of course that is something you can do in any bash script or custom tool, but I think python is good choice because of its rich ecosystem of library (numpy, matplotlib, scipy, ggplot to name only a few) and its clean syntax (but thats my personal opinion). And I am a big fan of the ipython notebook, so the bottom line is to bring OpenFOAM to the ipython notebook


Additionally Owls takes care to derive the right field names from your sets files and sets them as column names. Say if you you have a file "sets/10/axis_T.xy" you then can access the data by saying give me all "T" data, or data at time=10 or at the position axis. Velocity data is split into its components, e.g if you have "sets/10/axis_T.xy" you can access u,v,w.

Plotting the data works either by passing the desired data-columns to matplotlib or using the included .plot(), .scatter(), .show() functions. By easy plotting I mean that the things I am doing in 99% of my time should be straight forward. For example in 99% of my time I plot a given scalar over its coordinate, so I dont want to type .plot(x='Pos', y='T') all the time, hence 'Pos' is the default for x='Pos'. So you have to type only .plot(y='T') or even .plot('T').

There are other features i am working on (and are partially already available) like:

- consistent plot axis label. For example if I plot the velocity 'u' I want the axis to read 'Velocity [m/s]', So you only have to specify the label once and it will stay consistent. No more showing reports where you left accidentally "T [K]" but showing velocity. (implemented)

- data hashing, owls should hash data sets so you know that they are still the same. (partly working)

- reading decomposed data (partly working). I would like to read decomposed data directly and to generate sample lines directly in Owls but, without the need of reconstructing and sampling first. But probably python/Owls is to slow for that.

- etc ...

I hope that clears things a bit. I'll try to make a short screen cast on Owls usage if I find some time (probably not too soon :p). I guess seeing something in action is much simpler to understand than me having to write lengthy forum entries.

All the best
Gregor

PS: Have you had a look at the example notebook it provided under examples? Just start ipython notebook in that directory.

Thank you Gregor for this very helpful answer !
It's clearer now :)

I asked you this question because I was myself tired of using a similar workflow: sample + scripts to plot the data or even configure the sampleDict.

I recently thought about using the power of paraview python API (paraview.simple) within an ipython notebook to do my 1D and 2D post-processing to avoid using sample or other external tool. Honestly it was quite easy (even for me, and that's saying a lot :D) to define functions to load an OpenFOAM case and plot some data along lines or in slices within a notebook.

So when you said:

maybe paraview.simple could help you in this scope.

If I remember correctly, Bernard also made an interesting approach with his pyfoam library. Have a look here.

Thank you very much Gregor for your kind and precise answer. I wish you the best for your great project and I'm sure that the OpenFOAM community will warmly welcome it !

Regards,
François

Gregor,

I saw some recent activities on Owls git repository.
Does it mean that reading decomposed case and sampling from Owls is in progress ?

Thanks
Francois

Owls progress
 

Francois,

after being busy for a while with non-OpenFoam, i had some time to work on Owls some more.

The following things should be included in the upcoming version 0.7.

Python 3 support: In fact I myself use python3.4 only atm and it seems pretty stable. Python2.7 is also supported, but further testing is needed.

Improvement in plotting API. I am currently trying to simplify plotting of multiple data sets.

Paraview coupling: I included a script to embed snapshots generated with paraview. However the paraview-python library written by the kitware guys is not python3 compatible. For now paraview will be the way of choice to generate snapshots.

Reading decomposed data: It works reasonably well to read decomposed particle and eulerian field data. But at the moment connecting the data to the mesh is not implemented. I think I will spend the next weeks on getting the code into a reasonable shape for version 0.7 before working on the mesh stuff.

Anyway feel free to play around with the development version of 0.7 and feedback is always welcome.

Gregor

Great news Gregor and thank you for the feedback !

Regarding the Paraview coupling will you also use paraview-python for "in-owls" line sampling (not using the external "sample" application) ?

Many thanks for sharing Owls :)

Hi Gregor,

I love pandas and openfoam so I am very excited about Owls! :D

I have installed it and would like to use it to read my fields (in time directories) into dataframes so I can calculate average, standard deviations and plot histograms of the field data. Is this possible? Which command and syntax should I use?

Hi Nicole,

to start with owls, can do something like:

But be careful with large data sets, since owls tries to pack your data in memory per default.

The real benefit for me lies in the read_sets (for your sampled data, either in sets or postProcessing/sets) function and ow.MultiFrame.MultiFrame for multiple cases.