CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (http://www.cfd-online.com/Forums/openfoam-solving/)
-   -   OpenFoam logfile compression tip (http://www.cfd-online.com/Forums/openfoam-solving/63898-openfoam-logfile-compression-tip.html)

makaveli_lcf April 23, 2009 02:37

OpenFoam logfile compression tip
 
Hi all!

Making simulations in OpenFoam sometimes I get really huge log files. In some cases it can result in hard drive freespace running out! But all information can be useful afterwards and I cannot just redirect output to /dev/null.
So it is necessary to compress output on the fly. One solution in Linux OS is piping it to gzip/bzip2:
openfoamSolver 2>&1 | gzip > log.gz
But worst thing is that gzip blocks log.gz pipe for reading during simulation and one cannot get access to log data until task is finished! The solution is to use "compress" package. To obtain it in Ubuntu use
sudo apt-get install ncompress
Then it is possible
openfoamSolver 2>&1 | compress > log.Z
or make run script, containing
Code:

#!/bin/sh

checkMesh

if [ $? -eq 0 ]; then
    nohup openfoamSolver 2>&1 | compress > logfileName.Z &   
else
    exit -1
fi

Then you have a simulation running at background. Have a coffee break before coming back and analysing results!:)

jose_rodrig January 18, 2011 05:53

Continue with logfile
 
Hi! great post tip, btw..

My problem is a bit different though: I would like to know how it is possible to continue a logfile of a simulation that was interrupted and I wish to resume with the same logfile, without erasing the previous timesteps.

Thx

Jose

romant January 18, 2011 06:35

Quote:

Originally Posted by jose_rodrig (Post 290918)
My problem is a bit different though: I would like to know how it is possible to continue a logfile of a simulation that was interrupted and I wish to resume with the same logfile, without erasing the previous timesteps.

This can be done with
Code:

foamProgram >> logFile
the double error uses an already existing file. since the simulation will start from the last saved folder, you should delete everything that is written in the logfile after the folder save point.

jose_rodrig January 18, 2011 06:39

Great! many Thx

Regards

eelcovv January 18, 2011 10:15

Compress VTK files
 
Thanks for the compress tip!

In line with compression of data files, another question.
I commonly create output sample planes using the function object libfieldFunctionObjects.so. The problems is that the planes are only written in uncompressed ascii, so a lot of data is generated (especially for isocontour).
An example

Code:

PlanesVTK
{functions
(

      enabled  true;
      type surfaces;
      setFormat raw;
      surfaceFormat vtk;
      writeFormat binary;
      timePrecision 0;
      interpolationScheme cellPointFace;
      writeCompression compressed;
      functionObjectLibs ("libsampling.so");
      outputControl  timeStep;
      outputInterval  1;
      fields
      (
        magVorticity
      );
      surfaces
      (
        iso_magU0p5
          {
            type        isoSurfaceCell;
            isoField    magU;
            isoValue    0.8;
            interpolate true;
            regularise  false;
          }
      );
  } // END PlanesVTK is
);

creates asci data files of more then 40 MB, whereas I can compress it for almost 80%. The compression write option does not seem to work. Does somebody has an idea how to compress these kind of data output files during the run ?.
Thanks!

Regards,

Eelc

matteoL September 30, 2015 08:14

Hello,
have you had any luck exporting slices in vtk binary format?Thanks,
matteo

KateEisenhower September 1, 2016 08:29

Hello,

this is very interesting and may be of great helpt to me.
The problem is I don't really understand how you proceed with this compressed file. I mean normally one wants to extract data from this file during runtime (e.g. for plotting residuals). For doing this I need to extract the file and I am as far as before.
What am I missing here?

Best regards,

Kate

Quote:

Originally Posted by makaveli_lcf (Post 213870)
Hi all!

Making simulations in OpenFoam sometimes I get really huge log files. In some cases it can result in hard drive freespace running out! But all information can be useful afterwards and I cannot just redirect output to /dev/null.
So it is necessary to compress output on the fly. One solution in Linux OS is piping it to gzip/bzip2:
openfoamSolver 2>&1 | gzip > log.gz
But worst thing is that gzip blocks log.gz pipe for reading during simulation and one cannot get access to log data until task is finished! The solution is to use "compress" package. To obtain it in Ubuntu use
sudo apt-get install ncompress
Then it is possible
openfoamSolver 2>&1 | compress > log.Z
or make run script, containing
Code:

#!/bin/sh

checkMesh

if [ $? -eq 0 ]; then
    nohup openfoamSolver 2>&1 | compress > logfileName.Z &   
else
    exit -1
fi

Then you have a simulation running at background. Have a coffee break before coming back and analysing results!:)



All times are GMT -4. The time now is 04:19.