CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (https://www.cfd-online.com/Forums/openfoam-solving/)
-   -   2 errors: "error in IOstream ... operator<<" and "sigFpe in dieselFoam" (https://www.cfd-online.com/Forums/openfoam-solving/78047-2-errors-error-iostream-operator-sigfpe-dieselfoam.html)

Rickard.Solsjo July 12, 2010 02:11

2 errors: "error in IOstream ... operator<<" and "sigFpe in dieselFoam"
 
Hi, does anyone know what this means.
'
[0] [2] [3]
[0]
[0] --> FOAM FATAL IO ERROR:
[0] error in IOstream "/disk/global/home/rsolsjo/OpenFOAM/rsolsjo-1.6.x/run/RealCase3/processor0/0.00032/T" for operation Ostream& operator<<(Ostream&, const char)
[0]
[0] file: /disk/global/home/rsolsjo/OpenFOAM/rsolsjo-1.6.x/run/RealCase3/processor0/0.00032/T at line 101267.
[0]
[0] From function IOstream::check(const char*) const
[0] in file db/IOstreams/IOstreams/IOstream.C at line 91.
[0]
FOAM parallel run exiting
[0]

[2]
[2] --> FOAM FATAL IO ERROR:
[2] error in IOstream "/disk/global/home/rsolsjo/OpenFOAM/rsolsjo-1.6.x/run/RealCase3/processor2/0.00032/N-DODECANE" for operation Ostream& operator<<(Ostream&, const Scalar&)
[2]


and

[2] #0 Foam::error::printStack(Foam::Ostream&) in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libOpenFOAM.so"
[2] #1 Foam::sigFpe::sigFpeHandler(int) in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libOpenFOAM.so"
[2] #2 __restore_rt at sigaction.c:0
[2] #3 pow.L in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/applications/bin/linux64IccDPOpt/dieselFoam"
[2] #4 Foam::RanzMarshall::Nu(double, double) const in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"
[2] #5 Foam::RanzMarshall::relaxationTime(double, double, double, double, double, double) const in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"
[2] #6 Foam::parcel::setRelaxationTimes(int, double&, Foam::Field<double>&, double&, Foam::Field<double>&, Foam::spray const&, double, Foam::Vector<double> const&, double, double, Foam::Field<double> const&, Foam::Field<double> const&, double) in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"
[2] #7 Foam::parcel::move(Foam::spray&) in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"
[2] #8 void Foam::Cloud<Foam::parcel>::move<Foam::spray>(Foam: :spray&) in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"
[2] #9 Foam::spray::move() in "/disk/global/szrobi/OpenFOAM/OpenFOAM-1.6.x/lib/linux64IccDPOpt/libdieselSpray.so"

Thanks!!!

wyldckat July 12, 2010 06:21

Greetings Rickard,

It seems that you are running in parallel, and that the path you are sharing between the various instances of the solver is read-only. I.e. probably "/disk/global/" is shared with read-only options, or it failed to comply with the remote write request due to file size or broken network link.
If it's a broken network link, it could be due to some glitch in the version of remote file sharing technology you are using (NFS, samba, sshfs, etc...)

Best regards,
Bruno

NorbertB July 12, 2010 06:36

I had the same error (first one) and it seems your run has been killed by the system, perhaps due to an overflow or a quota limit.

No idea for the second one.

wyldckat July 12, 2010 06:42

It looks to me that the two errors are related, since both refer to "Foam::Ostream&"!

wiedangel July 11, 2011 05:02

Quote:

Originally Posted by NorbertB (Post 266857)
I had the same error (first one) and it seems your run has been killed by the system, perhaps due to an overflow or a quota limit.

No idea for the second one.

I think you are right about the quota limit ... I get the same error when the disk is full ... :(

arunsmec November 11, 2012 08:48

Same error
 
I got the same error while calculating average (i'm doing LES). But the error comes after a number of time steps (~3500). MY disk is not full. It has more than 25 GB free space. What could be the reason?

wyldckat November 11, 2012 10:19

Hi Arun,

Quote:

Originally Posted by arunsmec (Post 391477)
I got the same error while calculating average (i'm doing LES). But the error comes after a number of time steps (~3500). MY disk is not full. It has more than 25 GB free space. What could be the reason?

There are two types of quota limit:
  • Space limit, where one might only have a limit of 10GB on the shared filesystem.
  • File count limit, where the limit is actually on the number of files associated to your account.
I remember there was a thread that detailed a bit how one can offload data from the cluster during runtime, to avoid reaching the file count quota... here you go: http://www.cfd-online.com/Forums/ope...ne-folder.html

Best regards,
Bruno

arunsmec November 11, 2012 12:09

Thank you wyldckat for the quick reply! :)
I'll try this after my current run. I have solved the issue by creating more space in my disk.

But I would also like to know if it is possible to write outputs from different processors into a single file for a given time ?

wyldckat November 11, 2012 16:44

Quote:

Originally Posted by arunsmec (Post 391503)
But I would also like to know if it is possible to write outputs from different processors into a single file for a given time ?

Do you mean that you want to write the values during the parallel run? If so, that would require the master process to be doing the writing, which would slow down the run, since the master process would be delayed because it had to do more output than the other slave processes.

arunsmec November 12, 2012 01:15

The problem still remains unsolved. The data files are eating the space on my disk. The details are as follows

LES of round jet
no of cells ~ 6 million
no of processors ~ 8
delta T = 5 e-5
writeinterval = 200
fields being written : U, U_0, Umean, Uprime2mean, P, P_0, Pmean, Pprime2mean, phi, phi_0, nuSgs
size of files/processor/writeinterval:
P files ~ 8 MB
U files ~ 26 MB
phi ~ 25 MB
Uprime2mean ~ 55 MB


I am half way through my simulation. The data files altogether has a size of about 90 GB. I had to kill the process. If I want to proceed, I have to free some space and also make sure unwanted details are not being written. As I understand U_0 and phi_0 are fields of previous timestep required for restart. Is it necessary to save these files? Can i remove these files from the directories corresponding to earlier time steps?

Bernhard November 12, 2012 02:53

First, what does writeFormat and writeCompression read in your controlDict? Save your data at least in binary form, and maybe compressed. You can change the format of existing files using the utility foamFormatConvert.

You can also consider using purgeWrite, to cyclic overwrite of old files.

To answer your question, as long as you don't want to restart from a certain time-step, it is safe to remove any of the files in older time-steps. I think there is even a functionObject that controls the fields that are written during outputtimes.

arunsmec November 12, 2012 03:18

Thank you Bernhard
I am writing in ascii format without any compression. How much space I can save by compression?
I have not tried compression before. As compression stores files in zip format, should I decompress the files before executing the reconstructPar?

If I have to restart from time step t1, can I remove the *_0 fields from time steps older than t1?

Bernhard November 12, 2012 03:31

Also try to store in binary, it can save you a lot of disk space, but that depends on your case. In my experience it can save up to a factor of 6 in disk-space.

All OpenFOAM utilities can handle either binary and/or compressed data (as they all use the same readers). Some third-party tools like PyFoam cannot handle reading the binary files to my knowledge.

If you don't need the _0 files for anything, it is safe to remove them, but I would do that only when you are certain that you don't want to restart from an earlier time.

arunsmec November 12, 2012 05:00

Is paraview/paraFoam able to handle binary/compressed data

Bernhard November 12, 2012 05:01

Yes.
Tecplot can also handle it, as far as I know, except for binary meshed. In case of problem you can always return to ascii/uncompressed with foamFormatConvert -time.

f0208secretx August 8, 2015 13:46

I've run into similar problem when running openfoam jobs. I'm sure I still have plenty of disk space and there shouldn't be for the file count.

The error is not completely the same as my Openfoam was manually installed in a folder I have read/write access, and I've also taken care in env $TMPDIR to make sure I have read/write access to this too. However, the simulation will spit out IO error as attached once in a while and I have to restart the simulation from that time step, its very annoying..

system info:
OpenFOAM 2.4.0
CentOS 6.6
64 cores run (4 nodes with 16 cores each) using OpenMPI
pimpleFoam with SADDES turbulence model

I've enabled openmpi debugging. error section in my log:
Code:

[41]
[41]
[41] --> FOAM FATAL IO ERROR:
[41] error in IOstream "/scratch/users/jw969/OpenFOAM-run/SADDES_head_v10mph/processor41/1.4068/Ug" for operation Ostream& operator<<(Ostream&, const UList&)
[41]
[41] file: /scratch/users/jw969/OpenFOAM-run/SADDES_head_v10mph/processor41/1.4068/Ug at line 22.
[41]
[41]    From function IOstream::check(const char*) const
[41]    in file db/IOstreams/IOstreams/IOstream.C at line 99.                                                                          [41]                                                                                                                                    FOAM parallel run exiting
[41]
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 41 in communicator MPI_COMM_WORLD                                                                          with errorcode 1.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.                                                                      You may or may not see output from other processes, depending on
exactly when Open MPI kills them.


wyldckat August 15, 2015 12:40

Greetings f0208secretx,

If the error occurs sporadically, than my first guess would be that there is a time out problem or write error with the file/folder sharing system you are using in your cluster. You have 64 processes, 16 per machine, all writing at the same time to a remotely shared folder "/scratch".

My other guess would be that this occurs only when a bad value is being converted to be written to file. A few questions:
  1. Are you using "ascii" or "binary" mode defined in "system/controlDict"?
  2. Are you using "compressed" or "uncompressed" in "system/controlDict"?

Best regards,
Bruno

f0208secretx August 16, 2015 09:35

I now believe its a file count problem. I will report if this continue to happen. Thanks.


All times are GMT -4. The time now is 23:23.