[DLuo]

I've got an ensight data set written out by a 3rd party in Little Endian order.
However, when I read it into Paraview, it seems to pass both the Little Endian and Big Endian tests.
According to the vtkEnsight binary reader source code

// Just a sanity check. (0, 0 occurs often).
    // This condition would only occur for some really large files.
    if (tmpLE > 0 && tmpBE > 0)
      {
      vtkWarningMacro("Byte order is ambiguous.");
      }
 
    // If they are both valid, use the larger one.

It reverts to reading the data as a Big Endian file, which then results in a segmentation fault.

Because I can't get the data in Ensight Gold Format I'm stuck dealing with this vtkEnsight6BinaryReader.
Does anybody have any recommendations or suggestions?

[wyldckat]

Greetings Andy,

Which exact ParaView version are you using?

----
edit: I've done a very quick test in ParaView 4.1.0, but without a small test case, I'm unable to double check this.
The test was to use Python code to load the data directly with VTK:

import vtk
e = vtk.vtkEnSight6Reader()
e.SetByteOrderToLittleEndian

This is the farthest I went, because the last method exists and allows us to define the Endianness to be used by the reader, instead of relying on automatic definition.
I'm going to do some other tests with the ParaView Data sample, which provides EnSight Gold data and try to figure out how to use a Programmable Source to do a VTK data loading.
----

Best regards,
Bruno

[wyldckat]

OK, here's what I've done with some success:

1. Instead of "File->Open", I used "Sources->Programmable Source'.
2. In the "Output Data Set Type" drop-down box, I chose "vtkMultibockDataSet".
3. In the first big edit box "Script", I placed this source code:
   

   import vtk
   en = vtk.vtkEnSightGoldReader()
   en.SetFilePath('/home/user/ParaViewData-v4.0.1/Data/EnSight/')
   en.SetCaseFileName('elements.case')
   #en.SetByteOrderToLittleEndian() #commented out
   en.Update()
   
   self.GetOutput().ShallowCopy(en.GetOutput())

4. Then hit the "Apply" button.

It showed me the correct data, although I didn't have a nice interactive way to choose which data sets I wanted to load, but at least I could choose to render them or not.


I'll update this post, if I can find some non-Gold EnSight file... hopefully VTK has got one or two...

---------
edit: OK, found a test case in VTK 5.8's data package. The following code mostly worked, although it gave be an intriguing error message:

import vtk
en = vtk.vtkEnSight6BinaryReader()
en.SetFilePath('/home/user/VTKData/Data/EnSight')
en.SetCaseFileName('office6_bin.case')
#en.SetByteOrderToLittleEndian()
en.Update()

print en.GetByteOrderAsString() 

self.GetOutput().ShallowCopy(en.GetOutput())

The intriguing message was:

vtkSMPVRepresentationProxy (0x54e60c0): Could not determine array range.

But it seemed to have loaded the data correctly... er, no, the "vectors" field didn't get properly loaded.

If we uncomment the LittleEndian line, it won't load the data, as it's wrongly ordered. The "print" line will let us known which mode it is using for loading the data.

[DLuo]

Thanks for the reply Bruno,

I'm testing with version 4.1 on CentOS

I tried your example case with and without the en.SetByteOrderToLittleEndian()
Without it: I get the same warning and then the Segmentation Fault
With it: I just get Segmentation Fault

In both cases it appears that it doesn't get past line 6 because I get no output from either Paraview nor my CLI telling me the Endianness

[wyldckat]

Strange... are you using:

en = vtk.vtkEnSight6BinaryReader()
en.SetFilePath('/home/user/VTKData/Data/EnSight')
en.SetCaseFileName('office6_bin.case')

and properly defining the path and file name to the case file in question?

Perhaps the data is in EnSight 5?

Have you tried using the free version of EnSight? It's suppose to be limited, but perhaps it can help to either convert the data formats or at least validate if the data can in fact be loaded or not.

[DLuo]

Yes I was using the code you quoted above
I also tried importing as both MultiBlock and as Unstructured Data

I have similar data sets generated from the same source that read in fine; same solver, same analysis just a slight geometry change. So I don't think it's a matter of Ensight version.

The frustrating part is that I can read the data into other Ensight viewers but all my post Processing scripts are set up in ParaView and it's not trivial to port them over to something else.

Edit:
I don't have sys-admin privileges so can't install the free version of Ensight on my own

[wyldckat]

Mmm... it's possible that something is somewhat broken in the code that generates the EnSight data files.
Possibly the original data generation binaries are in 32 bit and ParaView is in 64 bit, leading to some array incorrect sizing or interpretation.

Do you know the array dimensions that are meant to be loaded on each case? Namely a good case vs a bad case? It's possible that one of them is over 2GiB integer value, leading to bad pointing issues.

Because the only other thing that comes to mind is to write an Endian converter for those data files

edit: By the way, what happens when you comment out the last line, namely the one that starts with "self"?

[DLuo]

Well the Ensight files read fine into Meta PostCFD and it's been confirmed to open in Ensight as well (not by me) so I don't think it's something with the generation of the Ensight file. And I'm pretty sure everything I've been dealing with is 64-bit.

I don't follow what you mean by array dimensions. Is there a quick way to check?

I'm hoping not to have to delve into an Endian converter. I tried playing with quick one-liners such as dd conv=swab but honestly I don't know what I'm doing there. From what I've read, I need more in depth knowledge about the structure of the binary file in order to do a proper Endian conversion.

Also when I comment out the last line beginning with "self" I get the same thing namely because the Segmentation Fault happens during the input portion of the code therefore it doesn't even get a chance to read that line.

BTW when I try that code with a "good" ensight data set, it throws errors if I leave in the line "en.SetByteOrderToLittleEndian()"

vtkEnSight6BinaryReader (0x2fbeb30): Invalid number of unstructured points; check that ByteOrder is set correctly.

Without it, it just prints BigEndian. The data doesn't actually get rendered.

Have I been misinformed? Are my data sets in fact Big Endian?

[wyldckat]

Originally Posted by DLuo

Well the Ensight files read fine into Meta PostCFD and it's been confirmed to open in Ensight as well (not by me) so I don't think it's something with the generation of the Ensight file. And I'm pretty sure everything I've been dealing with is 64-bit.

Mmm... OK, then it's probably more than just one bug in the ParaView/VTK reader.
What about asking the person who opened in EnSight to export into Gold format? As long as it's practical, that is... or at the very least, you can figure out if it's a lost cause or not, in case ParaView can't even open in Gold format.

Originally Posted by DLuo

I don't follow what you mean by array dimensions. Is there a quick way to check?

Check the number of points and/or number of cells reported by EnSight or Meta PostCFD. The dimensions of the arrays are directly associated, so it will only be a matter of multiplying by 2 or 3, in order to estimate the sizes of the arrays.

Originally Posted by DLuo

I'm hoping not to have to delve into an Endian converter. I tried playing with quick one-liners such as dd conv=swab but honestly I don't know what I'm doing there. From what I've read, I need more in depth knowledge about the structure of the binary file in order to do a proper Endian conversion.

It probably requires specific coding, since a simple byte swapping will not do the trick, as there might be data that is not in little or big Endian.

Originally Posted by DLuo

Also when I comment out the last line beginning with "self" I get the same thing namely because the Segmentation Fault happens during the input portion of the code therefore it doesn't even get a chance to read that line.

Too bad. I had hopes that it could have been an issue in the transfer of data into ParaView, but since it's not, the problem is during the reading of the data.

Originally Posted by DLuo

BTW when I try that code with a "good" ensight data set, it throws errors if I leave in the line "en.SetByteOrderToLittleEndian()"

vtkEnSight6BinaryReader (0x2fbeb30): Invalid number of unstructured points; check that ByteOrder is set correctly.

Without it, it just prints BigEndian. The data doesn't actually get rendered.

If the last line with "ShadowCopy" was still operational when you made that test, then the data is likely in LittleEndian.
Worst case scenario: the big data might be in BigEndian, but certain details might be in LittleEndian.

Originally Posted by DLuo

Have I been misinformed? Are my data sets in fact Big Endian?

In the tests I made, when using the incorrect Endian, it did not crash, but it also did not properly load the data for rendering. So essentially, nothing was rendered.


I did another quick look for any more hints that might be in the code documentation and I spotted this today: http://www.vtk.org/doc/nightly/html/...r.html#details

Warning
You must manually call Update on this reader and then connect the rest of the pipeline because (due to the nature of the file format) it is not possible to know ahead of time how many outputs you will have or what types they will be. This reader can only handle static EnSight datasets (both static geometry and variables).

Looks like one hint is that your data might have non-static geometry or variables.


Beyond this, the only somewhat practical idea that comes to mind is to build VTK 6.0 or 6.1 from source code in Debug mode and to diagnose the issue more up-close in the VTK code. Or use a release version of VTK and build a modified version of this class...
Of course that for such it would be useful to have a really small test case that reproduces the problem. Such a test case would still be needed for submitting a bug report in the VTK bug tracker, in case you prefer to defer the issue to the developers.

Contacting Kitware directly for paid support is also a possibility, but I don't know if it's practical for you (your company) or not.

[DLuo]

Regarding array sizes, the case has 22M points and 39M cells.

I think I might just have to abandon my scripts for this case and manually post-process the data in metaPost CFD.

If I happen across an installation of Ensight, I'll give the import/export a try and see if Paraview can read the new data.

Thanks for your help Bruno but it looks like this one will remain unresolved for some time.

[DLuo]

Hi Bruno,
So it turns out, the code you provided was on the correct path. A colleague of mine, who is much better versed in Python, was able to take your code and make it work/load the corrupted files into paraview.
In short, it turns out the files were in fact Big Endian, and that most of the datasets had a header that specified Big Endian. But in a few cases, it actually lacked this information, and thus Paraview struggled with it. I'm a little surprised because the source code for the vtkEnsight Binary reader is supposed to default to Big Endian if the byte order is ambiguous.

Honestly I can't really explain it any better because my python scripting is terrible. But for anyone else who has a similar problem I think the points Bruno brought up are a good starting point.

[wyldckat]

Hi Andy,

I'm glad your colleague managed to use my broken Python code to get things working!
And I hope you've documented this on your side, otherwise you might trigger the same issue in a few months from now and no longer have a working solution for it .

Best regards,
Bruno

[S_teph_2000]

Hello,

I've been trying to open CfX results converted into ensight gold binaries into paraview without success. The conversion is made using a tool in ANSYS CFX. I also tried Ascii format and other conversion options without success. I am using ANSYS 15 I get the following error:
p, li { white-space: pre-wrap; }



ERROR: In C:\DBD\pvs-x32\paraview\src\paraview\VTK\IO\EnSight\vtkEnSigh tReader.cxx, line 299
vtkEnSightGoldReader (12D308D0): error reading geometry file




ERROR: In C:\DBD\pvs-x32\paraview\src\paraview\VTK\Common\ExecutionMode l\vtkExecutive.cxx, line 784
vtkCompositeDataPipeline (12AC26D8): Algorithm vtkEnSightGoldReader(12D308D0) returned failure for request: vtkInformation (12AD8880)
Debug: Off
Modified Time: 211191
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA
ALGORITHM_AFTER_FORWARD: 1
FORWARD_DIRECTION: 0
FROM_OUTPUT_PORT: 0




Any suggestion???

[S_teph_2000]

Ok, I found the source of the problem: There were spaces included in the filename.
Changing the filename solved everything.