CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Main CFD Forum

read unformatted file in windows and linux

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   August 7, 2004, 21:26
Default read unformatted file in windows and linux
  #1
autofly
Guest
 
Posts: n/a
I want to read unformatted files which are generated in windows system directly in linux for the use of my fortran code. however, i ger error. only by convert the files into formatted files in windows and then read in linux and convert to unformatted. but the size of the files are terrible and time-expensive when i ftp it to linux. how i can convert between different systems with unformatted files for fortran? thanks
  Reply With Quote

Old   August 7, 2004, 23:38
Default Re: read unformatted file in windows and linux
  #2
Axel Rohde
Guest
 
Posts: n/a
What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older Fortran code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another.

But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, Word64) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system.

Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, Jmax, Kmax are 32-bit integers and PX(), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word.

(Sorry, but I don't seem to be able to preserve the formatting of my code on this text page)

(Declarations)

Public Imax As Long Public Jmax As Long Public Kmax As Long

Public PX() As Single Public PY() As Single Public PZ() As Single

Public Type Word32

Byte1 As String * 1

Byte2 As String * 1

Byte3 As String * 1

Byte4 As String * 1 End Type

Public LongGet As Word32 Public LongPut As Word32

(End Declarations)

Public Sub SavePlot3D_XYZ(FileName As String)

Open FileName For Binary As #1

Put #1, , Imax

Put #1, , Jmax

Put #1, , Kmax

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PX(i, j, k)

Next i

Next j

Next k

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PY(i, j, k)

Next i

Next j

Next k

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PZ(i, j, k)

Next i

Next j

Next k

Close #1

Open FileName For Binary As #1

NB = 1

Do While Not EOF(1)

Get #1, NB, LongGet

LongPut.Byte1 = LongGet.Byte4

LongPut.Byte2 = LongGet.Byte3

LongPut.Byte3 = LongGet.Byte2

LongPut.Byte4 = LongGet.Byte1

Put #1, NB, LongPut

NB = NB + 4

Loop

Close #1

End Sub
  Reply With Quote

Old   August 8, 2004, 00:01
Default Please read this response - HTML Version
  #3
Axel Rohde
Guest
 
Posts: n/a
<html xmlns="urn:schemas-microsoft-comfficeffice" xmlns:w="urn:schemas-microsoft-comffice:word" xmlns="http://www.w3.org/TR/REC-html40">

<head> <meta http-equiv=Content-Type content="text/html; charset=windows-1252"> <meta name=ProgId content=Word.Document> <meta name=Generator content="Microsoft Word 11"> <meta name=Originator content="Microsoft Word 11"> <link rel=File-List href="What%20Fortran%20calls_files/filelist.xml"> <title>What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data</title> <!--[if gte mso 9]><xml> <w:WordDocument> <w:SpellingState>Clean</w:SpellingState> <w:GrammarState>Clean</w:GrammarState> <wrawingGridHorizontalSpacing>3.6 pt</wrawingGridHorizontalSpacing> <wrawingGridVerticalSpacing>3.6 pt</wrawingGridVerticalSpacing> <wisplayHorizontalDrawingGridEvery>0</wisplayHorizontalDrawingGridEvery> <wisplayVerticalDrawingGridEvery>0</wisplayVerticalDrawingGridEvery> <w:UseMarginsForDrawingGridOrigin/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <wrawingGridHorizontalOrigin>84.95 pt</wrawingGridHorizontalOrigin> <wrawingGridVerticalOrigin>99.35 pt</wrawingGridVerticalOrigin> <w:Compatibility>

<w:SelectEntireFieldWithStartOrEnd/>

<w:UseWord2002TableStyleRules/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--> <style>

</style> <!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable

{mso-style-name:"Table Normal";

mso-tstyle-rowband-size:0;

mso-tstyle-colband-size:0;

mso-style-noshow:yes;

mso-style-parent:"";

mso-padding-alt:0in 5.4pt 0in 5.4pt;

mso-para-margin:0in;

mso-para-margin-bottom:.0001pt;

mso-pagination:widow-orphan;

font-size:10.0pt;

font-family:"Times New Roman";

mso-ansi-language:#0400;

mso-fareast-language:#0400;

mso-bidi-language:#0400;} </style> <![endif]--> </head>

<body lang=EN-US style='tab-interval:.5in'>

<div class=Section1>

<p class=MsoNormal align=left style='text-align:left'><span style='mso-bidi-font-size: 10.0pt;font-family:Verdana'>What <span class=GramE>Fortran</span> calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older <span class=GramE>Fortran</span> code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another. But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, <span class=GramE>Word64</span>) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system. Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, <span class=SpellE>Jmax</span>, <span class=SpellE>Kmax</span> are 32-bit integers and <span class=GramE>PX(</span>), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word. (Declarations) Public Imax As Long******* ' 32-bit integer Public <span class=SpellE>Jmax</span> As Long Public <span class=SpellE>Kmax</span> As Long Public PX() As Single******* ' 32-bit floating point Public PY() As Single Public PZ() As Single Public Type Word32 ** Byte1 As String * 1 ** Byte2 As String * 1 ** Byte3 As String * 1 ** Byte4 As String * 1 End Type Public <span class=SpellE>LongGet</span> As Word32 Public <span class=SpellE>LongPut</span> As Word32 (End Declarations) Public Sub SavePlot3D_XYZ(<span class=SpellE>FileName</span> As String) *** ** Open <span class=SpellE>FileName</span> For Binary As #1 ** Put #1, , Imax ** Put #1, , <span class=SpellE>Jmax</span> ** Put #1, , <span class=SpellE>Kmax</span> ***** ** For k = 1 To <span class=SpellE>Kmax</span> ***** For j = 1 To <span class=SpellE>Jmax</span> ******** For <span class=SpellE>i</span> = 1 To Imax *********** Put #1, , PX(<span class=SpellE>i</span>, j, k) ******** Next <span class=SpellE>i</span> ***** Next j ** Next k ****** ** For k = 1 To <span class=SpellE>Kmax</span> ***** For j = 1 To <span class=SpellE>Jmax</span> ******** For <span class=SpellE>i</span> = 1 To Imax *********** Put #1, , PY(<span class=SpellE>i</span>, j, k) ******** Next <span class=SpellE>i</span> ***** Next j ** Next k ****** ** For k = 1 To <span class=SpellE>Kmax</span> ***** For j = 1 To <span class=SpellE>Jmax</span> ******** For <span class=SpellE>i</span> = 1 To Imax *********** Put #1, , PZ(<span class=SpellE>i</span>, j, k) ******** Next <span class=SpellE>i</span> ***** Next j ** Next k * ** Close #1 * ** Open <span class=SpellE>FileName</span> For Binary As #1 ********** ** NB = 1 ********** ** Do While Not EOF(1) ** ***** Get #1, NB, <span class=SpellE>LongGet</span> ***** ***** LongPut.Byte1 = LongGet.Byte4 ***** LongPut.Byte2 = LongGet.Byte3 ***** LongPut.Byte3 = LongGet.Byte2 ***** LongPut.Byte4 = LongGet.Byte1 ***** ***** Put #1, NB, <span class=SpellE>LongPut</span> ***** ***** NB = NB + 4 ***** ** Loop ****** ** Close #1 End Sub</span><o></o>


<p class=MsoNormal><o>*</o>


</div>

</body>

</html>
  Reply With Quote

Old   August 8, 2004, 12:09
Default Re: read unformatted file in windows and linux
  #4
jasond
Guest
 
Posts: n/a
The unformatted->formatted->unformatted conversion process that you have had to use is the only guaranteed method to migrate unformatted files from one "system" to another. However, the compilers involved are more important than the systems involved. The other poster brings up byte ordering, but byte ordering is a architecture issue, not an OS issue. Most linux and windows systems are running on Intel hardware that has consistent byte ordering, so I think it is unlikely that this is the root problem. It is more likely that the compilers that you are using do not have compatible unformatted files. If you are using g77 (and even if you aren't), you might want to take a look at:

http://gcc.gnu.org/onlinedocs/gcc-3....ted-Files.html

Hope that this helps,

jason
  Reply With Quote

Old   August 9, 2004, 07:44
Default Re: read unformatted file in windows and linux
  #5
autofly
Guest
 
Posts: n/a
thank for your help! i transfer data from windows to linux. as in my code,the grid and field solutions are saved in plot3d format.as you said, i compile code in linux with F77 not, g77. in addition. i meet some problems that is read unformmated data files writen in fortran power station compiles, but be read in code compiled in visual fortran 6.5. the same problem appear. i can't read the files.so i agree with your opinion on the compiler.But is there another way? if g77 is the best choise, what i can do ?

  Reply With Quote

Old   August 9, 2004, 09:05
Default Correction: Byte Ordering
  #6
Axel Rohde
Guest
 
Posts: n/a
The byte ordering is a matter of architecture as the other poster correctly points out. So if you are running Linux on an Intel, you don't need to reverse the bytes. It has been a few years, since I wrote this conversion routine, so I forgot and confused some of the details. I believe I ran Plot3D on a Sun workstation which had an Alpha processor.

I did follow the link provided in the previous post, and I find it to be somewhat misleading in the following statement:

"The record length is of C type long; this means that it is 8 bytes on 64-bit systems such as Alpha GNU/Linux and 4 bytes on other systems, such as x86 GNU/Linux. Consequently such files cannot be exchanged between 64-bit and 32-bit systems, even with the same basic number format."

Intel processors can read either 32-bit (single precision) or 64-bit (double precision) floating point numbers. So as long as you reverse the byte order when transferring from an Alpha to an Intel machine, you should have no problem with the overall conversion. Incidentally, all math operations on Intel (co)processors are carried out in 80-bit precision internally. Thus classifying an Intel as a 32-bit "only" processor is a bit of an understatement in my opinion.
  Reply With Quote

Old   August 9, 2004, 10:11
Default Re: read unformatted file in windows and linux
  #7
Tom
Guest
 
Posts: n/a
The problem may be that your read statements in your fortran code are not identical to the write statements that created the file. FORTRAN adds record markers into the unformatted output at the beginning and end of each unformatted write. If read statements do not correspond correctly you will generally get an error.

Also when you transfer from windows to linux are you using ftp with the binary flag correctly set? If a binary file is transfered as ascii it's likely to be corrupted.
  Reply With Quote

Old   August 11, 2004, 03:24
Default Re: read unformatted file in windows and linux
  #8
jasond
Guest
 
Posts: n/a
Well, (in my opinion) there is no best choice if you must stay with the plot3d format. Some compilers (and OS's like the Cray OS's) have facilities that will allow for easier transition from compiler to compiler and system to system. Unless you have access to this sort of facility, you really can't avoid doing some programming in a language other than f77. The safest, easiest, and most portable option is to do the conversion to formatted (being careful to preserve as much precision as possible) and compress the formatted file if you have to keep it around.

jason
  Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Dual Boot Windows and Linux and Go Open Source andyj Main CFD Forum 2 October 21, 2010 16:49
Cross-compiling OpenFOAM 1.7.0 on Linux for Windows 32 and 64bits with Mingw-w64 wyldckat OpenFOAM Announcements from Other Sources 3 September 8, 2010 06:25
Installation of OpenFOAM-1.6 on Ubuntu 9.10 marval OpenFOAM Installation 2 March 17, 2010 09:33
Linux to Windows (dual boot system) Alex Main CFD Forum 9 May 3, 2007 03:36
Windows EXE file and Linux MING Main CFD Forum 4 October 22, 2004 05:31


All times are GMT -4. The time now is 13:50.