read unformatted file in windows and linux

August 7, 2004, 21:26

I want to read unformatted files which are generated in windows system directly in linux for the use of my fortran code. however, i ger error. only by convert the files into formatted files in windows and then read in linux and convert to unformatted. but the size of the files are terrible and time-expensive when i ftp it to linux. how i can convert between different systems with unformatted files for fortran? thanks

August 7, 2004, 23:38

What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older Fortran code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another.

But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, Word64) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system.

Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, Jmax, Kmax are 32-bit integers and PX(), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word.

(Sorry, but I don't seem to be able to preserve the formatting of my code on this text page)

(Declarations)

Public Imax As Long Public Jmax As Long Public Kmax As Long

Public PX() As Single Public PY() As Single Public PZ() As Single

Public Type Word32

Byte1 As String * 1

Byte2 As String * 1

Byte3 As String * 1

Byte4 As String * 1 End Type

Public LongGet As Word32 Public LongPut As Word32

(End Declarations)

Public Sub SavePlot3D_XYZ(FileName As String)

Open FileName For Binary As #1

Put #1, , Imax

Put #1, , Jmax

Put #1, , Kmax

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PX(i, j, k)

Next i

Next j

Next k

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PY(i, j, k)

Next i

Next j

Next k

For k = 1 To Kmax

For j = 1 To Jmax

For i = 1 To Imax

Put #1, , PZ(i, j, k)

Next i

Next j

Next k

Close #1

Open FileName For Binary As #1

NB = 1

Do While Not EOF(1)

Get #1, NB, LongGet

LongPut.Byte1 = LongGet.Byte4

LongPut.Byte2 = LongGet.Byte3

LongPut.Byte3 = LongGet.Byte2

LongPut.Byte4 = LongGet.Byte1

Put #1, NB, LongPut

NB = NB + 4

Loop

Close #1

End Sub

August 8, 2004, 00:01

<html xmlns

="urn:schemas-microsoft-com

ffice

ffice" xmlns:w="urn:schemas-microsoft-com

ffice:word" xmlns="http://www.w3.org/TR/REC-html40">

<head> <meta http-equiv=Content-Type content="text/html; charset=windows-1252"> <meta name=ProgId content=Word.Document> <meta name=Generator content="Microsoft Word 11"> <meta name=Originator content="Microsoft Word 11"> <link rel=File-List href="What%20Fortran%20calls_files/filelist.xml"> <title>What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data</title> <!--[if gte mso 9]><xml> <w:WordDocument> <w:SpellingState>Clean</w:SpellingState> <w:GrammarState>Clean</w:GrammarState> <w

rawingGridHorizontalSpacing>3.6 pt</w

rawingGridHorizontalSpacing> <w

rawingGridVerticalSpacing>3.6 pt</w

rawingGridVerticalSpacing> <w

isplayHorizontalDrawingGridEvery>0</w

isplayHorizontalDrawingGridEvery> <w

isplayVerticalDrawingGridEvery>0</w

isplayVerticalDrawingGridEvery> <w:UseMarginsForDrawingGridOrigin/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w

rawingGridHorizontalOrigin>84.95 pt</w

rawingGridHorizontalOrigin> <w

rawingGridVerticalOrigin>99.35 pt</w

rawingGridVerticalOrigin> <w:Compatibility>

<w:SelectEntireFieldWithStartOrEnd/>

<w:UseWord2002TableStyleRules/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--> <style>

</style>  </head>

<body lang=EN-US style='tab-interval:.5in'>

<div class=Section1>

What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older Fortran code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another. But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, Word64) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system. Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, Jmax, Kmax are 32-bit integers and PX(), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word. (Declarations) Public Imax As Long******* ' 32-bit integer Public Jmax As Long Public Kmax As Long Public PX() As Single******* ' 32-bit floating point Public PY() As Single Public PZ() As Single Public Type Word32 ** Byte1 As String * 1 ** Byte2 As String * 1 ** Byte3 As String * 1 ** Byte4 As String * 1 End Type Public LongGet As Word32 Public LongPut As Word32 (End Declarations) Public Sub SavePlot3D_XYZ(FileName As String) *** ** Open FileName For Binary As #1 ** Put #1, , Imax ** Put #1, , Jmax ** Put #1, , Kmax ***** ** For k = 1 To Kmax ***** For j = 1 To Jmax ******** For i = 1 To Imax *********** Put #1, , PX(i, j, k) ******** Next i ***** Next j ** Next k ****** ** For k = 1 To Kmax ***** For j = 1 To Jmax ******** For i = 1 To Imax *********** Put #1, , PY(i, j, k) ******** Next i ***** Next j ** Next k ****** ** For k = 1 To Kmax ***** For j = 1 To Jmax ******** For i = 1 To Imax *********** Put #1, , PZ(i, j, k) ******** Next i ***** Next j ** Next k * ** Close #1 * ** Open FileName For Binary As #1 ********** ** NB = 1 ********** ** Do While Not EOF(1) ** ***** Get #1, NB, LongGet ***** ***** LongPut.Byte1 = LongGet.Byte4 ***** LongPut.Byte2 = LongGet.Byte3 ***** LongPut.Byte3 = LongGet.Byte2 ***** LongPut.Byte4 = LongGet.Byte1 ***** ***** Put #1, NB, LongPut ***** ***** NB = NB + 4 ***** ** Loop ****** ** Close #1 End Sub<o

></o

>

<o

>*</o

>

</div>

</body>

</html>

August 8, 2004, 12:09

The unformatted->formatted->unformatted conversion process that you have had to use is the only guaranteed method to migrate unformatted files from one "system" to another. However, the compilers involved are more important than the systems involved. The other poster brings up byte ordering, but byte ordering is a architecture issue, not an OS issue. Most linux and windows systems are running on Intel hardware that has consistent byte ordering, so I think it is unlikely that this is the root problem. It is more likely that the compilers that you are using do not have compatible unformatted files. If you are using g77 (and even if you aren't), you might want to take a look at:

http://gcc.gnu.org/onlinedocs/gcc-3....ted-Files.html

Hope that this helps,

jason

August 9, 2004, 07:44

thank for your help! i transfer data from windows to linux. as in my code,the grid and field solutions are saved in plot3d format.as you said, i compile code in linux with F77 not, g77. in addition. i meet some problems that is read unformmated data files writen in fortran power station compiles, but be read in code compiled in visual fortran 6.5. the same problem appear. i can't read the files.so i agree with your opinion on the compiler.But is there another way? if g77 is the best choise, what i can do ?

August 9, 2004, 09:05

The byte ordering is a matter of architecture as the other poster correctly points out. So if you are running Linux on an Intel, you don't need to reverse the bytes. It has been a few years, since I wrote this conversion routine, so I forgot and confused some of the details. I believe I ran Plot3D on a Sun workstation which had an Alpha processor.

I did follow the link provided in the previous post, and I find it to be somewhat misleading in the following statement:

"The record length is of C type long; this means that it is 8 bytes on 64-bit systems such as Alpha GNU/Linux and 4 bytes on other systems, such as x86 GNU/Linux. Consequently such files cannot be exchanged between 64-bit and 32-bit systems, even with the same basic number format."

Intel processors can read either 32-bit (single precision) or 64-bit (double precision) floating point numbers. So as long as you reverse the byte order when transferring from an Alpha to an Intel machine, you should have no problem with the overall conversion. Incidentally, all math operations on Intel (co)processors are carried out in 80-bit precision internally. Thus classifying an Intel as a 32-bit "only" processor is a bit of an understatement in my opinion.

August 9, 2004, 10:11

The problem may be that your read statements in your fortran code are not identical to the write statements that created the file. FORTRAN adds record markers into the unformatted output at the beginning and end of each unformatted write. If read statements do not correspond correctly you will generally get an error.

Also when you transfer from windows to linux are you using ftp with the binary flag correctly set? If a binary file is transfered as ascii it's likely to be corrupted.

August 11, 2004, 03:24

Well, (in my opinion) there is no best choice if you must stay with the plot3d format. Some compilers (and OS's like the Cray OS's) have facilities that will allow for easier transition from compiler to compiler and system to system. Unless you have access to this sort of facility, you really can't avoid doing some programming in a language other than f77. The safest, easiest, and most portable option is to do the conversion to formatted (being careful to preserve as much precision as possible) and compress the formatted file if you have to keep it around.

jason

August 7, 2004, 23:38	Re: read unformatted file in windows and linux	#2
Axel Rohde Guest Posts: n/a	What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older Fortran code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another. But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, Word64) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system. Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, Jmax, Kmax are 32-bit integers and PX(), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word. (Sorry, but I don't seem to be able to preserve the formatting of my code on this text page) (Declarations) Public Imax As Long Public Jmax As Long Public Kmax As Long Public PX() As Single Public PY() As Single Public PZ() As Single Public Type Word32 Byte1 As String * 1 Byte2 As String * 1 Byte3 As String * 1 Byte4 As String * 1 End Type Public LongGet As Word32 Public LongPut As Word32 (End Declarations) Public Sub SavePlot3D_XYZ(FileName As String) Open FileName For Binary As #1 Put #1, , Imax Put #1, , Jmax Put #1, , Kmax For k = 1 To Kmax For j = 1 To Jmax For i = 1 To Imax Put #1, , PX(i, j, k) Next i Next j Next k For k = 1 To Kmax For j = 1 To Jmax For i = 1 To Imax Put #1, , PY(i, j, k) Next i Next j Next k For k = 1 To Kmax For j = 1 To Jmax For i = 1 To Imax Put #1, , PZ(i, j, k) Next i Next j Next k Close #1 Open FileName For Binary As #1 NB = 1 Do While Not EOF(1) Get #1, NB, LongGet LongPut.Byte1 = LongGet.Byte4 LongPut.Byte2 = LongGet.Byte3 LongPut.Byte3 = LongGet.Byte2 LongPut.Byte4 = LongGet.Byte1 Put #1, NB, LongPut NB = NB + 4 Loop Close #1 End Sub

August 8, 2004, 00:01	Please read this response - HTML Version	#3
Axel Rohde Guest Posts: n/a	<html xmlns="urn:schemas-microsoft-comfficeffice" xmlns:w="urn:schemas-microsoft-comffice:word" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=Content-Type content="text/html; charset=windows-1252"> <meta name=ProgId content=Word.Document> <meta name=Generator content="Microsoft Word 11"> <meta name=Originator content="Microsoft Word 11"> <link rel=File-List href="What%20Fortran%20calls_files/filelist.xml"> <title>What Fortran calls "unformatted" data is referred to in other languages (VB, C) as "binary" data</title> <!--[if gte mso 9]><xml> <w:WordDocument> <w:SpellingState>Clean</w:SpellingState> <w:GrammarState>Clean</w:GrammarState> <wrawingGridHorizontalSpacing>3.6 pt</wrawingGridHorizontalSpacing> <wrawingGridVerticalSpacing>3.6 pt</wrawingGridVerticalSpacing> <wisplayHorizontalDrawingGridEvery>0</wisplayHorizontalDrawingGridEvery> <wisplayVerticalDrawingGridEvery>0</wisplayVerticalDrawingGridEvery> <w:UseMarginsForDrawingGridOrigin/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <wrawingGridHorizontalOrigin>84.95 pt</wrawingGridHorizontalOrigin> <wrawingGridVerticalOrigin>99.35 pt</wrawingGridVerticalOrigin> <w:Compatibility> <w:SelectEntireFieldWithStartOrEnd/> <w:UseWord2002TableStyleRules/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--> <style> </style> <!--[if gte mso 10]> <style> /* Style Definitions / table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </head> <body lang=EN-US style='tab-interval:.5in'> <div class=Section1> <p class=MsoNormal align=left style='text-align:left'><span style='mso-bidi-font-size: 10.0pt;font-family:Verdana'>What <span class=GramE>Fortran</span> calls "unformatted" data is referred to in other languages (VB, C) as "binary" data. I assume your precision is already matched on both platforms, i.e. if you use 32-bit integers in Windows you should be reading 32-bit integers in Linux, ditto for 64-bit floating point numbers (double precision). Older <span class=GramE>Fortran</span> code such as Plot3D uses 32-bit floating point (single precision). So you have to know exactly what the precision is for integers and floating point numbers before you attempt to transfer your binary data from one platform to another. But there is another catch: Windows and Linux store the bytes (8-bit) within each word (Word16, Word32, <span class=GramE>Word64</span>) in reverse order. Therefore you need to reverse the bytes in your binary Windows file before you read it into a program running on a Linux system. Below is a routine that converts a mesh file saved on a Windows machine, so it can be read by Plot3D on a Linux computer. In this routine, Imax, <span class=SpellE>Jmax</span>, <span class=SpellE>Kmax</span> are 32-bit integers and <span class=GramE>PX(</span>), PY(), PZ() are 32-bit floating point numbers. The routine is written in Visual Basic, but you should be able to read the code and follow the logic of reversing bytes within each 32-bit word. (Declarations) Public Imax As Long**** ' 32-bit integer Public <span class=SpellE>Jmax</span> As Long Public <span class=SpellE>Kmax</span> As Long Public PX() As Single*** ' 32-bit floating point Public PY() As Single Public PZ() As Single Public Type Word32 Byte1 As String * 1 ** Byte2 As String * 1 ** Byte3 As String * 1 ** Byte4 As String * 1 End Type Public <span class=SpellE>LongGet</span> As Word32 Public <span class=SpellE>LongPut</span> As Word32 (End Declarations) Public Sub SavePlot3D_XYZ(<span class=SpellE>FileName</span> As String) * Open <span class=SpellE>FileName</span> For Binary As #1 Put #1, , Imax Put #1, , <span class=SpellE>Jmax</span> Put #1, , <span class=SpellE>Kmax</span> * For k = 1 To <span class=SpellE>Kmax</span> *** For j = 1 To <span class=SpellE>Jmax</span> **** For <span class=SpellE>i</span> = 1 To Imax ******* Put #1, , PX(<span class=SpellE>i</span>, j, k) **** Next <span class=SpellE>i</span> * Next j Next k **** For k = 1 To <span class=SpellE>Kmax</span> *** For j = 1 To <span class=SpellE>Jmax</span> **** For <span class=SpellE>i</span> = 1 To Imax ******* Put #1, , PY(<span class=SpellE>i</span>, j, k) **** Next <span class=SpellE>i</span> * Next j Next k **** For k = 1 To <span class=SpellE>Kmax</span> *** For j = 1 To <span class=SpellE>Jmax</span> **** For <span class=SpellE>i</span> = 1 To Imax ******* Put #1, , PZ(<span class=SpellE>i</span>, j, k) **** Next <span class=SpellE>i</span> * Next j Next k * ** Close #1 * Open <span class=SpellE>FileName</span> For Binary As #1 ****** NB = 1 ******** Do While Not EOF(1) * Get #1, NB, <span class=SpellE>LongGet</span> * * LongPut.Byte1 = LongGet.Byte4 * LongPut.Byte2 = LongGet.Byte3 * LongPut.Byte3 = LongGet.Byte2 * LongPut.Byte4 = LongGet.Byte1 * * Put #1, NB, <span class=SpellE>LongPut</span> * * NB = NB + 4 * Loop **** Close #1 End Sub</span><o></o> <p class=MsoNormal><o>*</o> </div> </body> </html>

August 8, 2004, 12:09	Re: read unformatted file in windows and linux	#4
jasond Guest Posts: n/a	The unformatted->formatted->unformatted conversion process that you have had to use is the only guaranteed method to migrate unformatted files from one "system" to another. However, the compilers involved are more important than the systems involved. The other poster brings up byte ordering, but byte ordering is a architecture issue, not an OS issue. Most linux and windows systems are running on Intel hardware that has consistent byte ordering, so I think it is unlikely that this is the root problem. It is more likely that the compilers that you are using do not have compatible unformatted files. If you are using g77 (and even if you aren't), you might want to take a look at: http://gcc.gnu.org/onlinedocs/gcc-3....ted-Files.html Hope that this helps, jason

August 9, 2004, 07:44	Re: read unformatted file in windows and linux	#5
autofly Guest Posts: n/a	thank for your help! i transfer data from windows to linux. as in my code,the grid and field solutions are saved in plot3d format.as you said, i compile code in linux with F77 not, g77. in addition. i meet some problems that is read unformmated data files writen in fortran power station compiles, but be read in code compiled in visual fortran 6.5. the same problem appear. i can't read the files.so i agree with your opinion on the compiler.But is there another way? if g77 is the best choise, what i can do ?

August 9, 2004, 09:05	Correction: Byte Ordering	#6
Axel Rohde Guest Posts: n/a	The byte ordering is a matter of architecture as the other poster correctly points out. So if you are running Linux on an Intel, you don't need to reverse the bytes. It has been a few years, since I wrote this conversion routine, so I forgot and confused some of the details. I believe I ran Plot3D on a Sun workstation which had an Alpha processor. I did follow the link provided in the previous post, and I find it to be somewhat misleading in the following statement: "The record length is of C type long; this means that it is 8 bytes on 64-bit systems such as Alpha GNU/Linux and 4 bytes on other systems, such as x86 GNU/Linux. Consequently such files cannot be exchanged between 64-bit and 32-bit systems, even with the same basic number format." Intel processors can read either 32-bit (single precision) or 64-bit (double precision) floating point numbers. So as long as you reverse the byte order when transferring from an Alpha to an Intel machine, you should have no problem with the overall conversion. Incidentally, all math operations on Intel (co)processors are carried out in 80-bit precision internally. Thus classifying an Intel as a 32-bit "only" processor is a bit of an understatement in my opinion.

August 7, 2004, 21:26	read unformatted file in windows and linux	#1
autofly Guest Posts: n/a	I want to read unformatted files which are generated in windows system directly in linux for the use of my fortran code. however, i ger error. only by convert the files into formatted files in windows and then read in linux and convert to unformatted. but the size of the files are terrible and time-expensive when i ftp it to linux. how i can convert between different systems with unformatted files for fortran? thanks

August 9, 2004, 10:11	Re: read unformatted file in windows and linux	#7
Tom Guest Posts: n/a	The problem may be that your read statements in your fortran code are not identical to the write statements that created the file. FORTRAN adds record markers into the unformatted output at the beginning and end of each unformatted write. If read statements do not correspond correctly you will generally get an error. Also when you transfer from windows to linux are you using ftp with the binary flag correctly set? If a binary file is transfered as ascii it's likely to be corrupted.

August 11, 2004, 03:24	Re: read unformatted file in windows and linux	#8
jasond Guest Posts: n/a	Well, (in my opinion) there is no best choice if you must stay with the plot3d format. Some compilers (and OS's like the Cray OS's) have facilities that will allow for easier transition from compiler to compiler and system to system. Unless you have access to this sort of facility, you really can't avoid doing some programming in a language other than f77. The safest, easiest, and most portable option is to do the conversion to formatted (being careful to preserve as much precision as possible) and compress the formatted file if you have to keep it around. jason

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Dual Boot Windows and Linux and Go Open Source	andyj	Main CFD Forum	2	October 21, 2010 16:49
Cross-compiling OpenFOAM 1.7.0 on Linux for Windows 32 and 64bits with Mingw-w64	wyldckat	OpenFOAM Announcements from Other Sources	3	September 8, 2010 06:25
Installation of OpenFOAM-1.6 on Ubuntu 9.10	marval	OpenFOAM Installation	2	March 17, 2010 08:33
Linux to Windows (dual boot system)	Alex	Main CFD Forum	9	May 3, 2007 03:36
Windows EXE file and Linux	MING	Main CFD Forum	4	October 22, 2004 05:31