CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

Unaligned accesses on IA64

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   June 10, 2008, 08:29
Default Dear OpenFOAMers, When I de
  #1
New Member
 
Andre
Join Date: Mar 2009
Posts: 19
Rep Power: 8
andre is on a distinguished road
Dear OpenFOAMers,

When I decompose my mesh and run in parallel on 4 cores (with all decomposition techniques), I get a lot of the following warnings:

simpleFoam(4449): unaligned access to 0x6000000001fbaf47, ip=0x2000000001ee7ad1

I can disable the warnings with:
prctl --unaligned=silent mpirun -np 4 simpleFoam . <casename> -parallel

However, the unaligned memory accesses could have a bad effect on the performance. I wonder if am the only one facing them, since I didn't find it else on the forum.

The mesh I am using is in the order of 200 MB.

I dont get these warnings with other meshes (e.g running pitzDaily on 4 cores)

Regards,

Andre
andre is offline   Reply With Quote

Old   June 10, 2008, 13:48
Default Do you have processor patches
  #2
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Do you have processor patches with odd (as in 'not even') sizes? Just check the output of decomposePar - see how many faces you have to neighbouring processors. Does this correlate to whether you get the warning or not?
mattijs is offline   Reply With Quote

Old   June 11, 2008, 05:10
Default Thank you for the always fast
  #3
New Member
 
Andre
Join Date: Mar 2009
Posts: 19
Rep Power: 8
andre is on a distinguished road
Thank you for the always fast replies. Let me output the results of decomposePar tool for this mesh:
----------------------------------------------------
Nprocs : 1
Create time

Time = 0
Create mesh


Calculating distribution of cells
Selecting decompositionMethod metis


Finished decomposition in 34.416 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
Number of cells = 1715464
Number of faces shared with processor 2 = 6885
Number of faces shared with processor 1 = 5946
Number of faces shared with processor 3 = 549
Number of processor patches = 3
Number of processor faces = 13380
Number of boundary faces = 82738

Processor 1
Number of cells = 1713283
Number of faces shared with processor 0 = 5946
Number of processor patches = 1
Number of processor faces = 5946
Number of boundary faces = 90936

Processor 2
Number of cells = 1714303
Number of faces shared with processor 0 = 6885
Number of faces shared with processor 3 = 6981
Number of processor patches = 2
Number of processor faces = 13866
Number of boundary faces = 83589

Processor 3
Number of cells = 1715374
Number of faces shared with processor 2 = 6981
Number of faces shared with processor 0 = 549
Number of processor patches = 2
Number of processor faces = 7530
Number of boundary faces = 85262

Number of processor faces = 20361
Max number of processor patches = 3
Max number of faces between processors = 13866

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.
----------------------------------------------------

This are the results for decomposing pitzDaily:
----------------------------------------------------
Nprocs : 1
Create time

Time = 0
Create mesh


Calculating distribution of cells
Selecting decompositionMethod metis


Finished decomposition in 0.032 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
Number of cells = 3105
Number of faces shared with processor 1 = 90
Number of processor patches = 1
Number of processor faces = 90
Number of boundary faces = 6382

Processor 1
Number of cells = 3023
Number of faces shared with processor 2 = 68
Number of faces shared with processor 0 = 90
Number of processor patches = 2
Number of processor faces = 158
Number of boundary faces = 6142

Processor 2
Number of cells = 3035
Number of faces shared with processor 3 = 71
Number of faces shared with processor 1 = 68
Number of processor patches = 2
Number of processor faces = 139
Number of boundary faces = 6175

Processor 3
Number of cells = 3062
Number of faces shared with processor 2 = 71
Number of processor patches = 1
Number of processor faces = 71
Number of boundary faces = 6311

Number of processor faces = 229
Max number of processor patches = 2
Max number of faces between processors = 158

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.
----------------------------------------------------
Regards,

Andre
andre is offline   Reply With Quote

Old   June 16, 2008, 08:57
Default Hey Mattijs Janssens, Can
  #4
New Member
 
Andre
Join Date: Mar 2009
Posts: 19
Rep Power: 8
andre is on a distinguished road
Hey Mattijs Janssens,

Can I safely ignore the warnings, without getting a huge decrease in performance.

Regards,

Andre
andre is offline   Reply With Quote

Old   June 23, 2008, 08:57
Default No idea. Still quite weird. Do
  #5
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
No idea. Still quite weird. Do you know where the message comes from? The only place I can think of where we interface with the system is inside mpi/Pstream/. During compilation check whether you get any pointer casting warnings.
mattijs is offline   Reply With Quote

Old   June 23, 2008, 10:37
Default I believe this issue is due to
  #6
Senior Member
 
Eugene de Villiers
Join Date: Mar 2009
Posts: 725
Rep Power: 12
eugene is on a distinguished road
I believe this issue is due to a bug/issue in your native SGI mpi. We encountered the same problem and it only occurs for one specific patch of RHEL 3 (I think Taroon 4) on the Altix.
eugene is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unaligned mesh. M FLUENT 0 September 16, 2008 10:47
Can I use the binary files for ia64 and x86_64 CPU in Linux OS mamaly60 OpenFOAM Installation 4 October 24, 2007 22:20
Someone using openfoam on IA64 begou OpenFOAM Installation 1 February 15, 2007 12:02
Compilation error on IA64 g411 begou OpenFOAM Installation 2 February 13, 2007 06:23
Compilation error on IA64 olol1976 OpenFOAM Installation 1 October 19, 2006 14:11


All times are GMT -4. The time now is 14:44.