CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   OpenFOAM (http://www.cfd-online.com/Forums/openfoam/)
-   -   Unaligned accesses on IA64 (http://www.cfd-online.com/Forums/openfoam/60801-unaligned-accesses-ia64.html)

andre June 10, 2008 09:29

Dear OpenFOAMers, When I de
 
Dear OpenFOAMers,

When I decompose my mesh and run in parallel on 4 cores (with all decomposition techniques), I get a lot of the following warnings:

simpleFoam(4449): unaligned access to 0x6000000001fbaf47, ip=0x2000000001ee7ad1

I can disable the warnings with:
prctl --unaligned=silent mpirun -np 4 simpleFoam . <casename> -parallel

However, the unaligned memory accesses could have a bad effect on the performance. I wonder if am the only one facing them, since I didn't find it else on the forum.

The mesh I am using is in the order of 200 MB.

I dont get these warnings with other meshes (e.g running pitzDaily on 4 cores)

Regards,

Andre

mattijs June 10, 2008 14:48

Do you have processor patches
 
Do you have processor patches with odd (as in 'not even') sizes? Just check the output of decomposePar - see how many faces you have to neighbouring processors. Does this correlate to whether you get the warning or not?

andre June 11, 2008 06:10

Thank you for the always fast
 
Thank you for the always fast replies. Let me output the results of decomposePar tool for this mesh:
----------------------------------------------------
Nprocs : 1
Create time

Time = 0
Create mesh


Calculating distribution of cells
Selecting decompositionMethod metis


Finished decomposition in 34.416 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
Number of cells = 1715464
Number of faces shared with processor 2 = 6885
Number of faces shared with processor 1 = 5946
Number of faces shared with processor 3 = 549
Number of processor patches = 3
Number of processor faces = 13380
Number of boundary faces = 82738

Processor 1
Number of cells = 1713283
Number of faces shared with processor 0 = 5946
Number of processor patches = 1
Number of processor faces = 5946
Number of boundary faces = 90936

Processor 2
Number of cells = 1714303
Number of faces shared with processor 0 = 6885
Number of faces shared with processor 3 = 6981
Number of processor patches = 2
Number of processor faces = 13866
Number of boundary faces = 83589

Processor 3
Number of cells = 1715374
Number of faces shared with processor 2 = 6981
Number of faces shared with processor 0 = 549
Number of processor patches = 2
Number of processor faces = 7530
Number of boundary faces = 85262

Number of processor faces = 20361
Max number of processor patches = 3
Max number of faces between processors = 13866

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.
----------------------------------------------------

This are the results for decomposing pitzDaily:
----------------------------------------------------
Nprocs : 1
Create time

Time = 0
Create mesh


Calculating distribution of cells
Selecting decompositionMethod metis


Finished decomposition in 0.032 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
Number of cells = 3105
Number of faces shared with processor 1 = 90
Number of processor patches = 1
Number of processor faces = 90
Number of boundary faces = 6382

Processor 1
Number of cells = 3023
Number of faces shared with processor 2 = 68
Number of faces shared with processor 0 = 90
Number of processor patches = 2
Number of processor faces = 158
Number of boundary faces = 6142

Processor 2
Number of cells = 3035
Number of faces shared with processor 3 = 71
Number of faces shared with processor 1 = 68
Number of processor patches = 2
Number of processor faces = 139
Number of boundary faces = 6175

Processor 3
Number of cells = 3062
Number of faces shared with processor 2 = 71
Number of processor patches = 1
Number of processor faces = 71
Number of boundary faces = 6311

Number of processor faces = 229
Max number of processor patches = 2
Max number of faces between processors = 158

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.
----------------------------------------------------
Regards,

Andre

andre June 16, 2008 09:57

Hey Mattijs Janssens, Can
 
Hey Mattijs Janssens,

Can I safely ignore the warnings, without getting a huge decrease in performance.

Regards,

Andre

mattijs June 23, 2008 09:57

No idea. Still quite weird. Do
 
No idea. Still quite weird. Do you know where the message comes from? The only place I can think of where we interface with the system is inside mpi/Pstream/. During compilation check whether you get any pointer casting warnings.

eugene June 23, 2008 11:37

I believe this issue is due to
 
I believe this issue is due to a bug/issue in your native SGI mpi. We encountered the same problem and it only occurs for one specific patch of RHEL 3 (I think Taroon 4) on the Altix.


All times are GMT -4. The time now is 07:20.