[mandman]

Please go thru the link. http://1drv.ms/1wMph1U
Except this any other logic than SPMD is also valuable

[mprinkey]

Linking to a .pdf is not a good way to get attention to your question.

Your question is a common one. Data dependencies are critically important and the issue you are asking about is called a "race condition." You can avoid these problems dividing the nodes into two groups. Color the odd-numbered lines red and the even-numbered lines black. Then use all of the cores to update the red regions (locking the black regions if necessary). After that use all of the cores to update the black regions. This division of updates will remove the data dependencies and a do across pragma for each color update will likely parallelize cleanly.

Red-Black (and more generally multi-coloring) can be used in a variety of ways to avoid cache coherency/race conditions. You can read much about red-black Gauss-Siedel iteration and its merits versus standard GS on current CPUs. What I've described above is simply a line-implicit version.

There are other ways to deal with this type of dependency issue too--the goal should always be to avoid reading from recently written data. Parallelization of line-implicit algorithms also bring other pitfalls--for example, x-direction line updates will be fast but y-direction line updates will incur many cache misses. Some times it is worthwhile to do an explicit data transpose to make line accesses always fast.