CFD Online Discussion Forums - Parallelizing nested loops

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)

- OpenFOAM Programming & Development (https://www.cfd-online.com/Forums/openfoam-programming-development/)

- - Parallelizing nested loops (https://www.cfd-online.com/Forums/openfoam-programming-development/133846-parallelizing-nested-loops.html)

Parallelizing nested loops

I've modified a solver by writing some triple nested loops which works fine for serial case, but not for parallel case.
In my 'for loops', i get information from different meshes and modify volVectorFields and also some user-defined vectorFields(All are in different decomposed domains).
Anybody knows how to parallelize these kinds of code?
I bring some simplified example of my code:

Code:

for (int n=0; n<10; ++n){

for (int i=0; i<2000; ++i){

forAll(U, cellI){

double xd =  points[i].x() - mesh.C()[cellI].x() ; // points[] contains coordinates in different decomposed domains 

if (xd>1) fx= 2222;

userVec[i] += fx*U[cellI];

U[cellI] = i*2;

}

userVec[i] = userVec[i] / 2 ;

}

}

I would appreciate any hint.

Any idea?
If it was only a c++ code, i should have split all the for loops according the number of ranks.
But maybe using forAll, would be generic enough to split the loops. But still two questions remain.

1) What should i do When i need to access the value of my user-defined vectorFields(or a volVectorField), in different sub domains...
2) The dependency of a value to the previously stored value in the nested for loop i.e: userVec[i] += ... ;

I would appreciate any hint.

If you haven't found out by ourself: You can share data between different processes in OpenFOAM using Pstream::scatter and its variation.

So you would probably want to first scatter your userVec then do the calculations on each partition (aka process) and then collect the results of all processes on the master (see Pstream::master()) and then sum them all up. Also have a look at OpenFOAM functions like gSum.