CFD Online Discussion Forums - problem parallel processing (meshSearch, openMPI)

Hi,

I want to perform a simple post-processing task (in parallel, using meshsearch).
Basically:

Quote:

- each processor loops over its interior field

-> current_pt=mesh.C()[cellI];
-> U_current[cellI]=U[cellI];

- if the value of field X (here mesh.C().component(2)) is above a threshold, it looks for an offset point:

-> sup_pt=current_pt+point(0.0 , 0.0, offset);
-> idx_sup = searchEngine.findCell(sup_pt)
-> U_sup[cellI]= Uint.interpolate(sup_pt,idx_sup)

- the new field Y gets the value Y[cellI]=U_curr[cellI]-U_sup[cellI]

Here is my implementation:

Quote:

meshSearch searchEngine(mesh);
interpolationCellPoint<vector> UInt(U);

vector U_sup=vector::zero;
vector U_curr=vector::zero;
point current_pt=point::zero;
point pt_sup=point::zero;

scalar offset=30.0;

forAll(mesh.C(), cellI)
{

current_pt=mesh.C()[cellI];
U_curr=U[cellI];

if ( current_pt[2]>offset )
{
pt_sup=current_pt+point(0.0,0.0,offset);

Pout << " pt :x "<< current_pt << endl;
Pout << " pt sup: "<< pt_sup << endl;

List<label> idx_sup(Pstream::nProcs(),0);
List<scalar> dist_sup(Pstream::nProcs(),GREAT);
List<vector> Usup(Pstream::nProcs(),vector::zero);

idx_sup[Pstream::myProcNo()] = searchEngine.findNearestCell(pt_sup,cellI);
reduce(idx_sup,maxOp<List<label> >());

searcherSup=1000000.0;

for (int i=0; i<Pstream::nProcs() ;i++)
{
dist_sup[i]=mag(mesh.C()[idx_sup[i]]-pt_sup);
Pout << " pt sup found: "<< i << " "<< mesh.C()[idx_sup[i]] << " " << dist_sup[i] <<endl;
if (dist_sup[i]<searcherSup)
{
searcherSup=dist_sup[i];
found_sup=i;

}

}
Pout << " pt sup kept: "<< mesh.C()[idx_sup[found_sup]] << endl << endl;

if (Pstream::myProcNo()==found_sup)
{
Usup[Pstream::myProcNo()]=UInt.interpolate(pt_sup,idx_sup[found_sup]);

}

reduce(Usup,maxOp<List<vector> >());

U_sup = Usup[found_sup];
Y[cellI] = U_curr-U_sup;

Info << " U : "<< U_curr << endl;
Info << " U sup: "<< U_sup << endl;
}

}
Pout << "Finished! << endl;

It seems that the meshSearch finds the right points, but the problem is that processor2 (due to my decomposition and the offset value) finishes its task first, and the other processors seem to get stuck has soon as this happens.

(By the way, it is a simplified version of my problem. I am not trying to find offset points outside my mesh, as this sample code would suggest)

Do you have an idea why? I have been struggling for quiet a while!!!

Thanks,

Fabien