CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

problem parallel processing (meshSearch, openMPI)

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   September 10, 2012, 04:23
Angry problem parallel processing (meshSearch, openMPI)
  #1
New Member
 
Fabien Farella
Join Date: Jan 2012
Posts: 7
Rep Power: 5
FabienF is on a distinguished road
Hi,

I want to perform a simple post-processing task (in parallel, using meshsearch).
Basically:
Quote:
- each processor loops over its interior field
-> current_pt=mesh.C()[cellI];
-> U_current[cellI]=U[cellI];
- if the value of field X (here mesh.C().component(2)) is above a threshold, it looks for an offset point:
-> sup_pt=current_pt+point(0.0 , 0.0, offset);
-> idx_sup = searchEngine.findCell(sup_pt)
-> U_sup[cellI]= Uint.interpolate(sup_pt,idx_sup)
- the new field Y gets the value Y[cellI]=U_curr[cellI]-U_sup[cellI]
Here is my implementation:

Quote:
meshSearch searchEngine(mesh);
interpolationCellPoint<vector> UInt(U);

vector U_sup=vector::zero;
vector U_curr=vector::zero;
point current_pt=point::zero;
point pt_sup=point::zero;

scalar offset=30.0;

forAll(mesh.C(), cellI)
{
current_pt=mesh.C()[cellI];
U_curr=U[cellI];

if ( current_pt[2]>offset )
{
pt_sup=current_pt+point(0.0,0.0,offset);

Pout << " pt :x "<< current_pt << endl;
Pout << " pt sup: "<< pt_sup << endl;

List<label> idx_sup(Pstream::nProcs(),0);
List<scalar> dist_sup(Pstream::nProcs(),GREAT);
List<vector> Usup(Pstream::nProcs(),vector::zero);

idx_sup[Pstream::myProcNo()] = searchEngine.findNearestCell(pt_sup,cellI);
reduce(idx_sup,maxOp<List<label> >());

searcherSup=1000000.0;

for (int i=0; i<Pstream::nProcs() ;i++)
{
dist_sup[i]=mag(mesh.C()[idx_sup[i]]-pt_sup);
Pout << " pt sup found: "<< i << " "<< mesh.C()[idx_sup[i]] << " " << dist_sup[i] <<endl;
if (dist_sup[i]<searcherSup)
{
searcherSup=dist_sup[i];
found_sup=i;
}
}
Pout << " pt sup kept: "<< mesh.C()[idx_sup[found_sup]] << endl << endl;

if (Pstream::myProcNo()==found_sup)
{
Usup[Pstream::myProcNo()]=UInt.interpolate(pt_sup,idx_sup[found_sup]);
}

reduce(Usup,maxOp<List<vector> >());

U_sup = Usup[found_sup];
Y[cellI] = U_curr-U_sup;

Info << " U : "<< U_curr << endl;
Info << " U sup: "<< U_sup << endl;
}
}
Pout << "Finished! << endl;
It seems that the meshSearch finds the right points, but the problem is that processor2 (due to my decomposition and the offset value) finishes its task first, and the other processors seem to get stuck has soon as this happens.

(By the way, it is a simplified version of my problem. I am not trying to find offset points outside my mesh, as this sample code would suggest)

Do you have an idea why? I have been struggling for quiet a while!!!

Thanks,

Fabien

Last edited by FabienF; September 11, 2012 at 05:54.
FabienF is offline   Reply With Quote

Reply

Tags
interpolation, meshsearch, mpi, parallel, postprocessing

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Parallel interDyMFoam cellLevel problem tgvosk OpenFOAM Running, Solving & CFD 5 February 19, 2014 03:24
Parallel processing problem newbie29 OpenFOAM Running, Solving & CFD 1 June 22, 2012 04:23
HP MPI warning...Distributed parallel processing Peter CFX 10 May 14, 2011 06:17
bubbly flow and parallel processing mvee FLUENT 0 September 12, 2007 05:08
About parallel processing in Linux tuks CFX 10 August 8, 2005 08:22


All times are GMT -4. The time now is 14:56.