MPI (Allreduce) over 1D and 2D arrays
Goodmorning I'm trying to parallelise a part of the code I'm developping.
A friend that works with MPI (in fortran enviroment) helped me and he wrote me some kind of "pseudo-code" instructions that I've to translate in C++/OpenFoam language. 1° I define an array of integers wich contain the number of "interesting" points for each subdomain. I defined it as: int nmiss[nProc]; where nProc is the number of processes involved. After some operations I need to "reduce" it and (as my collegue wrote me) I try to convert MPI_Allreduce(nmiss,1,nproc,MPI_INTEGER,MPI_SUM,Co mmunicator) in something like : reduce(nmiss,sumOp<int>()); After I repeat the same operation for an array double Pext_miss[nProc][nguess]; .. .. .. MPI_Allreduce(Pext_miss,1,(nproc*nmissmax*3),MPI_R EAL8,MPI_SUM,Communicator) Can please someone help me? Thank you Valerio |
Quote:
|
Hello Valerio,
just some comments. To my understanding, "reduce" combines data of a single variable on each processor. Like this: int nmiss; /* Do something, results on different values on each processor, e.g.: */ nmiss=Pstream::myProcNo()+1; reduce(nmiss,sumOp<int>()); Pout << nmiss <<endl; /* nmiss now has the value np*(np-1)/2 on all procs */ Not sure, if "reduce" works for lists. Maybe if you create some proper xxxOp-class (Or do they exist and I overlooked them?) Anyway, to sync lists among processors, I use constructs like the following: labelList offsets(Pstream::nProcs(), 0); offsets[Pstream::myProcNo()]=nFaces; /* nFaces has different value on each processor */ Pstream::listCombineGather(offsets, maxEqOp<label>()); Pstream::listCombineScatter(offsets); Afterwards, the list "offsets" is equal on each processor and contains the "nFaces" value from each other processor. Hope, that helps you somehow. Regards, Hannes |
Thank you for your answers. I've to be honest I've to study a little to understand what you mean:confused: but it's my fault of course!
I try to be more "clear". A part of my code (I copied it below) search some points P that corresponds to the center of a volume cell in the domain and moving from this points it calculate a new point called Pext that is not necessary in the center of a cell. I calculate the difference deltaCl of a variable called C_liq (in the points P and Pext) via interpolation and I store this value in the point P location (I need this variable to calculate a local velocity in the points P. The problem rises when I'm in parallel cause if P is in a domain there are cases (close to the subdomain boundaries) where Pext belongs to another subdomain. So I've to "collect" somehow this "missing" points and I've to calculate deltaCl somehow. Here is a list of the modifications I did to my code since yesterday (that's why sorry I didn't yet replied gschaider:) thank you!!) I try in the following to show you the "serial" version of the code part I'm talking about and in a second moment I'll copy what he told me to do. I've almost "translated" all his speudo-code except the reduce commands HTML Code:
forAll (mesh.cellCentres(),cellI) Equivalent version in "parallel" in the "pseudo-code" formulation. Tomorrow morning as I'll finish to modify it I'll post it in the proprer C++/openFOAM format the parallel version. What you've to get from the code below is just (if it's possible to understand,I know is a little bit confusing) the commands Allreduce that is what I've to "translate" in an OF compatible language. HTML Code:
!nsd='Number of cartesian space dimensions' |
Here it is the OpenFOAM version I've to check some parts and there are the All-reduce parts that need to be changed. For sure there are some errors except to the reduce part (I tried to finish it without compiling it) but it's just to have an idea.
Thank you!! Valerio HTML Code:
//if (Pstream::myProcNo() == 0) |
All times are GMT -4. The time now is 15:25. |