[palmera2]

Hi!

I'm programming a parallel application for interpolation and I need to exchange information between processors. In particular, I have a List p with different size and elements in each processor and I want to merge all p and share it among processors.

I split the problem in two parts:

1. Pass p from the different processors to processor0 and merge them in a List P
2. Share List P among all processors

For 1) I did the following:

label n=Pstream::nProcs();
DynamicList<point> samplePosCellCenter; 

// Sharing list samplePosCellCenter among processors
// Perhaps there is a more compact way of doing it through 
// reduce(List<T>, binaryOP<>)

//---------------------------------------------------------------------------//
// Compiling list on master processor
//---------------------------------------------------------------------------//
if (Pstream::myProcNo() == 0)
{
    // Output variable for processor 0
    forAll(samplePosCellID, cellID)
    {
        point cellCenter = mesh.C()[cellID];
        samplePosCellCenter.append(cellCenter);
    }

    // Information exchange between processors
    for(label i=1; i<n; i++)
    {
        // Local declaration of masterPlaceHolder
        List<point> masterPlaceHolder;
        // Create an input stream for processor i
        IPstream masterStream(Pstream::blocking, i);
        masterStream >> masterPlaceHolder;

        // Add elements of placeHolder to samplePosCellCenter
        forAll(masterPlaceHolder, proci)
        {
            point cellCenterI = masterPlaceHolder[proci];
            samplePosCellCenter.append(cellCenterI);
        }
    }
}
else
{
    // Create stream to send slavePlaceHolder to the main proc
    DynamicList<point> slavePlaceHolder;
    forAll(samplePosCellID, cellID)
    {
        point cellCenter = mesh.C()[cellID];
        slavePlaceHolder.append(cellCenter);
    }
    OPstream slaveStream(Pstream::blocking, 0);
    slaveStream << slavePlaceHolder;
}
//---------------------------------------------------------------------------//

For 2) I did as follows:

//---------------------------------------------------------------------------//
// Sharing master's list among slaves
//---------------------------------------------------------------------------//
if (Pstream::myProcNo() == 0)
{
    // Information exchange between processors
    for(label i=1; i<n; i++)
    {
        // Create an output stream for processor i
        OPstream masterStream(Pstream::blocking, i);
        masterStream << samplePosCellCenter;
    }
}
else
{
    // Create input stream to send share samplePosCellCenter from master to
    // slave processors
    IPstream slaveStream(Pstream::blocking, 0);
    slaveStream >> samplePosCellCenter;
}
//---------------------------------------------------------------------------//

The code above does work, but I'm pretty sure that there is a "cleaner" way of doing it. In the beginning I was thinking of using the reduce operation, but I didn't know which BinaryOp to use.


My question: Is there a cleaner way to merge lists from different processors?





Thank you in advance!

[olesen]

Not really sure what you want your final access to be. One flat list that is identical on all processors? Is this list appended with/without duplicates? Or do you just want everyone to know everything about each other?


https://www.openfoam.com/documentati...d51a9a321792d9


Eg,

.. taken from laserDTRM.C as an example



List<pointField> positions(Pstream::nProcs());


... fill in per processor

     

Pstream::gatherList(positions);
Pstream::scatterList(positions);

[palmera2]

Dear Olesen,


Thank you very much for your answer and time. I have one list different in size and contents in each processor and I want everyone to know about each other (to share the contents of the list). For example, for a parallel simulation using 2 cores I could have the following lists:

[0] Processor 0: 1(0 1)
[1] Processor 1: 2(1)

And I would like to have the following:

[0] Processor 0: 3(0 1 1)
[1] Processor 1: 3(0 1 1)

Your solution indeed solved the problem! I attach a test implementation below:

// OpenFOAM
#include "fvCFD.H"

int main(int argc, char *argv[])
{
#   include "setRootCase.H"
#   include "createTime.H"
#   include "createMesh.H"

//---------------------------------------------------------------------------//
// Creating a different (both in size and contents) list for each processor
//---------------------------------------------------------------------------//
    
    // Number of processors
    label n = Pstream::nProcs();

    // Create a different test list in each processor
    DynamicList<label> prociList;
    label proci = Pstream::myProcNo(); 
    for (label i=proci; i<n;i++)
    {
        prociList.append(i);
    }
    // Printing list's elements for each processor
    Pout<< "Processor " << proci << ": " << prociList 
        << nl; 
    Info<< nl << endl;

//---------------------------------------------------------------------------//


//---------------------------------------------------------------------------//
// Implementing Olesen's reply on cfd-online
//---------------------------------------------------------------------------//
    
    // Creating a placeholder list
    List<List<label> > shareList(n);
    // Transfering prociList into the proci'th element of placeholder 
    // list shareList. This removes the contents of prociList. 
    shareList[proci].transfer(prociList);
    // Currently, only the proci'th element of list shareList is populated.

    // Gather the different shareList's of slave processors in master processor
    Pstream::gatherList(shareList);
    // Share masters' shareList to the rest of processors
    Pstream::scatterList(shareList);
    // Creating a single list with all elements
    forAll(shareList, listI)
    {
        prociList.append(shareList[listI]);
    }
    Pout<< "Size of resulting prociList: " << prociList.size() << nl
        << "prociList in proc " << proci << ": " << prociList << nl;

//---------------------------------------------------------------------------//
}

Thanks again!

[olesen]

For what it's worth, you can add some more modern coding - should work with 1712 and later, but you should check.

List<labelList> shareList(Pstream::nProcs()) ;
shareList[Pstream::myProcNo()].transfer(prociList);

Pstream::gatherList(shareList);
Pstream::scatterList(shareList);
   
// Flatten
for (labelList& list : shareList) 
{
     procList.append(std::move(list)):
}

If you are dealing with a list of labels, the additional std::move() doesn't help much on the append (doesn't hurt either), but if your lists contain something "heavier" (like longer strings etc) this will recover their memory element-wise and reduce some allocations.