[heavy_user]

Hi All,
i ran into a huge problem and i need help!
I am doing a LES-PDF method using MC particles and the particles carry a lot of information (each particle has about 35 doubles which are transfered via stream and not reinitiated by a preinitialization list).
The paralellization is REALLY bad!
Using 48 cores - one board, quadsocket, xeon cpus - the time is spend as follows (318087 LES cells, 1556621 particles ):
- 99.4 % of the time is used for particles(rest for LES)
- The particle move takes 96.7 % of the particle time
- and the transfer takes 94 % of that time

so essentially the computer is only bussy moving particles.
The routine consuming all the resources (in myCloud.move() ) is

while ( true )      <- what the heck is that deathloop any ways !?
....
....
..
        if (!Pstream::parRun())
        {
            break;
        }

        // Allocate transfer buffers
        PstreamBuffers pBufs(Pstream::nonBlocking);

        // Stream into send buffers
        forAll(particleTransferLists, i)
        {
            if (particleTransferLists[i].size())
            {
                UOPstream particleStream
                (
                    neighbourProcs[i],
                    pBufs
                );

                particleStream
                    << patchIndexTransferLists[i]
                    << particleTransferLists[i];
            }
        }

        // Set up transfers when in non-blocking mode. Returns sizes (in bytes)
        // to be sent/received.
        labelListList allNTrans(Pstream::nProcs());

        pBufs.finishedSends(allNTrans);

        bool transfered = false;

        forAll(allNTrans, i)
        {
            forAll(allNTrans[i], j)
            {
                if (allNTrans[i][j])
                {
                    transfered = true;
                    break;
                }
            }
        }

        if (!transfered)
        {
            break;
        }

        // Retrieve from receive buffers
        forAll(neighbourProcs, i)
        {
            label neighbProci = neighbourProcs[i];

            label nRec = allNTrans[neighbProci][Pstream::myProcNo()];

            if (nRec)
            {
                UIPstream particleStream(neighbProci, pBufs);

                labelList receivePatchIndex(particleStream);

                IDLList<myHardBallParticle> newParticles
                (
                    particleStream,
                    //typename myHardBallParticle::iNew(polyMesh_)
             myHardBallParticle::iNew(*this)
                );

                label pI = 0;

        forAllIter(myIncompressibleCloud, newParticles, newpIter)
                {
                    myHardBallParticle& newp = newpIter();

                    label patchI = procPatches[receivePatchIndex[pI++]];

                    newp.correctAfterParallelTransfer(patchI, td);

                    addParticle(newParticles.remove(&newp));
                }
            }

Thats from OpenFOAM-2.1x (which promised new particle tracking )
I would appreciate any ideas or comments!!!