CFD Online Logo CFD Online URL
Home > Forums > OpenFOAM

particles parallelization transfere cloud move

Register Blogs Members List Search Today's Posts Mark Forums Read

LinkBack Thread Tools Display Modes
Old   July 2, 2013, 12:55
Default particles parallelization transfere cloud move
Senior Member
Join Date: Dec 2009
Posts: 112
Rep Power: 7
heavy_user is on a distinguished road
Hi All,
i ran into a huge problem and i need help!
I am doing a LES-PDF method using MC particles and the particles carry a lot of information (each particle has about 35 doubles which are transfered via stream and not reinitiated by a preinitialization list).
The paralellization is REALLY bad!
Using 48 cores - one board, quadsocket, xeon cpus - the time is spend as follows (318087 LES cells, 1556621 particles ):
- 99.4 % of the time is used for particles(rest for LES)
- The particle move takes 96.7 % of the particle time
- and the transfer takes 94 % of that time

so essentially the computer is only bussy moving particles.
The routine consuming all the resources (in myCloud.move() ) is
while ( true )      <- what the heck is that deathloop any ways !?
        if (!Pstream::parRun())

        // Allocate transfer buffers
        PstreamBuffers pBufs(Pstream::nonBlocking);

        // Stream into send buffers
        forAll(particleTransferLists, i)
            if (particleTransferLists[i].size())
                UOPstream particleStream

                    << patchIndexTransferLists[i]
                    << particleTransferLists[i];

        // Set up transfers when in non-blocking mode. Returns sizes (in bytes)
        // to be sent/received.
        labelListList allNTrans(Pstream::nProcs());


        bool transfered = false;

        forAll(allNTrans, i)
            forAll(allNTrans[i], j)
                if (allNTrans[i][j])
                    transfered = true;

        if (!transfered)

        // Retrieve from receive buffers
        forAll(neighbourProcs, i)
            label neighbProci = neighbourProcs[i];

            label nRec = allNTrans[neighbProci][Pstream::myProcNo()];

            if (nRec)
                UIPstream particleStream(neighbProci, pBufs);

                labelList receivePatchIndex(particleStream);

                IDLList<myHardBallParticle> newParticles
                    //typename myHardBallParticle::iNew(polyMesh_)

                label pI = 0;

        forAllIter(myIncompressibleCloud, newParticles, newpIter)
                    myHardBallParticle& newp = newpIter();

                    label patchI = procPatches[receivePatchIndex[pI++]];

                    newp.correctAfterParallelTransfer(patchI, td);

Thats from OpenFOAM-2.1x (which promised new particle tracking )
I would appreciate any ideas or comments!!!
heavy_user is offline   Reply With Quote


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

Similar Threads
Thread Thread Starter Forum Replies Last Post
trying to simulate two-phase jet flow with particles in surface injection ajkratos FLUENT 5 March 3, 2015 22:33
UDF for deleting particles in DPM imanmirzaii Fluent UDF and Scheme Programming 5 January 29, 2013 08:02
Straight through cyclone with particles rogerhoffmann CFX 1 May 21, 2012 18:42
particles model ati_ros61 FLOW-3D 3 December 6, 2009 17:03
how to determine the number of particles injected. welch FLUENT 1 November 1, 2005 06:11

All times are GMT -4. The time now is 09:41.