CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM

particles parallelization transfere cloud move

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   July 2, 2013, 13:55
Default particles parallelization transfere cloud move
  #1
Senior Member
 
Join Date: Dec 2009
Posts: 112
Rep Power: 16
heavy_user is on a distinguished road
Hi All,
i ran into a huge problem and i need help!
I am doing a LES-PDF method using MC particles and the particles carry a lot of information (each particle has about 35 doubles which are transfered via stream and not reinitiated by a preinitialization list).
The paralellization is REALLY bad!
Using 48 cores - one board, quadsocket, xeon cpus - the time is spend as follows (318087 LES cells, 1556621 particles ):
- 99.4 % of the time is used for particles(rest for LES)
- The particle move takes 96.7 % of the particle time
- and the transfer takes 94 % of that time

so essentially the computer is only bussy moving particles.
The routine consuming all the resources (in myCloud.move() ) is
Code:
while ( true )      <- what the heck is that deathloop any ways !?
....
....
..
        if (!Pstream::parRun())
        {
            break;
        }

        // Allocate transfer buffers
        PstreamBuffers pBufs(Pstream::nonBlocking);

        // Stream into send buffers
        forAll(particleTransferLists, i)
        {
            if (particleTransferLists[i].size())
            {
                UOPstream particleStream
                (
                    neighbourProcs[i],
                    pBufs
                );

                particleStream
                    << patchIndexTransferLists[i]
                    << particleTransferLists[i];
            }
        }

        // Set up transfers when in non-blocking mode. Returns sizes (in bytes)
        // to be sent/received.
        labelListList allNTrans(Pstream::nProcs());

        pBufs.finishedSends(allNTrans);

        bool transfered = false;

        forAll(allNTrans, i)
        {
            forAll(allNTrans[i], j)
            {
                if (allNTrans[i][j])
                {
                    transfered = true;
                    break;
                }
            }
        }

        if (!transfered)
        {
            break;
        }

        // Retrieve from receive buffers
        forAll(neighbourProcs, i)
        {
            label neighbProci = neighbourProcs[i];

            label nRec = allNTrans[neighbProci][Pstream::myProcNo()];

            if (nRec)
            {
                UIPstream particleStream(neighbProci, pBufs);

                labelList receivePatchIndex(particleStream);

                IDLList<myHardBallParticle> newParticles
                (
                    particleStream,
                    //typename myHardBallParticle::iNew(polyMesh_)
             myHardBallParticle::iNew(*this)
                );

                label pI = 0;

        forAllIter(myIncompressibleCloud, newParticles, newpIter)
                {
                    myHardBallParticle& newp = newpIter();

                    label patchI = procPatches[receivePatchIndex[pI++]];

                    newp.correctAfterParallelTransfer(patchI, td);

                    addParticle(newParticles.remove(&newp));
                }
            }
Thats from OpenFOAM-2.1x (which promised new particle tracking )
I would appreciate any ideas or comments!!!
heavy_user is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to determine the number of particles injected. welch FLUENT 2 January 18, 2024 05:08
UDF for deleting particles in DPM imanmirzaii Fluent UDF and Scheme Programming 12 November 25, 2020 20:27
trying to simulate two-phase jet flow with particles in surface injection ajkratos FLUENT 5 March 3, 2015 22:33
Straight through cyclone with particles rogerhoffmann CFX 1 May 21, 2012 19:42
particles model ati_ros61 FLOW-3D 3 December 6, 2009 17:03


All times are GMT -4. The time now is 03:05.