CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Programming & Development

New Application halts without error in parallel run

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   April 20, 2016, 10:58
Default Solved: New Application halts without error in parallel run
  #1
New Member
 
P. Silkeit
Join Date: Jun 2015
Posts: 6
Rep Power: 3
psilkeit is on a distinguished road
Dear FOAMers,
I'm working on an apllication which utilizes the values of a scalarField interpolated to the faces and the area of the faces of a faceZone to calculate a scalar value at an observer point at an artificial time step which is stored in a scalarListList. The application runs fine on a single core and produces the correct results, when the calculation is performed in parallel the application stalls without aborting or an error message when the calculation is performed for the first time.

After some research into the matter i think the parallelization does not work automatically since i am not working with the usual fields. The search lead me to try the following approach to write out the data ( i think the calculation runs fine in parallel aswell but its a processor comunication problem but thats just a guess...)
Code:
// The important variable
scalarListList  stuff  // the stuff i want to calculate
stuff = scalarListList( number of observers);
forAll(observers,obsI)
{
   //artificially calculated timesteps
   stuff[obsI]=scalarList( number of artificial time steps ) 
}
// The calculation of the variable
forAll(observerPoints,pointI)
{
  forAll(zoneFaces,faceI)
  {
      stuff[pointI][artificialTimeI]= calculation dependent on pointI and faceI; 
      // the correct artificialTimeI is found in side the above calculation
}
}

// attempt to gather and write the variable
    List<List< List<scalar> >> gatheredStuff(Pstream::nProcs());
    gatheredStuff[Pstream::myProcNo()] = stuff;
    Pstream::gatherList(gatheredStuff);

// Only the master proc should write out the accumulated data
    if (Pstream::master())
    {
        if(runTime.write())
        {
                mkDir(runTime.path()/"Results");
                fileName outputFile("Results.TXT");
                OFstream os(runTime.path()/"Results"/outputFile);
                os << "Observer Points Test " << endl;
                forAll(artificialTimeSteps, stepI)
                {
                  os << timeSteps[stepI];
                  os << "\t";
                     forAll(observerPoints,pointI)
                      {
                         os << gatheredStuff[pointI][stepI];
                         os << "\t";
                      }
                  os << endl;
                }
        }
    }
As I said the calculation and the writing of the data works like a charm on a single processor but the application freezes in parallel when it is performed for the first time.

Any input, suggestion or help giving me a hint why this freezes and how to prevent this is highly appreciated.

Best regards,
Patrick

Last edited by psilkeit; April 26, 2016 at 10:36. Reason: Solved
psilkeit is offline   Reply With Quote

Old   April 21, 2016, 21:38
Default
  #2
Member
 
Andrew King
Join Date: Mar 2009
Location: Perth, Western Australia, Australia
Posts: 82
Rep Power: 9
andersking is on a distinguished road
I think you should use runTime.outputTime() in your test instead of runTime.write(). write() is linked to fields, which have parallel aware IO. starting this only on the master ends up with a communications mismatch.

The other option is to switch the Pstream::master() check with runTime.write() (but this may also rewrite the fields).

Cheers,
Andrew
__________________
Dr Andrew King
Fluid Dynamics Research Group
Curtin University
andersking is online now   Reply With Quote

Old   April 22, 2016, 03:43
Default
  #3
New Member
 
P. Silkeit
Join Date: Jun 2015
Posts: 6
Rep Power: 3
psilkeit is on a distinguished road
Dear Andrew,

thanks for your reply that makes sense and I changed it accordingly, it doesn't solve my problem though...

To further locate the problem I added the following Pout statements:
Code:
Pout << "Processor Numer 1 " << Pstream::myProcNo() << endl;    
List<List< List<scalar> >> gatheredStuff(Pstream::nProcs());

Pout << "Processor Numer 2 " << Pstream::myProcNo() << endl;    
gatheredStuff[Pstream::myProcNo()] = stuff;

Pout << "Processor Numer 3 " << Pstream::myProcNo() << endl;    
Pstream::gatherList(gatheredStuff);

Pout << "Processor Numer 4 " << Pstream::myProcNo() << endl;    

if (Pstream::master())
{ 
   if (runTime.outputTime())
   {
        etc...
The output gives:
Code:
Performing the New Application 

[1] Processor Number 1 1
[1] Processor Number 2 1
[1] Processor Number 3 1
So my guess is that the Pstream::gatherList() function does not work properly in my case... Could it be that it cannot operate on a ListListList?

Secondly I parallelize on two processors shouldn't the Pout give two outputs for each call then ? Or did is misunderstand something there?

Any further help is highly appreciated.

Cheers
psilkeit is offline   Reply With Quote

Old   April 26, 2016, 10:35
Smile Solved
  #4
New Member
 
P. Silkeit
Join Date: Jun 2015
Posts: 6
Rep Power: 3
psilkeit is on a distinguished road
Dear All,
I managed to solve the problem. The Pstream::gatherList comand does work. What i didn't realize was that I had to do further operations on the list to add together the corresponding parts that are contained in the list after gathering and then write it out.

I thus reduce the gatheredStuff to a list equal in size to stuff by adding the corresponding parts in a for loop

Therefor, the skeleton code above is actually working

I changed the write procedure according to this post OFstream for parallel processing
so the output file is written to the case directory and not the master processor directory.

Thanks to everyone who looked at this.

Greetz
psilkeit is offline   Reply With Quote

Reply

Tags
parallel, parallel code

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Case running in serial, but Parallel run gives error atmcfd OpenFOAM Running, Solving & CFD 18 March 26, 2016 13:40
Some questions about a multi region case run in parallel zfaraday OpenFOAM Running, Solving & CFD 4 October 28, 2015 13:10
problem during mpi in server: expected Scalar, found on line 0 the word 'nan' muth OpenFOAM Running, Solving & CFD 2 April 10, 2015 05:42
Can not run OpenFOAM in parallel in clusters, help! ripperjack OpenFOAM Running, Solving & CFD 5 May 6, 2014 15:25
parallel Grief: BoundaryFields ok in single CPU but NOT in Parallel JR22 OpenFOAM Running, Solving & CFD 2 April 19, 2013 16:49


All times are GMT -4. The time now is 02:31.