CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Programming & Development (https://www.cfd-online.com/Forums/openfoam-programming-development/)
-   -   Why my program cannot ran in parallel??? (https://www.cfd-online.com/Forums/openfoam-programming-development/215148-why-my-program-cannot-ran-parallel.html)

wdx_cfd February 24, 2019 03:21
Why my program cannot ran in parallel???
 
Hi FOAMers,

I am doing some modifications on interFOAM, OpenFOAM v6. It is something about the dynamic mesh and I have changed the functions "pimple.loop()" to another one which is developed by myself. My new program runs good with the single core but will crash if I run it parallel. Here is the error information:

Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected '(', found on line 0 the punctuation token ':'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 131.
[1]
FOAM parallel run exiting
and another similar case:
Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected <int> or '(', found on line 0 the word 'g'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 148.
[1]
FOAM parallel run exiting
[1]
[32]
[32]
[32] --> FOAM FATAL IO ERROR:
[32] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[32]
[32] file: IOstream at line 0.
[32]
[32]    From function void Foam::IOstream::fatalCheck(const char*) const
[32]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[32]
FOAM parallel run exiting
[32]
[8]
[8]
[8] --> FOAM FATAL IO ERROR:
[8] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[8]
[8] file: IOstream at line 0.
[8]
[8]    From function void Foam::IOstream::fatalCheck(const char*) const
[8]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[8]
FOAM parallel run exiting
I find that this error happens at the moment of (CourantNo.H):
Code:
   
scalarField sumPhi
    (
        fvc::surfaceSum(mag(phi))().primitiveField()
    );
What's more, this error will occur after the pimple outer loop is stopped by my newly added criterion (i.e. entering into the next time step and calculating the courantNo as mentioned above). It seems that there is something important in the function of "pimple.loop()" for parallel running but not ciritical for serial running. I have read "pimple.loop()" but I think I did not find anything useful about this parallel running problem.

I don't know much about the parallel running, but I am very confused why this happens? Could anyone give me some suggestions?

Thanks a lot！

WDX

massive_turbulence February 24, 2019 15:51
Quote:
Originally Posted by wdx_cfd (Post 725803)
Hi FOAMers,

I am doing some modifications on interFOAM, OpenFOAM v6. It is something about the dynamic mesh and I have changed the functions "pimple.loop()" to another one which is developed by myself. My new program runs good with the single core but will crash if I run it parallel. Here is the error information:

Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected '(', found on line 0 the punctuation token ':'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 131.
[1]
FOAM parallel run exiting
and another similar case:
Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected <int> or '(', found on line 0 the word 'g'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 148.
[1]
FOAM parallel run exiting
[1]
[32]
[32]
[32] --> FOAM FATAL IO ERROR:
[32] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[32]
[32] file: IOstream at line 0.
[32]
[32]    From function void Foam::IOstream::fatalCheck(const char*) const
[32]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[32]
FOAM parallel run exiting
[32]
[8]
[8]
[8] --> FOAM FATAL IO ERROR:
[8] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[8]
[8] file: IOstream at line 0.
[8]
[8]    From function void Foam::IOstream::fatalCheck(const char*) const
[8]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[8]
FOAM parallel run exiting
I find that this error happens at the moment of (CourantNo.H):
Code:
   
scalarField sumPhi
    (
        fvc::surfaceSum(mag(phi))().primitiveField()
    );
What's more, this error will occur once the number of the current pimple outer loop larger than the total number that read from the fvSolution (I still need the value in the dictionary because the first two time step is controled by the pimple algorithm). I have read "pimple.loop()" but I think I did not find anything useful about this parallel running problem.

I don't know much about the parallel running, but I am very confused why this happens? Could anyone give me some suggestions?

Thanks a lot！

WDX
My guess with this function is that something is wrong with the flux between the different CPU spaces or domains. Without seeing your entire case I doubt anyone will be able to help you.

Code:
   
scalarField sumPhi
    (
        fvc::surfaceSum(mag(phi))().primitiveField()
    );

wdx_cfd February 24, 2019 19:42
Quote:
Originally Posted by massive_turbulence (Post 725829)
My guess with this function is that something is wrong with the flux between the different CPU spaces or domains. Without seeing your entire case I doubt anyone will be able to help you.

Code:
   
scalarField sumPhi
    (
        fvc::surfaceSum(mag(phi))().primitiveField()
    );

Thank you for your reply. The entire code is a little bit complicated because I modified both the solver and the sixDof module. So updating the entire code is not an easy thing...

I think the problem is due to my newly added criterion for the pimple outer loop. Once the pimple outer loop is stopped by it, the program will crash (when parallel running).

I'll continue to find out the reason why this error occurs.

:D

einstein_zee June 27, 2019 05:49
Quote:
Originally Posted by wdx_cfd (Post 725803)
Hi FOAMers,

I am doing some modifications on interFOAM, OpenFOAM v6. It is something about the dynamic mesh and I have changed the functions "pimple.loop()" to another one which is developed by myself. My new program runs good with the single core but will crash if I run it parallel. Here is the error information:

Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected '(', found on line 0 the punctuation token ':'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 131.
[1]
FOAM parallel run exiting
and another similar case:
Code:
[1] --> FOAM FATAL IO ERROR:
[1] incorrect first token, expected <int> or '(', found on line 0 the word 'g'
[1]
[1] file: IOstream at line 0.
[1]
[1]    From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = Foam::Field<Foam::Vector<double> >]
[1]    in file /home/wdx/OpenFOAM-6/src/OpenFOAM/lnInclude/ListIO.C at line 148.
[1]
FOAM parallel run exiting
[1]
[32]
[32]
[32] --> FOAM FATAL IO ERROR:
[32] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[32]
[32] file: IOstream at line 0.
[32]
[32]    From function void Foam::IOstream::fatalCheck(const char*) const
[32]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[32]
FOAM parallel run exiting
[32]
[8]
[8]
[8] --> FOAM FATAL IO ERROR:
[8] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[8]
[8] file: IOstream at line 0.
[8]
[8]    From function void Foam::IOstream::fatalCheck(const char*) const
[8]    in file db/IOstreams/IOstreams/IOstream.C at line 109.
[8]
FOAM parallel run exiting
I find that this error happens at the moment of (CourantNo.H):
Code:
   
scalarField sumPhi
    (
        fvc::surfaceSum(mag(phi))().primitiveField()
    );
What's more, this error will occur after the pimple outer loop is stopped by my newly added criterion (i.e. entering into the next time step and calculating the courantNo as mentioned above). It seems that there is something important in the function of "pimple.loop()" for parallel running but not ciritical for serial running. I have read "pimple.loop()" but I think I did not find anything useful about this parallel running problem.

I don't know much about the parallel running, but I am very confused why this happens? Could anyone give me some suggestions?

Thanks a lot！

WDX
Hey there,

I doubt that the problem would be because of "CourantNo.H". Cause interFoam already works perfectly fine in parallel :). To pinpoint the real trouble you might switch to run in debug mode. Moreover, a basic understanding of how MPI works and how OF manages it is needed(at the very least, in case you want to remedy this on your own).

Linxiangfeng April 13, 2023 23:15
Quote:
Originally Posted by wdx_cfd (Post 725844)
Thank you for your reply. The entire code is a little bit complicated because I modified both the solver and the sixDof module. So updating the entire code is not an easy thing...

I think the problem is due to my newly added criterion for the pimple outer loop. Once the pimple outer loop is stopped by it, the program will crash (when parallel running).

I'll continue to find out the reason why this error occurs.

:D
Have you solved your problem? I met similar problems, but had no idea what to do with it

olesen April 14, 2023 03:56
Quote:
Originally Posted by Linxiangfeng (Post 848154)
Have you solved your problem? I met similar problems, but had no idea what to do with it
This is a typical beginners problem when writing things for parallel. You need to ensure that your program logic is consistent. It is no problem if you stop based on some a global value (eg, the residual), or based on a value which is globally consistent (eg, the iterations of a loop), but you CANNOT stop based on some local value without ensuring that everyone knows about it.
Examples,
Code:
// Exit when there are more than N cells with excess temperature


volScalarField T = ....;
 
while (....)
{
    label nTooHot = 0;
    forAll(T, celli)
    {
        if (T[celli] > 1000) ++nTooHot;
    }

    if (nTooHot > 20)
    {
        Pout<< "Have " << nTooHot << " cells exceeding temperature" << nl;
        break;
    }
}
Can you see what will happen here? One (or more) processors may hit that condition, while the others do not. This means that the while loop continues for those other branches and they will continue to calculate, solve, communicating but will not be in the same program branch as the ones that already exited the loop.

There are several remedies, depending on which logic you actually intend.
For example,
Code:
// Break on overall value.
label nTooHot = ...;  // summation loop as before

// make into a global

reduce(nTooHot, sumOp<label>());
if (nTooHot > 20) { break; }

// This is frequently used, so there is a convenient wrapper.

label nTooHot = ...;  // summation loop as before

 if (returnReduce(nTooHot, sumOp<label>()) > 20)
{
      break;
}
Or else
Code:
// Break on local logic
label nTooHot = ...;  // summation loop as before

bool needToStop = (nTooHot > 20);


// a)
if (returnReduce(needToStop, orOp<bool>())) break;


// b)

if (returnReduce((nTooHot > 20), orOp<bool>())) break;

 

// c)

 reduce(needToStop, orOp<bool>());
 if (needToStop)
{
    break;
}



// Using OpenFOAM 2212
 
if (returnReduceOr(nTooHot > 20))
{
    break;
}
The reduce and returnReduce functions have internal parallel/non-parallel checks, so they also work in non-parallel.


All times are GMT -4. The time now is 06:35.