CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM (https://www.cfd-online.com/Forums/openfoam/)
-   -   while and for "loops" in parallel computing (https://www.cfd-online.com/Forums/openfoam/90346-while-loops-parallel-computing.html)

Alucard July 7, 2011 10:50

while and for "loops" in parallel computing
 
Goodmoring this is my first thread.
I'm quite new in OpenFoam (I'm working on it since last month and I come from "Fortran" so C++ and OpenFoam at the same time in 1month are a big challenge for me so i apologize if I ask trivial things!


So I describe my problem: I'm solving a coupled solidification problem so at each step in time I have:
the resolution of NS equation (piso scheme)
the resolution of Temperature (energy) equation (Voller T-gl coupling)
and the solute transfer one

the equations are coupled so some inner loops are performed at each time step in order to be consistent.

I developed it and it work in "serial". now when I try to parallelize it I've a problem on a loop I do in order to calculate a residual "res_T"
...
...
{
dimensionedScalar kappaEff=cond_T/rho0/Cp0;
volScalarField gl0=gl;
compteur=0;

res_T=10.;
conv_T=false;

while(res_T>0.0001){

compteur++;
fvScalarMatrix TEqn
(
fvm::Sp(1/runTime.deltaT(),T)
- oT/runTime.deltaT()
+ L_fus/Cp0*(gl-ogl)/runTime.deltaT()
+ fvm::div(phi,T)
- kappaEff*fvm::laplacian(T)
);

TEqn.relax();
gl0=gl;
T0=T;
iterations=TEqn.solve().nIterations();

H=Cp0*T+L_fus*gl;
#include "equilibrage.H"

res1_T=0.; res2_T=0.;
for ( int ifield = 0 ; ifield < H.size() ; ifield++ )
{
// Info<< H.size() << endl;
res1_T=res1_T+fabs(T[ifield]-T0[ifield]);
res2_T=res2_T+T[ifield];
}

res_T=res1_T/res2_T;
}
}

}

...
...
the probem is that the "while" condition doesn't work in parallel (in serial it does).
So I tried to printout H.size that in serial gives me 3000 cause I've a 50X60 2D domain
and on 8 CPUs I've the 390 value (interal plus "shared" interface cells for each part the domain is subdivided in I guess).

If I replace this kind of while with a fake "for" where I perform 10 iterations (just to be sure the coupling is done) the code works well and the results are good (I've the serial version for comparison).


SO, please, can someone explain me if there is an error (sure there is!) in what I'm writing and how I've to redefine a "parallel" working while-for loop?

Thank you in advance (and remember I'm a newbie in C++ so be kind!)
Valerio

nimasam July 7, 2011 13:24

as i know "while" is not suitable for parallel processing , because when you define "while" you have a condition which should be check in every iteration so in "while" there is no definite numbers of iteration but in parallel processing you give a definite numbers of iteration to several threads to do it simultaneously and independently.
so you should use "for" but increase ur iteration number to reach that condition!

Alucard July 7, 2011 14:19

Quote:

Originally Posted by nimasam (Post 315207)
as i know "while" is not suitable for parallel processing , because when you define "while" you have a condition which should be check in every iteration so in "while" there is no definite numbers of iteration but in parallel processing you give a definite numbers of iteration to several threads to do it simultaneously and independently.
so you should use "for" but increase ur iteration number to reach that condition!

Thank you for the answer.
Anyway from a logical point of view I still guess that doing a "while" is possibile in theory. You just calculate res_T on each subdomain and after that you sum them and you have the final res_T that is "global" and you perform the test.Otherwhise I guess there is the possibility to "recompose" the field afer the solution od the equation and perform the for loop on the global field directly.
I'm reading and looking at the same time to the openfoam (icofoam as an exemple) sources in order to understand the right way to do that!
Thank you again. if I find something (as long that my C++ training continues) I'll reply to you.

chegdan July 7, 2011 16:29

Quote:

Originally Posted by Alucard (Post 315209)
Thank you for the answer.
Anyway from a logical point of view I still guess that doing a "while" is possibile in theory. You just calculate res_T on each subdomain and after that you sum them and you have the final res_T that is "global" and you perform the test.Otherwhise I guess there is the possibility to "recompose" the field afer the solution od the equation and perform the for loop on the global field directly.
I'm reading and looking at the same time to the openfoam (icofoam as an exemple) sources in order to understand the right way to do that!
Thank you again. if I find something (as long that my C++ training continues) I'll reply to you.

If your goal is to sum the magnitude of the residual on the global level, then you might want to try gSumMag(res_T). I know that is used in the lduMatrix solvers to get the global residual for the linear system solvers. This can be seen in the $FOAM_SRC/OpenFOAM/matrices/lduMatrix/solvers/PCG/PCG.C file. Hope this helps.

Dan

Bernhard July 8, 2011 03:48

And otherwise you can do something like this:
reduce(res_T,sumOp<scalar>());

By the way, a more convenient way to loop in OpenFOAM is using the forAll loop, nearly anything that you ever want to loop about as an argument. So you don't have to worry about the amount of elements in this argument.

By the way, I advise you to construct your solver in a different way. You can set the residual control in fvSolution then.

Alucard July 8, 2011 10:49

Quote:

Originally Posted by Bernhard (Post 315258)
And otherwise you can do something like this:
reduce(res_T,sumOp<scalar>());

By the way, a more convenient way to loop in OpenFOAM is using the forAll loop, nearly anything that you ever want to loop about as an argument. So you don't have to worry about the amount of elements in this argument.

By the way, I advise you to construct your solver in a different way. You can set the residual control in fvSolution then.

Thank you Bernhard I tried it and it seems to work! (I've to check if the results are good or not) ..I'll tell you. Thank you too for the general suggestions about the good way to write a code: I'll try to follow them.

Quote:

Originally Posted by chegdan (Post 315228)
If your goal is to sum the magnitude of the residual on the global level, then you might want to try gSumMag(res_T). I know that is used in the lduMatrix solvers to get the global residual for the linear system solvers. This can be seen in the $FOAM_SRC/OpenFOAM/matrices/lduMatrix/solvers/PCG/PCG.C file. Hope this helps.

Dan

Thank you Dan I read the PCG.C file yesterday evening and I'll test this afternoon your solution too (I'm quite sure it will work because if it works for the general Ax=b solver in parallel, it has to work for other purposes too).

Thanks!:cool:
--

Valerio

ahmmedshakil July 25, 2013 10:39

Hi Alucard,
Have you solved the problem? Can you share the code ?


All times are GMT -4. The time now is 04:45.