CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

while and for "loops" in parallel computing

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 7, 2011, 09:50
Default while and for "loops" in parallel computing
  #1
Member
 
valerio
Join Date: Jun 2009
Location: Nancy
Posts: 30
Rep Power: 8
Alucard is on a distinguished road
Goodmoring this is my first thread.
I'm quite new in OpenFoam (I'm working on it since last month and I come from "Fortran" so C++ and OpenFoam at the same time in 1month are a big challenge for me so i apologize if I ask trivial things!


So I describe my problem: I'm solving a coupled solidification problem so at each step in time I have:
the resolution of NS equation (piso scheme)
the resolution of Temperature (energy) equation (Voller T-gl coupling)
and the solute transfer one

the equations are coupled so some inner loops are performed at each time step in order to be consistent.

I developed it and it work in "serial". now when I try to parallelize it I've a problem on a loop I do in order to calculate a residual "res_T"
...
...
{
dimensionedScalar kappaEff=cond_T/rho0/Cp0;
volScalarField gl0=gl;
compteur=0;

res_T=10.;
conv_T=false;

while(res_T>0.0001){

compteur++;
fvScalarMatrix TEqn
(
fvm::Sp(1/runTime.deltaT(),T)
- oT/runTime.deltaT()
+ L_fus/Cp0*(gl-ogl)/runTime.deltaT()
+ fvm::div(phi,T)
- kappaEff*fvm::laplacian(T)
);

TEqn.relax();
gl0=gl;
T0=T;
iterations=TEqn.solve().nIterations();

H=Cp0*T+L_fus*gl;
#include "equilibrage.H"

res1_T=0.; res2_T=0.;
for ( int ifield = 0 ; ifield < H.size() ; ifield++ )
{
// Info<< H.size() << endl;
res1_T=res1_T+fabs(T[ifield]-T0[ifield]);
res2_T=res2_T+T[ifield];
}

res_T=res1_T/res2_T;
}
}

}

...
...
the probem is that the "while" condition doesn't work in parallel (in serial it does).
So I tried to printout H.size that in serial gives me 3000 cause I've a 50X60 2D domain
and on 8 CPUs I've the 390 value (interal plus "shared" interface cells for each part the domain is subdivided in I guess).

If I replace this kind of while with a fake "for" where I perform 10 iterations (just to be sure the coupling is done) the code works well and the results are good (I've the serial version for comparison).


SO, please, can someone explain me if there is an error (sure there is!) in what I'm writing and how I've to redefine a "parallel" working while-for loop?

Thank you in advance (and remember I'm a newbie in C++ so be kind!)
Valerio
Alucard is offline   Reply With Quote

Old   July 7, 2011, 12:24
Default
  #2
Senior Member
 
Nima Sam
Join Date: Sep 2009
Location: Tehran, Iran
Posts: 1,123
Blog Entries: 1
Rep Power: 14
nimasam is on a distinguished road
Send a message via Yahoo to nimasam
as i know "while" is not suitable for parallel processing , because when you define "while" you have a condition which should be check in every iteration so in "while" there is no definite numbers of iteration but in parallel processing you give a definite numbers of iteration to several threads to do it simultaneously and independently.
so you should use "for" but increase ur iteration number to reach that condition!
nimasam is offline   Reply With Quote

Old   July 7, 2011, 13:19
Default
  #3
Member
 
valerio
Join Date: Jun 2009
Location: Nancy
Posts: 30
Rep Power: 8
Alucard is on a distinguished road
Quote:
Originally Posted by nimasam View Post
as i know "while" is not suitable for parallel processing , because when you define "while" you have a condition which should be check in every iteration so in "while" there is no definite numbers of iteration but in parallel processing you give a definite numbers of iteration to several threads to do it simultaneously and independently.
so you should use "for" but increase ur iteration number to reach that condition!
Thank you for the answer.
Anyway from a logical point of view I still guess that doing a "while" is possibile in theory. You just calculate res_T on each subdomain and after that you sum them and you have the final res_T that is "global" and you perform the test.Otherwhise I guess there is the possibility to "recompose" the field afer the solution od the equation and perform the for loop on the global field directly.
I'm reading and looking at the same time to the openfoam (icofoam as an exemple) sources in order to understand the right way to do that!
Thank you again. if I find something (as long that my C++ training continues) I'll reply to you.
Alucard is offline   Reply With Quote

Old   July 7, 2011, 15:29
Default
  #4
Senior Member
 
chegdan's Avatar
 
Daniel P. Combest
Join Date: Mar 2009
Location: St. Louis, USA
Posts: 535
Rep Power: 18
chegdan will become famous soon enough
Quote:
Originally Posted by Alucard View Post
Thank you for the answer.
Anyway from a logical point of view I still guess that doing a "while" is possibile in theory. You just calculate res_T on each subdomain and after that you sum them and you have the final res_T that is "global" and you perform the test.Otherwhise I guess there is the possibility to "recompose" the field afer the solution od the equation and perform the for loop on the global field directly.
I'm reading and looking at the same time to the openfoam (icofoam as an exemple) sources in order to understand the right way to do that!
Thank you again. if I find something (as long that my C++ training continues) I'll reply to you.
If your goal is to sum the magnitude of the residual on the global level, then you might want to try gSumMag(res_T). I know that is used in the lduMatrix solvers to get the global residual for the linear system solvers. This can be seen in the $FOAM_SRC/OpenFOAM/matrices/lduMatrix/solvers/PCG/PCG.C file. Hope this helps.

Dan
chegdan is offline   Reply With Quote

Old   July 8, 2011, 02:48
Default
  #5
Senior Member
 
Bernhard
Join Date: Sep 2009
Location: Delft
Posts: 790
Rep Power: 12
Bernhard is on a distinguished road
And otherwise you can do something like this:
reduce(res_T,sumOp<scalar>());

By the way, a more convenient way to loop in OpenFOAM is using the forAll loop, nearly anything that you ever want to loop about as an argument. So you don't have to worry about the amount of elements in this argument.

By the way, I advise you to construct your solver in a different way. You can set the residual control in fvSolution then.
Bernhard is offline   Reply With Quote

Old   July 8, 2011, 09:49
Default
  #6
Member
 
valerio
Join Date: Jun 2009
Location: Nancy
Posts: 30
Rep Power: 8
Alucard is on a distinguished road
Quote:
Originally Posted by Bernhard View Post
And otherwise you can do something like this:
reduce(res_T,sumOp<scalar>());

By the way, a more convenient way to loop in OpenFOAM is using the forAll loop, nearly anything that you ever want to loop about as an argument. So you don't have to worry about the amount of elements in this argument.

By the way, I advise you to construct your solver in a different way. You can set the residual control in fvSolution then.
Thank you Bernhard I tried it and it seems to work! (I've to check if the results are good or not) ..I'll tell you. Thank you too for the general suggestions about the good way to write a code: I'll try to follow them.

Quote:
Originally Posted by chegdan View Post
If your goal is to sum the magnitude of the residual on the global level, then you might want to try gSumMag(res_T). I know that is used in the lduMatrix solvers to get the global residual for the linear system solvers. This can be seen in the $FOAM_SRC/OpenFOAM/matrices/lduMatrix/solvers/PCG/PCG.C file. Hope this helps.

Dan
Thank you Dan I read the PCG.C file yesterday evening and I'll test this afternoon your solution too (I'm quite sure it will work because if it works for the general Ax=b solver in parallel, it has to work for other purposes too).

Thanks!
--

Valerio
Alucard is offline   Reply With Quote

Old   July 25, 2013, 09:39
Default
  #7
Senior Member
 
Mohammad Shakil Ahmmed
Join Date: Oct 2012
Location: AU
Posts: 122
Rep Power: 5
ahmmedshakil is on a distinguished road
Hi Alucard,
Have you solved the problem? Can you share the code ?
ahmmedshakil is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 17:32.