|
[Sponsors] |
Bug in fvc descretization toolbox during paralle run |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
September 27, 2007, 13:59 |
Description:
I have run into
|
#1 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
Description:
I have run into a problem during the parallization a solver which, unless I am mistaken, may be a bug in some of the descretization schemes / processor communications. in the solver, I am performing the following higher order derivative operations: newFlux = fvc::snGrad ( fvc::laplacian(gamma)) * mesh.magSf(); gammaNew = gammaOld - fvc::div(newFlux, gamma); gammaNew.correctBoundaryConditions(); I am using a Gauss cubic corrected scheme for the laplacian, corrected for the surface normal gradient, and a vanLeer scheme for the div calculation. When operated on a single processor, there is no problem (i.e. the solution approaches that which is expected, see attached profiles). When run on 4 processors, the solution changes depending on the way the grid is decomposed. I have shown results for the grid split 4 times in the x-direction, and for the grid split in the x and y direction. The results for a single processor is overlaid with the grid split locations of the other simulations for comparison. It appears as if either the divergence term or the snGrad of laplacian(gamma) is not being propertly evaluated across the processor boundaries. I have traced every other calculation related to this portion of the code, and as far as I can tell the error is being introduced through these expressions. Note that when run in 3D, variations in the value of gamma appeared to propegate outwards from the locations where the processor boundarys were present between the decomposed sections of the grid. The grid in all cases is orthogonal, consisting of a standard hex with equal cell widths along each face. The scope of simulations I am looking to perform are too large to do in a reasonable time on single processors. If anyone knows how I can resolve this issue, your input would be appreciated. Solver/Application: Custom Solver, Bug related to fvc:: operations in parallel Source file: N/A Testcase: N/A Platform: linux cluster. Version: 1.4 |
|
September 27, 2007, 14:04 |
Single Processor Solution:
|
#2 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
Single Processor Solution:
Split 4 times Horizontally: Split once in each direction: |
|
September 27, 2007, 14:06 |
As it may be difficult to note
|
#3 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
As it may be difficult to note the differences by viewing them like this, the difference becomes relatively easy to see if you download them and overlay them (or view them sequentially).
|
|
September 27, 2007, 14:19 |
What happens if you use Gauss
|
#4 |
Senior Member
Hrvoje Jasak
Join Date: Mar 2009
Location: London, England
Posts: 1,905
Rep Power: 33 |
What happens if you use Gauss linear corrected instead?
Hrv
__________________
Hrvoje Jasak Providing commercial FOAM/OpenFOAM and CFD Consulting: http://wikki.co.uk |
|
September 27, 2007, 15:03 |
I applied the Gauss linear cor
|
#5 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
I applied the Gauss linear corrected scheme to the laplacian of gamma, and there was no visible change from the results shown above.
I have had similar results using the MUSCL, limitedVanLeer and limitedMUSCL schemes for the convection term. I have only tried corrected for the snGrad term. |
|
September 27, 2007, 15:48 |
I have found a solution, altho
|
#6 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
I have found a solution, although it seems rather odd. The newFlux variable I have been using was either defined locally (i.e. surfaceScalarField newFlux = expression) or defined in the createField.H file with the IOobject configuration including IOobject::NO_READ and IOobject::NO_WRITE (which I assumed would be fine as I have no interest in writing these variables to the time folders). However, when IOobject::AUTO_WRITE is used, this issue is no longer present...
Is that because the AUTO_WRITE property needs to be set in order for the processor boundaries to be taken into account when subsequently working with the variable...? |
|
September 27, 2007, 16:00 |
Scratch the last. it was the
|
#7 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
Scratch the last. it was the other modification I had made.
I have a zero-flux condition on the wall of the grid, which I need to correct the boundary conditions for. The error is being introduced when I call correctBoundaryConditions() for the newFlux variable... As far as I can tell, the processor boundary conditions are over-writing the current value of newFlux with whatever was stored during the previous time step. |
|
September 27, 2007, 16:09 |
This raises another issue for
|
#8 |
Member
Adam Donaldson
Join Date: Mar 2009
Location: Ottawa, Ontario, Canada
Posts: 37
Rep Power: 17 |
This raises another issue for me though... how do I ensure that the newFlux is equal to zero where a zero-flux boundary condition exists without over-writing the processor boundary with the previous values... The velocity uses the correctBoundaryConditions() call in the pressure-velocity coupling loop without a similar problem, any idea why it is appearing with this variable?
|
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
How to run ICEM-CFD in paralle model? | wayne | CFX | 5 | April 4, 2013 18:39 |
How or When I can get CGNS Paralle Abillity? | dragson | Main CFD Forum | 1 | June 12, 2007 11:11 |
Gas Dynamics Toolbox | Alina | Main CFD Forum | 0 | October 21, 2005 13:23 |
descretization of equations in Fuent | faiz rauf | FLUENT | 1 | October 3, 2004 09:01 |
Matlab pde toolbox | srijit goswami | Main CFD Forum | 1 | January 30, 2001 20:34 |