[lakeat]

Dear all,

Have a nice day!

I have some questions,

1. How to decide which solver should be used, GAMG or smoothSolver or PCG? I have read some papers and some threads, but still have no idea about the guidelines.
   
2. What is the difference between pimplefoam and pisofoam?

Thank you.

[chiven]

Hi, Daniel, nice to meet you herein.

About question 2,
Navier-Stokes equations solving methods are different, they are PISO algorithm and PIMPLE (merged PISO-SIMPLE) algorithm respectively. Maybe the following references can help you to understand more details:
1. J.H. Ferziger and M. Peric Computational Methods for Fluid Dynamics 3rd ed. Springer 2002.
2. Hrvoje Jasak, PhD 1996, PDF of thesis posted at: Error analysis and estimation in the Finite Volume method with applications to fluid flows.
3. Henrik Rusche, PhD 2002 Computational fluid dynamics of dispersed two-phase flows at high phase fractions.

about question 1, I also am not so clear, but for the p, I always use smoothsolver to get a faster convergence.

Best regards,
Chiven

[sandy]

Originally Posted by lakeat

Dear all,

Have a nice day!

I have some questions,

1. How to decide which solver should be used, GAMG or smoothSolver or PCG? I have read some papers and some threads, but still have no idea about the guidelines.
2. What is the difference between pimplefoam and pisofoam?

Thank you.

PCG is the preconditioning CG method and can just be used to solve the symmetric matrix; smoothSolver is Guass Siedle method namely a very very old and slower method, but it can almost be used to solve any equation, I think. MG is multi-grids method. Many people think PBiCG, PCG and MG are very efficient methods, however, I think, maybe smoothSolver is most safest, and at least it is enough to solve U equation.

My advisor dislikes piso algorithm so that we are all not really familar with it. We all like SIMPLEC algorithm. But many people think Piso is good for unstable flows.

Am I wrong?

[lakeat]

Thanks for your help.

Actually, What I am struggling with is Question 1, ie.
Guidelines for solver choosing

I am using pisoFoam simulating a cylinder flow, and my experience has shown that GAMG use much smaller time to do the same job than PCG.

1. Is it correct to use GAMG for p?
2. What if I use GAMG for U?
3. How about the other turbulence variables?
4. Prof. Jasak's SAMG sounds far better than the others, can I use it in my case?

Okay, let me add more questions here, I think the answers of these questions are very useful for every users, and should be put on top.

I have suffered the problem of parallel computing's speed-up, I remember once somebody talked a speed-up of 11.05 using 8 processors! That's impressive! How did he make it? The cylinder case I am solving is about 860000 grids, I found 32 processors or even 64 processors are much slower than what I thought. So, I want to know more detailedly,
Guidelines for decomposition:

1. How many processors should be used for a given case like cylinder flow?
2. What is the best grid numbers for each processor?
3. Has OpenFOAM being used with more than 64 processors? How is it?

Thank you.

[sandy]

Originally Posted by lakeat

Thanks for your help.

Actually, What I am struggling with is Question 1, ie.
Guidelines for solver choosing

I am using pisoFoam simulating a cylinder flow, and my experience has shown that GAMG use much smaller time to do the same job than PCG.

1. Is it correct to use GAMG for p?
2. What if I use GAMG for U?
3. How about the other turbulence variables?
4. Prof. Jasak's SAMG sounds far better than the others, can I use it in my case?

Thank you.

Hi Daniel, GAMG method can efficiently solve a positive definite, diagonally dominant matrix. However, in fact, as I known, p equation is actually a singular equation so that it is so difficult to be solved by most of current methods. You really think GAMG is efficient to solve it? Maybe PCG or PBiCG is better, I think. I seldom use GAMG. According to some papers, it is also very difficult to complex problems, if you don't want to choose Guass Seidel, I guess maybe you choose PCG or PBiCG more better. However, I don't know whether or not OpenFOAM will actually judge a matrix is really symmetric or asymmetric, before the PCG or PBiCG is implemented. Do you know where I can find the codes to judge a matrix in OpenFOAM?

I am so sorry I never hear of SAMG. Hope it is useful to you.

Sandy

[alberto]

Hi Sandy,

Originally Posted by sandy

Hi Daniel, GAMG method can efficiently solve a positive definite, diagonally dominant matrix. However, in fact, as I known, p equation is actually a singular equation so that it is so difficult to be solved by most of current methods.

you are using the word "singular" in the wrong context. A "singular equation" cannot be solved at all, and in the case of the pressure equation you say it is "singular" if the matrix associated to the discretized equation is singular (its determinant is zero). This it clearly not the case in general, so you cannot say the pressure equation is singular.

However, I don't know whether or not OpenFOAM will actually judge a matrix is really symmetric or asymmetric, before the PCG or PBiCG is implemented. Do you know where I can find the codes to judge a matrix in OpenFOAM?

Yes OpenFOAM checks if a matrix is symmetric or not. Just try to use a symmetric solver for an asymmetric matrix, and you'll notice that!

Best,
A.

[alberto]

Hi lakeat,

Originally Posted by lakeat

Thanks for your help.
I am using pisoFoam simulating a cylinder flow, and my experience has shown that GAMG use much smaller time to do the same job than PCG.

That's exactly the goal of multi-grid methods: increase the convergence rate of the solution by using different grids, starting from a coarse one, mapping to a finer one and proceeding until the desired solution is obtained.

1. Is it correct to use GAMG for p?

Why not? It is a linear solver, and as such it can be used to solve any linear system. The criterion of choise is the performance to obtain the desired accuracy. Using GAMG (and in general AMG solvers) for the solution of the pressure equation makes sense because its convergence might be slow, and using a multi-grid method should make it faster. Keep in mind that it is convenient to use AMG-type solvers when the cost of the coarsening/remapping/refining process is outweighted by the increase in speed.

2. What if I use GAMG for U?

See above.

3. How about the other turbulence variables?

See above. Probably you would not notice much difference? Try and compare ;-)

I have suffered the problem of parallel computing's speed-up, I remember once somebody talked a speed-up of 11.05 using 8 processors! That's impressive! How did he make it? The cylinder case I am solving is about 860000 grids, I found 32 processors or even 64 processors are much slower than what I thought. So, I want to know more detailedly,

How did they define the speedup?

1. How many processors should be used for a given case like cylinder flow?
2. What is the best grid numbers for each processor?

This really depends on your case, on the solver you use, on what conditions you are considering. You can have general guidelines but they would be quite vague in my opinion.
Once you respect the basic rules of trying to decompose your domain minimizing the information exchange (in other words in the case of a symple cylinder try not to cut along the lenght of the pipe but make sections orthogonal to the pipe axis, for example), make some experiment and see what fits your needs.

Best,

[lakeat]

Hi,

Thanks again. Your explanation is very clear and I will do the comparison.
I have chance now to access the fastest super-computer nationwide, I must compare and tailor out the number of processors needed for a given case.

Now apparently, I think simple and hierarchical are just for some academical comparison, it is now metis and scotch that are much superior to them. I have compared and seen a great advantages of scotch over the others, and it's easy for use too. I will use scoth henceforth.

FYI, my cylinder case has 860000 grids, o-type, using scoth, I did not see a better performance in using 32 cpus over 16 cpus over 8 cpus, I will give the pictures as soon as my hands are free. This is why I begin to think about the question:

"Is there any more choice to improve the parallel performance?"

I saw Prof. Jasak are recommending SAMG, In your opinion, will SAMG be better than GAMG and suitable for my case? (pisoFoam) Thank you.

Concerning the speedup, I get the info from thread -- http://www.cfd-online.com/Forums/ope...foam-13-a.html
The paper can be downloaded from:
(http://www.jncasr.ac.in/kirti/current_science.pdf)
That's astonishing, I have never experience that's high.


Thank you.

[alberto]

Originally Posted by sandy

My advisor dislikes piso algorithm so that we are all not really familar with it. We all like SIMPLEC algorithm. But many people think Piso is good for unstable flows.

Well, maybe he should motivate why he dislikes PISO algorithms. It is not a question of taste when you select an algorithm, but a question of what works best for a given case. In other words, you should know PISO algorithms too and choose what fits your needs.

The SIMPLE algorithm in its original form (Caretto et al, 1972, Proceedings of the Third International Conference on Numerical Methods in Fluid Mechanics, Paris) is based on obtaining a pressure correction equation in which the velocity corrections are dropped because unknown (See, for example, Ferziger and Peric book). This is quite crude, and is one of the reasons why the method is not that efficient in terms of convergence rate. The effect of neglecting these terms is absent once the solution is converged however, and the method can be used "as is" for steady problems, where under-relaxation can be used to keep the solution stable.

An unsteady version of the SIMPLE algorithm can be written, by performing two loops: the external loop in time, and an internal loop to obtain a convergence solution at a given time step. In the internal loop, under-relaxation can be used, as long as a sufficient number of iterations is ensured, in order to reach the actual solution at that given time.

SIMPLEC is a version of SIMPLE where the velocity correction is introduced. However its convergence rate is very similar to what obtained with SIMPLE, using the correct under-relaxation: URF_p = 1 - URF_u (Again, see Ferziger and Peric book).

A more refined approach is the PISO one where you

• ignore the velocity corrections in the pressure equation as in the SIMPLE algorithm at the first step
• apply the velocity correction and then perform a number of other corrector steps, originating an interative procedure to solve the pressure equation treating the velocity correction term in it in an explicit manner.

as suggested by Issa (1986, Journal of Computational Physics). The PISO algorithm is considered particularly effective for unsteady flows.

Best,

[lakeat]

Originally Posted by alberto

Well, maybe he should motivate why he dislikes PISO algorithms. It is not a question of taste when you selecting an algorithm, but a question of what works best for a given case. In other words, you should know PISO algorithms too and choose what fits your needs.

A more refined approach is the PISO one where you

• ignore the velocity corrections in the pressure equation as in the SIMPLE algorithm at the first step
• apply the velocity correction and then perform a number of other corrector steps, originating an interative procedure to solve the pressure equation treating the velocity correction term in it in an explicit manner.

as suggested by Issa (1986, Journal of Computational Physics). The PISO algorithm is considered particularly effective for unsteady flows.

Best,

Hi, Sandy, There are advisors and advisors ...

[alberto]

Originally Posted by lakeat

Hi,

Now apparently, I think simple and hierarchical are just for some academical comparison, it is now metis and scotch that are much superior to them. I have compared and seen a great advantages of scotch over the others, and it's easy for use too. I will use scoth henceforth.

For a cylinder flow? A "simple" manual decomposition should do the job. In my experience metis is OK if you have complicated cases where finding a good decomposition by hand is tricky, but for simple cases like a channel flow or a pipe flow, a hand-made decomposition is far easier and might be more effective. I still have to try scotch. How does it decompose the pipe?

FYI, my cylinder case has 860000 grids, o-type, using scoth, I did not see a better performance in using 32 cpus over 16 cpus over 8 cpus, I will give the pictures as soon as my hands are free. This is why I begin to think about the question:

"Is there any more choice to improve the parallel performance?"

Well, I don't know what processors you have handy. But with 860.000 cells, for a normal single-phase simulation, 8-12 processors are enough.

I saw Prof. Jasak are recommending SAMG, In your opinion, will SAMG be better than GAMG and suitable for my case? (pisoFoam) Thank you.

I do not have any experience with SAMG. If he tested it in a similar case, I would say so.

Concerning the speedup, I get the info from thread -- http://www.cfd-online.com/Forums/ope...foam-13-a.html
The paper can be downloaded from:
(http://www.jncasr.ac.in/kirti/current_science.pdf)
That's astonishing, I have never experience that's high.

Thanks for the links.

Best,

[sandy]

Hi Alberto, I still care of two question:

1. Which code or file to judge a symmetric or asymmetric matrix in OpenFOAM - 1.5?
2. If you think p equation is not always a singular matrix, why we have to spend so much CPU time during the numerical simulation. Which step is the most expensive. I think, except for the grid generation, it is the solution of the linear equation in every iterative step. Am I wrong? The key is what kind of matrix is the p equation???

I am actually weak to analyze the matrix structure.

Have a nice weekend.
Sandy

[alberto]

Originally Posted by sandy

Hi Alberto, I still care of two question:

1. Which code or file to judge a symmetric or asymmetric matrix in OpenFOAM - 1.5?

Solvers are defined (with some exception for CG methods) for both symmetric and asymmetric matrix. Check the matrix implementation and the solver implementation in OpenFOAM. You might start with a search in the Doxygen documentation for "matrix", "symmetric matrix" and also for the solver classes.

2. If you think p equation is not always a singular matrix, why we have to spend so much CPU time during the numerical simulation. Which step is the most expensive. I think, except for the grid generation, it is the solution of the linear equation in every iterative step. Am I wrong? The key is what kind of matrix is the p equation???

I think you are confusing some basic concept of linear algebra. Let me try to clarify, but I suggest you to read a good book about the topic before proceeding with CFD.

Consider a square matrix (matrix with n rows and n columns, of real elements). You say that the matrix is invertible (or not singualar) if a matrix exists so that
,
where I is the identity matrix, and is the inverse matrix.

If such a matrix does not exist, the matrix is said to be singular, or not invertible.

You can easily understand if a matrix is invertible by computing its determinant, because a square matrix is singular if and only if its determinant is zero.

Now, let's consider a linear system in the form

where x is the unknown vector, b the known vector and A the matrix. Its solution is clearly given formally by

and exists only if is invertible, or, in other words, not singular.

If you have a linear system whose matrix has null determinant, you can still have solutions, but you need to evaluate the rank of the matrix, and compare it to the rank of the complete matrix (rectangular), obtained by adding to A the column b.
According to Rouché-Capelli theorem, you know that

• Ax = b has solutions only if the rank of A (incomplete matrix) is equal to the rank of the complete matrix (A|b) obtained as said above.
• The solution is unique only if rank(A) = rank(A|b) = n, where n is the dimension of the matrix
• If rank(A) = rank(A|b) < n, the system has infinite solutions, with a moltiplicity equal to n - rank(A).

To go back to the pressure equation, keeping in mind what said above, you cannot say the matrix originating from its discretization is singular, since it is actually admit inverse (you indirectly determine it by solving the linear system itself).

The matrix of the linear system originating from the discretization of the pressure Poisson equation in standard form is tridiagonal or, more generally, block tridiagonal. There are specific algorithms for the solution of this kind of matrix (Thomas algorithm and its block version for example), which are very effective due to their low computational cost (Thomas algorithm costs O(n), while Gauss method costs O(n^3), for example).

I hope this helps.

Best,

[sandy]

Originally Posted by alberto

According to Rouché-Capelli theorem, you know that

• Ax = b has solutions only if the rank of A (incomplete matrix) is equal to the rank of the complete matrix (A|b) obtained as said above.
• The solution is unique only if rank(A) = rank(A|b) = n, where n is the dimension of the matrix
• If rank(A) = rank(A|b) < n, the system has infinite solutions, with a moltiplicity equal to n - rank(A).

Hi Alberto, if rank(A) = rank(A|b) < n, what is its determinant? And whether or not it is invertible and has solutions? This kind of matrix is not singular matrix, right? Maybe the pressure equation is this kind of matrix? Thanks.

[alberto]

Originally Posted by sandy

Hi Alberto, if rank(A) = rank(A|b) < n, what is its determinant? And whether or not it is invertible and has solutions? This kind of matrix is not singular matrix, right? Maybe the pressure equation is this kind of matrix? Thanks.

Hi Sandy,

if the rank of A is less than n, the determinant of A is zero, and A is singular. Let's consider an example I took from my linear algebra book (Greco, Valabrega, Lezioni di Algebra e Geometria, Levrotto & Bella, 1992).

Let's consider A=

1 -2 1 -1
1 -2 0 -2
-2 4 -1 3
-2 4 -2 2

and a known vector b = {2, -3, 1, -4}, so that the complete matrix is (A|b) =

1 -2 1 -1 | 2
1 -2 0 -2 | -3
-2 4 -1 3 | 1
-2 4 -2 2 | -4

If you careful look at the matrix A, you notice that the first row and the last row are linearly dependent, because you can obtain the last row by multiplying the first one by -2. This leads to a null determinant of the matrix, for a known property of matrix determinants (you can calculate it to check). This means that the rank of the matrix is not 4.

Now, let's remind one practical definition of matrix rank: A matrix has rank r if the number of linearly independent rows (or column) is r, and r+1 rows (or columns) are always linearly dependent.

According to this definition, the rank of matrix A is 2, since also the second row can be obtained as a linear combination of the first and the third rows, so the rank cannot be 3.

Similarly, you can consider the complete matrix, and show its rank is 2.

According to Rouche-Capelli theorem you have that the system admits solutions, which are infinite, with multiplicity equal to 4 - 2 = 2. In practice, you find these solutions by taking two linearly independent equations from the system and use them to define the solution, which will have two variables free to vary.

Notice that you can show that saying a matrix has zero determinant is equivalent to say it has two (or more) linearly dependent rows (or columns).

Back to the pressure equation, I guess at this point it is clear that the matrix of the linear system obtained from the discretization of the Poisson equation has nothingto do with a singular matrix.

Usually the difficulty in the numerical solution (analytically it's not a problem clearly) of a linear system is related to the condition number of its matrix (the highest this number is, the hardest it is to numerically solve the system accurately because of the limited machine precision).

Now, please refer to a good linear algebra book. You need to know these elements of linear algebra if you want to properly understand how CFD codes and numerical algorithms work. It is really in your interest, and if I got it right you are a student, so take advantage of that and attend algebra classes

I hope this helps.

Best,

[sandy]

Back to the pressure equation, I guess at this point it is clear that the matrix of the linear system obtained from the discretization of the Poisson equation has nothingto do with a singular matrix.

Hi Alberto, I am a student. I am appreciating so much for your kind help. Could you further explain how to derive the pressure equation is singular matrix, when giving normal velocity BC and the first class boundary condition namely p = constant? Why the pressure equation has solutions in Nuemann boundary condition?

Thanks again.
Sandy

[sandy]

In addition, I actually need to know how to analyze the matrix structure in OpenFOAM. In my case, an "implicitly" solved source term was added into this Poisson equation. I want to know whether the new pressure equation still keep symmetric or not??? NO! right? Is it singular either?? What boundary conditions can make it well convergence. It is the first time I meet with this kind of pressure equation. Please help me out. Thanks a lot.

[alberto]

Originally Posted by sandy

Hi Alberto, I am a student. I am appreciating so much for your kind help. Could you further explain how to derive the pressure equation is singular matrix, when giving normal velocity BC and the first class boundary condition namely p = constant? Why the pressure equation has solutions in Nuemann boundary condition?

Thanks again.
Sandy

Hi Sandy,

why don't you derive the matrix yourself in a simple case like a rectangle, with these conditions, assuming the flow incompressible to make it simpler:

top: fixed pressure, zero velocity gradient
bottom: fixed velocity, zero pressure gradient
sides: zero velocity, zero pressure gradient

which is the typical case in a channel? Why do you think that matrix is singular?

Note that in the case you use Neumann BC's on all boundaries for the pressure, you obtain an ill posed problem since you never specify a value explicitly for the pressure, and you generally need some addtional information to solve it.

Best,

[alberto]

Originally Posted by sandy

In addition, I actually need to know how to analyze the matrix structure in OpenFOAM. In my case, an "implicitly" solved source term was added into this Poisson equation. I want to know whether the new pressure equation still keep symmetric or not??? NO! right? Is it singular either?? What boundary conditions can make it well convergence. It is the first time I meet with this kind of pressure equation. Please help me out. Thanks a lot.

I would suggest to derive the discretized equation for your case and check yourself, so you will learn how it works. You can use a good linear algebra book and some basic CFD books as a starting point to build your base:

• Ferziger and Peric, Computational methods for fluid dynamics, Springer.
• Tannehill, Anderson, Pletcher, Computational fluid mechanics and heat transfer, Taylor & Francis

Best,

[sandy]

Note that in the case you use Neumann BC's on all boundaries for the pressure, you obtain an ill posed problem since you never specify a value explicitly for the pressure, and you generally need some addtional information to solve it.

Hi Alberto, if I give the pdRefcell and pdRefvalue, is it still an ill posed problem?? I feel grey. Now I am actually in the Nuemann BC's CFD, namely zero Gradient in the academic informations. Thank you very much.