OpenFOAM version 1.6 details
Dear all,
Have a nice day! I have some questions,
Thank you. |
Hi, Daniel, nice to meet you herein.
About question 2, Navier-Stokes equations solving methods are different, they are PISO algorithm and PIMPLE (merged PISO-SIMPLE) algorithm respectively. Maybe the following references can help you to understand more details: 1. J.H. Ferziger and M. Peric Computational Methods for Fluid Dynamics 3rd ed. Springer 2002. 2. Hrvoje Jasak, PhD 1996, PDF of thesis posted at: Error analysis and estimation in the Finite Volume method with applications to fluid flows. 3. Henrik Rusche, PhD 2002 Computational fluid dynamics of dispersed two-phase flows at high phase fractions. about question 1, I also am not so clear, but for the p, I always use smoothsolver to get a faster convergence. Best regards, Chiven |
Quote:
My advisor dislikes piso algorithm so that we are all not really familar with it. We all like SIMPLEC algorithm. But many people think Piso is good for unstable flows. Am I wrong? |
Thanks for your help.
Actually, What I am struggling with is Question 1, ie. Guidelines for solver choosing I am using pisoFoam simulating a cylinder flow, and my experience has shown that GAMG use much smaller time to do the same job than PCG.
Okay, let me add more questions here, I think the answers of these questions are very useful for every users, and should be put on top. I have suffered the problem of parallel computing's speed-up, I remember once somebody talked a speed-up of 11.05 using 8 processors! That's impressive! How did he make it? The cylinder case I am solving is about 860000 grids, I found 32 processors or even 64 processors are much slower than what I thought. So, I want to know more detailedly, Guidelines for decomposition:
Thank you. |
Quote:
I am so sorry I never hear of SAMG. Hope it is useful to you. Sandy |
Hi Sandy,
Quote:
Quote:
Best, A. |
Hi lakeat,
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Once you respect the basic rules of trying to decompose your domain minimizing the information exchange (in other words in the case of a symple cylinder try not to cut along the lenght of the pipe but make sections orthogonal to the pipe axis, for example), make some experiment and see what fits your needs. Best, |
Hi,
Thanks again. Your explanation is very clear and I will do the comparison. I have chance now to access the fastest super-computer nationwide, I must compare and tailor out the number of processors needed for a given case. Now apparently, I think simple and hierarchical are just for some academical comparison, it is now metis and scotch that are much superior to them. I have compared and seen a great advantages of scotch over the others, and it's easy for use too. I will use scoth henceforth. FYI, my cylinder case has 860000 grids, o-type, using scoth, I did not see a better performance in using 32 cpus over 16 cpus over 8 cpus, I will give the pictures as soon as my hands are free. This is why I begin to think about the question: "Is there any more choice to improve the parallel performance?" I saw Prof. Jasak are recommending SAMG, In your opinion, will SAMG be better than GAMG and suitable for my case? (pisoFoam) Thank you. Concerning the speedup, I get the info from thread -- http://www.cfd-online.com/Forums/ope...foam-13-a.html The paper can be downloaded from: (http://www.jncasr.ac.in/kirti/current_science.pdf) That's astonishing, I have never experience that's high. Thank you. :) |
Quote:
The SIMPLE algorithm in its original form (Caretto et al, 1972, Proceedings of the Third International Conference on Numerical Methods in Fluid Mechanics, Paris) is based on obtaining a pressure correction equation in which the velocity corrections are dropped because unknown (See, for example, Ferziger and Peric book). This is quite crude, and is one of the reasons why the method is not that efficient in terms of convergence rate. The effect of neglecting these terms is absent once the solution is converged however, and the method can be used "as is" for steady problems, where under-relaxation can be used to keep the solution stable. An unsteady version of the SIMPLE algorithm can be written, by performing two loops: the external loop in time, and an internal loop to obtain a convergence solution at a given time step. In the internal loop, under-relaxation can be used, as long as a sufficient number of iterations is ensured, in order to reach the actual solution at that given time. SIMPLEC is a version of SIMPLE where the velocity correction is introduced. However its convergence rate is very similar to what obtained with SIMPLE, using the correct under-relaxation: URF_p = 1 - URF_u (Again, see Ferziger and Peric book). A more refined approach is the PISO one where you
Best, |
Quote:
Hi, Sandy, There are advisors and advisors ... :D:D:D |
Quote:
Quote:
Quote:
Quote:
Best, |
Hi Alberto, I still care of two question:
1. Which code or file to judge a symmetric or asymmetric matrix in OpenFOAM - 1.5? 2. If you think p equation is not always a singular matrix, why we have to spend so much CPU time during the numerical simulation. Which step is the most expensive. I think, except for the grid generation, it is the solution of the linear equation in every iterative step. Am I wrong? The key is what kind of matrix is the p equation??? I am actually weak to analyze the matrix structure. Have a nice weekend. Sandy |
Quote:
|
Quote:
if the rank of A is less than n, the determinant of A is zero, and A is singular. Let's consider an example I took from my linear algebra book (Greco, Valabrega, Lezioni di Algebra e Geometria, Levrotto & Bella, 1992). Let's consider A= 1 -2 1 -1 1 -2 0 -2 -2 4 -1 3 -2 4 -2 2 and a known vector b = {2, -3, 1, -4}, so that the complete matrix is (A|b) = 1 -2 1 -1 | 2 1 -2 0 -2 | -3 -2 4 -1 3 | 1 -2 4 -2 2 | -4 If you careful look at the matrix A, you notice that the first row and the last row are linearly dependent, because you can obtain the last row by multiplying the first one by -2. This leads to a null determinant of the matrix, for a known property of matrix determinants (you can calculate it to check). This means that the rank of the matrix is not 4. Now, let's remind one practical definition of matrix rank: A matrix has rank r if the number of linearly independent rows (or column) is r, and r+1 rows (or columns) are always linearly dependent. According to this definition, the rank of matrix A is 2, since also the second row can be obtained as a linear combination of the first and the third rows, so the rank cannot be 3. Similarly, you can consider the complete matrix, and show its rank is 2. According to Rouche-Capelli theorem you have that the system admits solutions, which are infinite, with multiplicity equal to 4 - 2 = 2. In practice, you find these solutions by taking two linearly independent equations from the system and use them to define the solution, which will have two variables free to vary. Notice that you can show that saying a matrix has zero determinant is equivalent to say it has two (or more) linearly dependent rows (or columns). Back to the pressure equation, I guess at this point it is clear that the matrix of the linear system obtained from the discretization of the Poisson equation has nothingto do with a singular matrix. Usually the difficulty in the numerical solution (analytically it's not a problem clearly) of a linear system is related to the condition number of its matrix (the highest this number is, the hardest it is to numerically solve the system accurately because of the limited machine precision). Now, please refer to a good linear algebra book. You need to know these elements of linear algebra if you want to properly understand how CFD codes and numerical algorithms work. It is really in your interest, and if I got it right you are a student, so take advantage of that and attend algebra classes :D I hope this helps. Best, |
Quote:
Thanks again. Sandy |
In addition, I actually need to know how to analyze the matrix structure in OpenFOAM. In my case, an "implicitly" solved source term was added into this Poisson equation. I want to know whether the new pressure equation still keep symmetric or not??? NO! right? Is it singular either?? What boundary conditions can make it well convergence. It is the first time I meet with this kind of pressure equation. Please help me out. Thanks a lot.
|
Quote:
why don't you derive the matrix yourself in a simple case like a rectangle, with these conditions, assuming the flow incompressible to make it simpler: top: fixed pressure, zero velocity gradient bottom: fixed velocity, zero pressure gradient sides: zero velocity, zero pressure gradient which is the typical case in a channel? Why do you think that matrix is singular? Note that in the case you use Neumann BC's on all boundaries for the pressure, you obtain an ill posed problem since you never specify a value explicitly for the pressure, and you generally need some addtional information to solve it. Best, |
Quote:
|
Quote:
|
Quote:
Best, |
Quote:
|
Quote:
I was very very lucky during my studies and now in my post-doc to have good advisors though! :cool: |
Quote:
|
Hi Sandy,
I'm not sure how Lakeat is trying to simulate the flow, but if the Reynolds number is high enough (and it does not need to be very high), and he is doing LES, that number of cells does not seem huge to me. Keep in mind that in LES you must resolve scales until well inside the inertial subrange of the spectrum. Best, |
WOW~, in 2D, it still needs so huge grids? If that, I should choose a good rest but work so hard.
|
Quote:
I have never do an extensive survey, but I am not sure how good it will be to use RANS for a cylinder flow?????????? Will RANS give the Cd and Cl correct enough???? But Apparently, I will not expect good results from 2D simulation. The energy cascade is completely wrong in 2D simulation. The Re (3900) is not high though, but already costs me a lot. This is LES. I am expecting a much larger mesh for my next case Re=140000, the grid number would be 7 million. You know what, someone told me that the "Bird Nest" in Beijing, certain man use just 80000 grids to do the simulation, and I was shocked, "how could he manage to do that?" "How could he get the content of TURBULENCE?" :confused::eek: |
Quote:
Quote:
Quote:
Best, |
2 Attachment(s)
First, I want to say thank you, Alberto, and also thank you Sandy.
Okay, let me make myself clearer, the cylinder case mesh is an O-type mesh, with Nx*Ny*Nz=165*165*32, Re=3900, with benchmark of LES simulation already published that can be easily followed. To achieve Re=3900, I simply set inlet velocity 1m/s, and cylinder diameter 1m. Flow region is a circle with diameter of 15m, which means standing at the cylinder center, the distance to the inlet and out let are both 7.5m. The timeStep is set to 0.0025 to keep Courant number no larger than unity. Of course, this is full LES, a wall resolved LES, completely 3D simulation for a 2D geometry, as we can imagine from a wind tunnel section model. Periodic B.C. are applied in the front and back, I do not use convectiveOutlet for the moment. Then I need to consider: 1. To achieve the highest efficiency, how many CPUs do I need for this case? 2. To achieve the highest efficiency and accuracy, which solver should I use for p and for U? The 1st one, your experience is 8~16 processors are enough, right? The 2nd one, you recommend GAMG for p and smoothsolver for U, right? Here are some findings, some of which make me headache now, No. 1 My teacher strongly doubt my simulation result, he said the simulation time is too long, and is unacceptable. Code:
Time = 397.925 No. 2 Based on your experience, can you give a judge, are these figures correct? (I use pyFoam to get these figures) I mean time spent. Attachment 877Attachment 878 Note also the question in the picture. Will you meet the problem of unstable simulation time (ie. ExecutionTime) since the flow is unsteady? Waiting for you, |
Quote:
In addition, what is your initial condition? If you use a uniform, not perturbed, condition it is going to take a long time to actually develop turbulence structures. Eugene de Villiers published a code to initialize a perturbed flow in a cylinder: search the forum for it. Moreover, the lenght of the system has to be enough not to feel the effect of periodic conditions (I would say this was checked in the original work). Quote:
Quote:
Quote:
In other words, you initialize a perturbed flow as said above, you run until the flow is completely developed (and this takes time, a lot!), and then you reset the averages and start averaging for a sufficient number of characteristic times to get your statistics. What does your teacher takes as reference to say the computational time is unacceptable? P.S. What are you using as pressure residual? Best, A. |
I never think that GAMG should be chosen in parallel because it is so complex. U equation does not spend your CPU time too much, whatever which solver is chose. The key is P equation forever.
|
Quote:
|
Hi guys,
I feel there is some miscommunication between you. From what I understood of Lakeat's problem (correct me if I'm wrong), he is simulating the external flow around a cylinder, not an internal pipe flow. When he says he uses periodic conditions at front and back I think he means at both ends of the cylinder. Concerning Lakeat's concerns, 1) Many factors can affect your parallel performance. First, it depends on the computer you're using. A poor connection between the computing nodes can really slow down the calculation, causing a big difference in ExcutionTime and ClockTime. Second, if the connection is good, I've found that the OpenFOAM compilation can greatly influence the computational time. Are you using a pre-compiled OpenFOAM ? If you do, I strongly suggest you compile OpenFOAM locally on the computer you're using for your parallel computations, preferably with the local compiler (such as an Intel compiler). Concerning the decomposition, a "simple" decomposition can be used on simple geometries. If you aren't so sure how to decompose it, you should use MeTis. From my personal experience of unsteady simulations of external flows, I've seen good speedup (linear or better) with as less as 10-15k cells per processor. But that depends on the computer and the case. 2) I've used the GAMG solver for the pressure equation in many parallel cases and I think it performs well, generally faster than PCG. I don't agree with sandy. How the does fact that it is more complex make it slower ? Did you compare it with other solvers in parallel cases ? Hope this helps |
Quote:
About the domain decomposition, in the case of a flow around a cylinder, I agree with Simon when he suggests to use METIS, since it becomes more complicated to do by hand. I'm a bit skeptical about 10.000 cells/processor. In my experience a good trade-off is something between 40.000 and 80.000 cells per processor (meaning core), especially if you are not in a hurry to get the results and you share resources with others ;-) Quote:
Best, |
Quote:
Quote:
You can see from my last post: Quote:
And 1.516% is not that big, right? Quote:
Will different mpi implementations like openmpi, mpich, mvapich, make great differences on the speed? Say, if I am decomposing the case to several nodes, and if I am using just one node. And I remember wiki writes: Quote:
Quote:
Quote:
simple one: Quote:
Quote:
Quote:
And for a first try, my cylinder case mesh, 860000/15000~57 processors??!!!! Is this what you meant?? Quote:
PS: pls call me Daniel... Thank you all. |
Hi,
if you do not use a hardware-optimized implementation of the parallel libraries (HP, Intel, ...), you won't notice much difference switching between OpenMPI (which comes from LAM) and MPICH generic libraries. Performance mainly depends on how you compile them and your code. Quote:
Quote:
Best, |
Quote:
Concerning the mpi implementations I'm not so sure, since with OpenFOAM I've only used OpenMPI. The best way to answer your question about the number of nodes is to try different combinations of nodes and processors per nodes. The best ratio will depend on the connection speed between the nodes, the connection speed between the processors and the usage of the processors on the different nodes you're using. Quote:
Which compiler did you use ? Are there different compilers available ? You could try using a different compiler. Quote:
Quote:
|
Quote:
In addtion, maybe I will try GAMG for p equation. In fact, GAMG is solved by ICCG or Block-ICCG (Why not is BiCG?!!! In coarse grid, all matrices are symmetric??!!! Who can explain it??) in coarse grids, and maybe solved by Guass Seidel in the fine grid? So, it should not be difficult to uderstand this method maybe is faster than PCG. WOW~, really?? GAMG just can be used in solving symmetrical matrix namely p equation?? I just find it!:) If that, what kinds of Mutil-Grids methods can be used to solve the asymmetrical matrix in OpenFOAM? @Sorry Daniel, I actually know nothing to parallel. :( |
Hi Daniel,
another way to accelerate flow development is, to map fluctuations of a 'boxTurb' generated velocity field onto your initial RANS velocities. Fabian |
Quote:
Best, |
All times are GMT -4. The time now is 08:15. |