http://www.cfd-online.com/W/index.php?title=Special:Contributions/Ayyoubzadeh&feed=atom&limit=50&target=Ayyoubzadeh&year=&month=CFD-Wiki - User contributions [en]2016-05-28T10:30:00ZFrom CFD-WikiMediaWiki 1.16.5http://www.cfd-online.com/Wiki/Introduction_to_turbulence/Statistical_analysis/Estimation_from_a_finite_number_of_realizationsIntroduction to turbulence/Statistical analysis/Estimation from a finite number of realizations2007-08-31T16:42:05Z<p>Ayyoubzadeh: /* Bias and convergence of estimators */</p>
<hr />
<div>{{Introduction to turbulence menu}}<br />
== Estimators for averaged quantities ==<br />
<br />
Since there can never an infinite number of realizations from which ensemble averages (and probability densities) can be computed, it is essential to ask: ''How many realizations are enough?'' The answer to this question must be sought by looking at the statistical properties of estimators based on a finite number of realization. There are two questions which must be answered. The first one is:<br />
<br />
* Is the expected value (or mean value) of the estimator equal to the true ensemble mean? Or in other words, is yje estimator ''unbiased?''<br />
<br />
The second question is<br />
<br />
* Does the difference between the and that of the true mean decrease as the number of realizations increases? Or in other words, does the estimator ''converge'' in a statistical sense (or converge in probability). <font color="orange">Figure 2.9</font> illustrates the problems which can arise.<br />
<br />
== Bias and convergence of estimators ==<br />
<br />
A procedure for answering these questions will be illustrated by considerind a simple '''estimator''' for the mean, the arithmetic mean considered above, <math>X_{N}</math>. For <math>N</math> independent realizations <math>x_{n}, n=1,2,...,N</math> where <math>N</math> is finite, <math>X_{N}</math> is given by:<br />
<br />
:<math>X_{N}=\frac{1}{N}\sum^{N}_{n=1} x_{n}</math> <br />
<br />
<font color="orange" size="3">Figure 2.9 not uploaded yet</font><br />
<br />
Now, as we observed in our simple coin-flipping experiment, since the <math>x_{n}</math> are random, so must be the value of the estimator <math>X_{N}</math>. For the estimator to be ''unbiased'', the mean value of <math>X_{N}</math> must be true ensemble mean, <math>X</math>, i.e.<br />
<br />
:<math>\lim_{N\rightarrow\infty} X_{N} = X</math> <br />
<br />
It is easy to see that since the operations of averaging adding commute,<br />
<br />
:<math> <br />
\begin{matrix}<br />
\left\langle X_{N} \right\rangle & = & \left\langle \frac{1}{N} \sum^{N}_{n=1} x_{n} \right\rangle \\<br />
& = & \frac{1}{N} \sum^{N}_{n=1} \left\langle x_{n} \right\rangle \\<br />
& = & \frac{1}{N} NX = X \\<br />
\end{matrix}<br />
</math> <br />
<br />
(Note that the expected value of each <math>x_{n}</math> is just <math>X</math> since the <math>x_{n}</math> are assumed identically distributed). Thus <math>x_{N}</math> is, in fact, an ''unbiased estimator for the mean''.<br />
<br />
The question of ''convergence'' of the estimator can be addressed by defining the square of '''variability of the estimator''', say <math>\epsilon^{2}_{X_{N}}</math>, to be:<br />
<br />
:<math> <br />
\epsilon^{2}_{X_{N}}\equiv \frac{var \left\{ X_{N} \right\} }{X^{2}} = \frac{\left\langle \left( X_{N}- X \right)^{2} \right\rangle }{X^{2}}<br />
</math> <br />
<br />
Now we want to examine what happens to <math>\epsilon_{X_{N}}</math> as the number of realizations increases. For the estimator to converge it is clear that <math>\epsilon_{x}</math> should decrease as the number of sample increases. Obviously, we need to examine the variance of <math>X_{N}</math> first. It is given by:<br />
<br />
:<math> <br />
\begin{matrix}<br />
var \left\{ X_{N} \right\} & = & \left\langle X_{N} - X^{2} \right\rangle \\<br />
& = & \left\langle \left[ \lim_{N\rightarrow\infty} \frac{1}{N} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle - X^{2}\\<br />
\end{matrix}<br />
</math> <br />
<br />
since <math>\left\langle X_{N} \right\rangle = X</math> from the equation for <math>\langle X_{N} \rangle</math> above. Using the fact that operations of averaging and summation commute, the squared summation can be expanded as follows:<br />
<br />
:<math> <br />
\begin{matrix}<br />
\left\langle \left[ \lim_{N\rightarrow\infty} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle & = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}} \sum^{N}_{n=1} \sum^{N}_{m=1} \left\langle \left( x_{n} - X \right) \left( x_{m} - X \right) \right\rangle \\<br />
& = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}}\sum^{N}_{n=1}\left\langle \left( x_{n} - X \right)^{2} \right\rangle \\<br />
& = & \frac{1}{N} var \left\{ x \right\} \\<br />
\end{matrix}<br />
</math> <br />
<br />
where the next to last step follows from the fact that the <math>x_{n}</math> are assumed to be statistically independent samples (and hence uncorrelated), and the last step from the definition of the variance. It follows immediately by substitution into the equation for <math>\epsilon^{2}_{X_{N}}</math> above that the square of the variability of the estimator, <math>X_{N}</math>, is given by:<br />
<br />
:<math> <br />
\begin{matrix}<br />
\epsilon^{2}_{X_{N}}& =& \frac{1}{N}\frac{var\left\{x\right\}}{X^{2}} \\<br />
& = & \frac{1}{N} \left[ \frac{\sigma_{x}}{X} \right]^{2} \\ <br />
\end{matrix}<br />
</math> <br />
<br />
Thus ''the variability of the estimator depends inversely on the number of independent realizations, <math>N</math>, and linearly on the relative fluctuation level of the random variable itself <math>\sigma_{x}/ X</math>''. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the <math>x</math> itself, the greater the fluctuation ( relative to the mean of <math>x</math>, <math>\left\langle x \right\rangle = X</math> ), then the more independent samples it will take to achieve a specified accuracy.<br />
<br />
'''Example:''' In a given ensemble the relative fluctuation level is 12% (i.e. <math>\sigma_{x}/ X = 0.12</math>). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?<br />
<br />
'''Answer'''Using the equation for <math>\epsilon^{2}_{X_{N}}</math> above, and taking <math>\epsilon_{X_{N}}=0.01</math>, it follows that:<br />
<br />
:<math> <br />
\left(0.01 \right)^{2} = \frac{1}{N}\left(0.12 \right)^{2}<br />
</math> <br />
<br />
or <math>N \geq 144</math>.<br />
<br />
{{Turbulence credit wkgeorge}}<br />
<br />
{{Chapter navigation|Multivariate random variables|Generalization to the estimator of any quantity}}</div>Ayyoubzadehhttp://www.cfd-online.com/Wiki/Introduction_to_turbulence/Statistical_analysis/ProbabilityIntroduction to turbulence/Statistical analysis/Probability2007-08-31T16:30:28Z<p>Ayyoubzadeh: /* Skewness and kurtosis */</p>
<hr />
<div>{{Introduction to turbulence menu}}<br />
== The histogram and probability density function ==<br />
<br />
The frequency of occurrence of a given ''amplitude'' (or value) from a finite number of realizations of a random variable can be displayed by dividing the range of possible values of the random variables into a number of slots (or windows). Since all possible values are covered, each realization fits into only one window. For every realization a count is entered into the appropriate window. When all the realizations have been considered, the number of counts in each window is divided by the total number of realizations. The result is called the '''histogram''' (or ''frequency of occurrence'' diagram). From the definition it follows immediately that the sum of the values of all the windows is exactly one.<br />
<br />
The shape of a histogram depends on the ''statistical distribution of the random variable'', but it also depends on the total number of realizations, ''N'', and the size of the slots, <math> \Delta c </math>. The histogram can be represented symbolically by the function <math> H_{x}(c,\Delta c,N)</math> where <math> c\leq x < c + \Delta c </math>, <math> \Delta c </math> is the slot width, and <math> N </math> is the number of realizations of the random variable. Thus the histogram shows the relative frequency of occurrence of a given value range in a given ensemble. <font color="orange">Figure 2.3</font> illustrates a typical histogram. If the size of the sample is increased so that the number of realizations in each window increases, the diagram will become less erratic and will be more representative of the actual ''probability'' of occurrence of the amplitudes of the signal itself, as long as the window size is sufficiently small.<br />
<br />
<font color="orange" size="3">Figure 2.3 not uploaded yet</font><br />
<br />
If the number of realizations, <math> N </math>, increases without bound as the window size, <math> \Delta c </math> , goes to zero, the histogram divided by the window size goes to a limiting curve called the probability density function, <math> B_{x} \left( c \right) </math>. That is,<br />
<br />
:<math> <br />
B_{x} \left( c \right) \equiv \lim_{{ N \rightarrow \infty}, \Delta c \rightarrow 0} H \left( c , \Delta c , N \right) / \Delta c<br />
</math><br />
<br />
Note that as the window width goes to zero, so does the number of realizations which fall into it, <math> N H </math>. Thus it is only when this number (or relative number) is divided by the slot width that a meaningful limit is achieved.<br />
<br />
The '''probability density function''' (or '''pdf''') has the following properties:<br />
<br />
* Property 1:<br />
<br />
:<math>B_{x} \left( c \right) > 0</math><br />
<br />
always.<br />
<br />
* Property 2:<br />
<br />
:<math>Prob \left\{c < x < c + dc \right\} = B_{x} \left(c \right) dc</math><br />
<br />
where <math> Prob \left\{ \right\}</math> is read "the probability that".<br />
<br />
* Property 3:<br />
<br />
:<math>Prob \left\{ x < c \right\} = \int ^{c}_{- \infty }B_{x} \left(c \right) dc</math><br />
<br />
* Property 4:<br />
<br />
:<math>\int ^{\infty}_{- \infty }B_{x} \left(x \right) dx = 1</math><br />
<br />
The condition imposed by property (1) simply states that negative probabilities are impossible, while property (4) assures that the probability is unity that a realization takes on some value. Property (2) gives the probability of finding the realization in a interval around a certain value, while property (3) provides the probability that the realization is less than a prescribed value. Note the necessity of distinguishing between the running variable, <math> x </math> , and the integration variable, <math> c </math>, in property (2) and (3).<br />
<br />
Since <math> B_{x} \left( c \right) dc </math> gives the probability of the random variable <math> x </math> assuming a value between <math> c </math> and <math> c + dc </math>, any moment of the distribution can be computed by integrating the appropriate power of <math> x </math> over all possible values. Thus the <math> n </math> - th moment is given by:<br />
<br />
:<math> <br />
\left\langle x^{n} \right\rangle = \int^{\infty}_{- \infty} c^{n} B_{x} \left(c \right) dc<br />
</math><br />
<br />
'''Exercise:''' Show (by returning to the definitions) that the value of the moment determined in this manner is exactly equal to the ensemble average defined earlier in the definition of the <math>m</math>-th moment. (Hint: use the definition of an integral as a limiting sum.)<br />
<br />
If the probability density is given, the moments of all orders can be determined. For example, the variance can be determined by:<br />
<br />
:<math> <br />
var \left\{ x \right\} = \left\langle \left( x - X \right)^2 \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^2 B_{x} \left(c \right) dc<br />
</math><br />
<br />
The central moments give information about the shape of the probability density function, and ''vice versa''. <font color="orange">Figure 2.4</font> shows three distributions which have the same mean and standard deviation, but are clearly quite different. Beneath them are shown random functions of time, which might have generated them. Distribution (b) has a higher value of the fourth central moment than does distribution (a). This can be easily seen from the definition<br />
<br />
<font color="orange" size="3">Figure 2.4 not uploaded yet</font><br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{4} \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^4 B_{x} \left(c \right) dc <br />
</math><br />
<br />
since the fourth power emphasizes the fact that distribution (b) has more weight in the tails than does distribution (a). <br />
<br />
It is also easy to see that because of the symmetry of pdf's in (a) and (b) all the odd central moments will be zero. Distributions (c) and (d), on the other hand, have non-zero values for the odd moments, because of their asymmtry. For example, <br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{3} \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^3 B_{x} \left(c \right) dc <br />
</math><br />
<br />
is equal to zero if <math> B_{x} </math> is an even function.<br />
<br />
== The probability distribution ==<br />
<br />
Sometimes it is convienient to work with the '''probability distribution''' instead of with probability density function. The probability distribution is defined as the probability that the random variable has a value less than or equal to a given value. Thus from the equation for property (3), the probability distribution is given by<br />
<br />
:<math> <br />
F_{x} \left( c \right) = Prob \left\{ x < c \right\} = \int^{c}_{-\infty} B_{x} \left( c' \right) d c' <br />
</math> <br />
<br />
Note that we had to introduce the integration variable, <math> c' </math>, since <math> c </math> occured in the limits.<br />
<br />
This equation can be inverted by differentiating by <math> c </math> to obtain<br />
<br />
:<math> <br />
B_{x} \left( c \right) = \frac{dF_{x}}{dc} <br />
</math> <br />
<br />
== Gaussian (or normal) distributions ==<br />
<br />
One of the most important pdf's in turbulence is the Gaussian or Normal distribution defined by <br />
<br />
:<math> <br />
B_{xG} \left( c \right) = \frac{1}{\sqrt{2\pi} \sigma_{x}} e^{-\left( c - X \right)^{2} / 2 \sigma^{2} }<br />
</math> <br />
<br />
where <math>X</math> is the mean and <math> \sigma </math> is the standard derivation. The factor <math> 1 / \sqrt{2\pi} \sigma_{x}</math> insures that the integral of the pdf ocer all values is unity as required. It is easy to prove that this is the case by completing the squares in the integration of the exponential. <br />
<br />
The Gaussian distribution is unusual in that it is completely determined by its first two moments, <math>X</math> and <math> \sigma </math>. This is ''not'' typical of most turbulence distributions. Nonetheless, it is sometimes useful to approximate turbulence as being Gaussian, often because of the absence of simple alternatives.<br />
<br />
It is straightforward to show by integrating by parts that all the even central moments above the second are given by the following recursive relationship,<br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{n} \right\rangle = \left( n - 1 \right) \left( n - 3 \right) ....3.1 \sigma^{n} <br />
</math> <br />
<br />
Thus the fourth central moment is <math> 3 \sigma^{4} </math> the sixth is <math> 15 \sigma^{6} </math>, and so forth.<br />
<br />
'''Exercise:''' Prove this: The probability distribution corresponding to the Gaussian distribution can be obtained by integrating the Gaussian pdf from <math>- \infty</math> to <math>x = c</math>; i.e.,<br />
<br />
:<math><br />
F_{xG} \left( c \right) =<br />
\frac{1}{\sqrt{2\pi} \sigma_{x}}<br />
\int^{c}_{- \infty}<br />
e^{(c' - X)^2 / 2 \sigma^2} dc'<br />
</math><br />
<br />
The integral is related to the erf-function tabulated in many standard tables.<br />
<br />
== Skewness and kurtosis ==<br />
<br />
Because of their importance in characterizing the shape of the pdf, it is useful to definescaled versions of third and fourth central moments, the ''skewness'' and ''kurtosis'' respectively. The ''skewness'' is defined as third central moment divided by three*halves of the second; i.e.<br />
<br />
:<math> <br />
S = \frac{\left\langle \left( x- X \right)^{3} \right\rangle }{ \left\langle \left( x- X \right)^{2} \right\rangle^{3/2} }<br />
</math> <br />
<br />
The ''kurtosis'' defined as the fourth central moment divided by the square of the second; i.e. <br />
<br />
:<math> <br />
K = \frac{\left\langle \left( x- X \right)^{4} \right\rangle }{ \left\langle \left( x- X \right)^{2} \right\rangle^{2} } <br />
</math> <br />
<br />
Both these are easy to remember if you note the <math>S</math> and <math>K</math> must be dimensionless.<br />
<br />
The pdf's in <font color="orange">Figure 2.4</font> can be distinguished by means of their skewness and kurtosis. The random variable shown in (b) has a higher kurtosis than that in (a). Thus the kurtosis can be used as an indication of the tails of a pdf, a higher kurtosis indicating that relatively larger excursions from the mean are more probable. The skewness of (a) and (b) are zero, whereas those for (c) and (d) are non-zero. Thus, as its name implies, a non-zero skewness indicates a skewed or asymmetric pdf, which in turn means that larger excursions in one direction are more probable than in the other. For a Gaussian pdf, the skewness is zero and then kurtosis is equal to three. The flatness factor, defined as <math>( K-3 )</math>, is sometimes used to indicate deviations from Gaussian behavior.<br />
<br />
'''Exercise:''' Prove that the kurtosis of a Gaussian distributed random variable is 3.<br />
<br />
{| class="toccolours" style="margin: 2em auto; clear: both; text-align:center;"<br />
|-<br />
| [[Statistical analysis in turbulence|Up to statistical analysis]] | [[Ensemble average in turbulence|Back to ensemble average]] | [[Multivariate random variables|Forward to multivariate random variables]]<br />
|}<br />
<br />
{{Turbulence credit wkgeorge}}<br />
<br />
{{Chapter navigation|Ensemble average|Multivariate random variables}}</div>Ayyoubzadehhttp://www.cfd-online.com/Wiki/Introduction_to_turbulence/Statistical_analysis/ProbabilityIntroduction to turbulence/Statistical analysis/Probability2007-08-31T16:25:48Z<p>Ayyoubzadeh: /* Gaussian (or normal) distributions */</p>
<hr />
<div>{{Introduction to turbulence menu}}<br />
== The histogram and probability density function ==<br />
<br />
The frequency of occurrence of a given ''amplitude'' (or value) from a finite number of realizations of a random variable can be displayed by dividing the range of possible values of the random variables into a number of slots (or windows). Since all possible values are covered, each realization fits into only one window. For every realization a count is entered into the appropriate window. When all the realizations have been considered, the number of counts in each window is divided by the total number of realizations. The result is called the '''histogram''' (or ''frequency of occurrence'' diagram). From the definition it follows immediately that the sum of the values of all the windows is exactly one.<br />
<br />
The shape of a histogram depends on the ''statistical distribution of the random variable'', but it also depends on the total number of realizations, ''N'', and the size of the slots, <math> \Delta c </math>. The histogram can be represented symbolically by the function <math> H_{x}(c,\Delta c,N)</math> where <math> c\leq x < c + \Delta c </math>, <math> \Delta c </math> is the slot width, and <math> N </math> is the number of realizations of the random variable. Thus the histogram shows the relative frequency of occurrence of a given value range in a given ensemble. <font color="orange">Figure 2.3</font> illustrates a typical histogram. If the size of the sample is increased so that the number of realizations in each window increases, the diagram will become less erratic and will be more representative of the actual ''probability'' of occurrence of the amplitudes of the signal itself, as long as the window size is sufficiently small.<br />
<br />
<font color="orange" size="3">Figure 2.3 not uploaded yet</font><br />
<br />
If the number of realizations, <math> N </math>, increases without bound as the window size, <math> \Delta c </math> , goes to zero, the histogram divided by the window size goes to a limiting curve called the probability density function, <math> B_{x} \left( c \right) </math>. That is,<br />
<br />
:<math> <br />
B_{x} \left( c \right) \equiv \lim_{{ N \rightarrow \infty}, \Delta c \rightarrow 0} H \left( c , \Delta c , N \right) / \Delta c<br />
</math><br />
<br />
Note that as the window width goes to zero, so does the number of realizations which fall into it, <math> N H </math>. Thus it is only when this number (or relative number) is divided by the slot width that a meaningful limit is achieved.<br />
<br />
The '''probability density function''' (or '''pdf''') has the following properties:<br />
<br />
* Property 1:<br />
<br />
:<math>B_{x} \left( c \right) > 0</math><br />
<br />
always.<br />
<br />
* Property 2:<br />
<br />
:<math>Prob \left\{c < x < c + dc \right\} = B_{x} \left(c \right) dc</math><br />
<br />
where <math> Prob \left\{ \right\}</math> is read "the probability that".<br />
<br />
* Property 3:<br />
<br />
:<math>Prob \left\{ x < c \right\} = \int ^{c}_{- \infty }B_{x} \left(c \right) dc</math><br />
<br />
* Property 4:<br />
<br />
:<math>\int ^{\infty}_{- \infty }B_{x} \left(x \right) dx = 1</math><br />
<br />
The condition imposed by property (1) simply states that negative probabilities are impossible, while property (4) assures that the probability is unity that a realization takes on some value. Property (2) gives the probability of finding the realization in a interval around a certain value, while property (3) provides the probability that the realization is less than a prescribed value. Note the necessity of distinguishing between the running variable, <math> x </math> , and the integration variable, <math> c </math>, in property (2) and (3).<br />
<br />
Since <math> B_{x} \left( c \right) dc </math> gives the probability of the random variable <math> x </math> assuming a value between <math> c </math> and <math> c + dc </math>, any moment of the distribution can be computed by integrating the appropriate power of <math> x </math> over all possible values. Thus the <math> n </math> - th moment is given by:<br />
<br />
:<math> <br />
\left\langle x^{n} \right\rangle = \int^{\infty}_{- \infty} c^{n} B_{x} \left(c \right) dc<br />
</math><br />
<br />
'''Exercise:''' Show (by returning to the definitions) that the value of the moment determined in this manner is exactly equal to the ensemble average defined earlier in the definition of the <math>m</math>-th moment. (Hint: use the definition of an integral as a limiting sum.)<br />
<br />
If the probability density is given, the moments of all orders can be determined. For example, the variance can be determined by:<br />
<br />
:<math> <br />
var \left\{ x \right\} = \left\langle \left( x - X \right)^2 \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^2 B_{x} \left(c \right) dc<br />
</math><br />
<br />
The central moments give information about the shape of the probability density function, and ''vice versa''. <font color="orange">Figure 2.4</font> shows three distributions which have the same mean and standard deviation, but are clearly quite different. Beneath them are shown random functions of time, which might have generated them. Distribution (b) has a higher value of the fourth central moment than does distribution (a). This can be easily seen from the definition<br />
<br />
<font color="orange" size="3">Figure 2.4 not uploaded yet</font><br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{4} \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^4 B_{x} \left(c \right) dc <br />
</math><br />
<br />
since the fourth power emphasizes the fact that distribution (b) has more weight in the tails than does distribution (a). <br />
<br />
It is also easy to see that because of the symmetry of pdf's in (a) and (b) all the odd central moments will be zero. Distributions (c) and (d), on the other hand, have non-zero values for the odd moments, because of their asymmtry. For example, <br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{3} \right\rangle = \int^{\infty}_{- \infty} \left(c - X \right)^3 B_{x} \left(c \right) dc <br />
</math><br />
<br />
is equal to zero if <math> B_{x} </math> is an even function.<br />
<br />
== The probability distribution ==<br />
<br />
Sometimes it is convienient to work with the '''probability distribution''' instead of with probability density function. The probability distribution is defined as the probability that the random variable has a value less than or equal to a given value. Thus from the equation for property (3), the probability distribution is given by<br />
<br />
:<math> <br />
F_{x} \left( c \right) = Prob \left\{ x < c \right\} = \int^{c}_{-\infty} B_{x} \left( c' \right) d c' <br />
</math> <br />
<br />
Note that we had to introduce the integration variable, <math> c' </math>, since <math> c </math> occured in the limits.<br />
<br />
This equation can be inverted by differentiating by <math> c </math> to obtain<br />
<br />
:<math> <br />
B_{x} \left( c \right) = \frac{dF_{x}}{dc} <br />
</math> <br />
<br />
== Gaussian (or normal) distributions ==<br />
<br />
One of the most important pdf's in turbulence is the Gaussian or Normal distribution defined by <br />
<br />
:<math> <br />
B_{xG} \left( c \right) = \frac{1}{\sqrt{2\pi} \sigma_{x}} e^{-\left( c - X \right)^{2} / 2 \sigma^{2} }<br />
</math> <br />
<br />
where <math>X</math> is the mean and <math> \sigma </math> is the standard derivation. The factor <math> 1 / \sqrt{2\pi} \sigma_{x}</math> insures that the integral of the pdf ocer all values is unity as required. It is easy to prove that this is the case by completing the squares in the integration of the exponential. <br />
<br />
The Gaussian distribution is unusual in that it is completely determined by its first two moments, <math>X</math> and <math> \sigma </math>. This is ''not'' typical of most turbulence distributions. Nonetheless, it is sometimes useful to approximate turbulence as being Gaussian, often because of the absence of simple alternatives.<br />
<br />
It is straightforward to show by integrating by parts that all the even central moments above the second are given by the following recursive relationship,<br />
<br />
:<math> <br />
\left\langle \left( x - X \right)^{n} \right\rangle = \left( n - 1 \right) \left( n - 3 \right) ....3.1 \sigma^{n} <br />
</math> <br />
<br />
Thus the fourth central moment is <math> 3 \sigma^{4} </math> the sixth is <math> 15 \sigma^{6} </math>, and so forth.<br />
<br />
'''Exercise:''' Prove this: The probability distribution corresponding to the Gaussian distribution can be obtained by integrating the Gaussian pdf from <math>- \infty</math> to <math>x = c</math>; i.e.,<br />
<br />
:<math><br />
F_{xG} \left( c \right) =<br />
\frac{1}{\sqrt{2\pi} \sigma_{x}}<br />
\int^{c}_{- \infty}<br />
e^{(c' - X)^2 / 2 \sigma^2} dc'<br />
</math><br />
<br />
The integral is related to the erf-function tabulated in many standard tables.<br />
<br />
== Skewness and kurtosis ==<br />
<br />
Because of their importance in characterizing the shape of the pdf, it is useful to definescaled versions of third and fourth central moments, the ''skewness'' and ''kurtosis'' respectively. The ''skewness'' is defined as third central moment divided by three*halves of the second; i.e.<br />
<br />
:<math> <br />
S = \frac{\left\langle \left( x- X \right)^{3} \right\rangle }{ \left\langle \left( x- X \right)^{2} \right\rangle^{3/2} }<br />
</math> <br />
<br />
The ''kurtosis'' defined as the fourth central moment divided by the square of the second; i.e. <br />
<br />
:<math> <br />
K = \frac{\left\langle \left( x- X \right)^{4} \right\rangle }{ \left\langle \left( x- X \right)^{2} \right\rangle^{2} } <br />
</math> <br />
<br />
Both these are easy to remember if you note the <math>S</math> and <math>K</math> must be dimensionless.<br />
<br />
The pdf's in <font color="orange">Figure 2.4</font> can be distinguished by means of their skewness and kurtosis. The random variable shown in (b) has a higher kurtosis than that in (a). Thus the kurtosis can be used as an indication of the tails of a pdf, a higher kurtosis indicating that relatively larger excursions from the mean are more probable. The skewness of (a) and (b) are zero, whereas those for (c) and (d) are non-zero. Thus, as its name implies, a non-zero skewness indicates a skewed or asymmetric pdf, which in turn means that larger excursions in one direction are more probable tan in the other. For a Gaussian pdf, the skewness is zero and then kurtosis is equal to three. The flatness factor, defined as <math>( K-3 )</math>, is sometimes used to indicate deviations from Gaussian behavior.<br />
<br />
'''Exercise:''' Prove that the kurtosis of a Gaussian distributed random variable is 3.<br />
<br />
{| class="toccolours" style="margin: 2em auto; clear: both; text-align:center;"<br />
|-<br />
| [[Statistical analysis in turbulence|Up to statistical analysis]] | [[Ensemble average in turbulence|Back to ensemble average]] | [[Multivariate random variables|Forward to multivariate random variables]]<br />
|}<br />
<br />
{{Turbulence credit wkgeorge}}<br />
<br />
{{Chapter navigation|Ensemble average|Multivariate random variables}}</div>Ayyoubzadehhttp://www.cfd-online.com/Wiki/Introduction_to_turbulence/Statistical_analysis/Ensemble_averageIntroduction to turbulence/Statistical analysis/Ensemble average2007-08-31T16:11:36Z<p>Ayyoubzadeh: /* Higher moments */</p>
<hr />
<div>{{Introduction to turbulence menu}}<br />
== The mean or ensemble average ==<br />
<br />
The concept of an ''ensemble average'' is based upon the existence of independent statistical event. For example, consider a number of inviduals who are simultaneously flipping unbiased coins. If a value of one is assigned to a head and the value of zero to a tail, then the ''arithmetic average'' of the numbers generated is defined as<br />
<br />
:<math>X_{N}=\frac{1}{N} \sum{x_{n}}</math><br />
<br />
where our <math> n </math> th flip is denoted as <math> x_{n} </math> and <math> N </math> is the total number of flips.<br />
<br />
Now if all the coins are the same, it doesn't really matter whether we flip one coin <math> N </math> times, or <math> N </math> coins a single time. The key is that they must all be ''independent events'' - meaning the probability of achieving a head or tail in a given flip must be completely independent of what happens in all the other flips. Obviously we can't just flip one coin and count it <math> N </math> times; these cleary would not be independent events<br />
<br />
'''Exercise:''' Carry out an experiment where you flip a coin 100 times in groups of 10 flips each. Compare the values you get for <math>X_{10}</math> for each of the 10 groups, and note how they differ from the value of <math>X_{100}</math>.<br />
<br />
Unless you had a very unusual experimental result, you probably noticed that the value of the <math> X_{10} </math>'s was also a random variable and differed from ensemble to ensemble. Also the greater the number of flips in the ensemle, the closer you got to <math>X_{N}=1/2 </math>. Obviously the bigger <math> N </math> , the less fluctuation there is in <math> X_{N} </math> <br />
<br />
Now imagine that we are trying to establish the nature of a random variable <math> x </math>. The <math>n</math>th ''realization'' of <math> x </math> is denoted as <math> x_{n}</math>. The ''ensemble average'' of <math> x </math> is denoted as <math> X </math> (or <math> \left\langle x \right\rangle </math> ), and ''is defined as''<br />
<br />
:<math>X = \left\langle x \right\rangle \equiv \lim_{N \rightarrow \infty} \frac{1}{N} \sum^{N}_{n=1} x_{n}</math><br />
<br />
Obviously it is impossible to obtain the ensemble average experimentally, since we can never achieve an infinite number of independent realizations. The most we can ever obtain is the arithmetic mean for the number of realizations we have. For this reason the arithmetic mean can also referred to as the ''estimator'' for the true mean ensemble average.<br />
<br />
Even though the true mean (or ensemble average) is unobtainable, nonetheless, the idea is still very useful. Most importantly,we can almost always be sure the ensemble average exists, even if we can only estimate what it really is. The fact of its existence, however, does not always mean that it is easy to obtain in practice. All the theoretical deductions in this course will use this ensemble average. Obviously this will mean we have to account for these "statistical differenced" between true means and estimates when comparing our theoretical results to actual measurements or computations.<br />
<br />
<font color="orange" size=3>Figure 2.1 not uploaded yet</font><br />
<br />
In general, the <math> x_{n}</math> could be realizations of any random variable. The <math> X </math> defined by the ensemle average definition defined above represents the ensemble average of it. The quantity <math> X </math> is sometimes referred to as the ''expected value '' of the random variables <math> x </math> , or even simple its ''mean''.<br />
<br />
For example, the velocity vector at a given point in space and time <math>\vec{x},t </math> , in a given turbulent flow can be considered to be a random variable, say <math> u_{i} \left( \vec{x},t \right) </math>. If there were a large number of identical experiments so that the <math> u^{\left( n \right)}_{i} \left( \vec{x},t \right) </math> in each of them were identically distributed, then the ensemble average of <math> u^{\left( n \right)}_{i} \left( \vec{x},t \right) </math> would be given by<br />
<br />
:<math>\left\langle u_{i} \left( \vec{x} , t \right) \right\rangle = U_{i} \left( \vec{x} , t \right) \equiv \lim_{N \rightarrow \infty} \frac{1}{N} \sum^{N}_{n=1} u^{ \left( n \right) }_{i} \left( \vec{x} , t \right)</math><br />
<br />
Note that this ensemble average, <math> U_{i} \left( \vec{x},t \right) </math> , will, in general, vary with independent variables <math>\vec{x}</math> and <math>t</math>. It will be seen later, that under certain conditions the ensemble average is the same as the average which would be generated by averaging in time. Even when a time average is not meaningful, however, the ensemble average can still be defined; e.g., as in non-stationary or periodic flow. Only ensemble averages will be used in the development of the turbulence equations here unless otherwise stated.<br />
<br />
== Fluctuations about the mean ==<br />
<br />
It is often important to know how a random variable is distributed about the mean. For example, figure 2.1 illustrates portions of two random functions of time which have identical means, but are obviously members of different ensembles since the amplitudes of their fluctuations are not distributed the same. it is possible to distinguish between them by examining the statistical properties of the fluctuations about the mean (or simply the fluctuations) defined by:<br />
<br />
:<math>x' = x - X</math><br />
<br />
It is easy to see that the average of the fluctuation is zero, i.e.,<br />
<br />
:<math>\left\langle x'' \right\rangle = 0</math><br />
<br />
On the other hand, the ensemble average of the square of the fluctuation is ''not'' zero. In fact, it is such an important statistical measure we give it a special name, the '''variance''', and represent it symbolically by either <math> var \left[ x \right] </math> or <math> \left\langle \left( x' \right) ^{2} \right\rangle </math> <br />
The ''variance'' is defined as<br />
<br />
:<math>var \left[ x \right] \equiv \left\langle \left( x' \right) ^{2} \right\rangle = \left\langle \left[ x - X \right]^{2} \right\rangle</math><br />
<br />
:<math>= \lim_{N\rightarrow \infty} \frac{1}{N} \sum^{N}_{n=1} \left[ x_{n} - X \right]^{2}</math><br />
<br />
Note that the variance, like the ensemble average itself, can never really be measured, since it would require an infinite number of members of the ensemble.<br />
<br />
It is straightforward to show from the definition of ensemble average the variance can be written as <br />
<br />
:<math>var \left[ x \right] = \left\langle x^{2} \right\rangle - X^{2}</math><br />
<br />
Thus the variance is the ''second-moment'' minus the square of the ''first-moment'' (or mean). In this naming convention, the ensemble mean is the ''first moment''.<br />
<br />
The variance can also referred to as the ''second central moment of x''. The word central implies that the mean has been subtracted off before squaring and averaging. The reasons for this will be clear below. If two random variables are identically distributed, then they must have the same mean and variance.<br />
<br />
The variance is closely related to another statistical quantity called the ''standard deviation'' or root mean square (''rms'') value of the random variable <math> x </math> , which is denoted by the symbol, <math> \sigma_{x} </math>. Thus,<br />
<br />
:<math>\sigma_{x} \equiv \left( var \left[ x \right] \right)^{1/2}</math><br />
<br />
or <br />
<br />
:<math>\sigma^{2}_{x} = var \left[ x \right]</math><br />
<br />
<font color="orange" size=3>Figure 2.2 not uploaded yet</font><br />
<br />
== Higher moments ==<br />
<br />
Figure 2.2 illustrates two random variables of time which have the same mean and also the same variances, but clearly they are still quite different. It is useful, therefore, to define higher moments of the distribution to assist in distinguishing these differences.<br />
<br />
The <math>m</math>-th moment of the random variable is defined as<br />
<br />
:<math>\left\langle x^{m} \right\rangle = \lim_{N \rightarrow \infty} \frac{1}{N} \sum^{N}_{n=1} x^{m}_{n}</math><br />
<br />
It is usually more convenient to work with the ''central moments'' defined by:<br />
<br />
:<math>\left\langle \left( x' \right)^{m} \right\rangle = \left\langle \left( x-X \right)^{m} \right\rangle = \lim_{N \rightarrow \infty} \frac{1}{N} \sum^{N}_{n=1} \left[x_{n} - X \right]^{m}</math><br />
<br />
The central moments give direct information on the distribution of the values of the random variable about the mean. It is easy to see that the variance is the second central moment (i.e., <math> m=2 </math> ).<br />
<br />
<br />
{{Turbulence credit wkgeorge}}<br />
<br />
{{Chapter navigation||Probability}}</div>Ayyoubzadeh