Least approximation of derivative

TurbJet · May 12, 2020, 00:05

Greetings,

Recently, I've read some papers in which they are using least-square to approximate the 2bd-order derivatives of a given 1-D data. I couldn't figure out how to do this, and I couldn't find any helpful references online.

Can anyone give me some hint, or recommend some references?

Appreciate it!

Edit: To be more specific, my purpose is to evaluate the following 2nd-order derivative

$\frac{d^2F(\kappa)}{d(\log\kappa)^2}$

where $\kappa$ is just the integer wavenumber, and $F(\kappa)$ is a given function in spectral space.

tas38 · May 12, 2020, 00:52

Quote:

Originally Posted by TurbJet

Greetings,

Recently, I've read some papers in which they are using least-square to approximate the derivatives. I couldn't figure out how to do this, and I couldn't find any reference online.

Can anyone give me some hint, or recommend some references?

Appreciate it!

Are you referring to a cell-centered gradient approximation using a least-squares-based approach?

sbaffini · May 12, 2020, 04:32

The general idea is that you write a 1st order taylor series approximation of a variable phi in the neighborhood n of the point c where you need the gradient (it is used for gradients in general) and assume that:

phi_n = phi_c + dphi/dx_i * (x_n-x_c)_i

Write this for all the available neighbors of a given point c and you get a system of equations that, in most cases, needs to be solved by a least squares method because you have more neighbors than gradient components.

However, the exact least squares solution of the equations is available and you don't typically solve it by a numerical procedure.

There is abundant literature on this. I suggest the book by Blazek as textbook reference because it is the one that covers it best (he and Haselbacher actually developed one of the available closed solutions to the problem).

EDIT: a different version of the exact solution (basically, you have the one based on solving the normal equations and the one based on QR factorization) can also be found in several of the early works of Prof. Z. J. Wang but, unfortunately, each one of them has a slight little error, and you need to look at all of them together to catch it.

Let me also highlight that the work of Blazek and Haselbacher is more general and has, indeed, been extended to higher than first order approximations (used for k-exact rexonstructions) as well as to filters for unstructured grids.

Doru Caraeni also put forward some developments/derivations for the easy application for 3rd order codes

Finally, Nishikawa and coworkers have also made several studies recently

Eifoehn4 · May 12, 2020, 13:20

Or you simply use this slide:

edit: By the way it is called Least Squares!

TurbJet · May 12, 2020, 21:37

Quote:

Originally Posted by tas38

Are you referring to a cell-centered gradient approximation using a least-squares-based approach?

Maybe, but I am not sure. The paper I read is using this least-squares approximation to approximate the 2nd-order derivative of the energy flux of HIT in spectral space. The reason they're using this is because the underlying grid is non-uniform.

TurbJet · May 12, 2020, 21:42

Quote:

Originally Posted by Eifoehn4

Or you simply use this slide:

edit: By the way it is called Least Squares!

Is this applicable to 1D case? I am just looking for an approach to differentiate a function on a non-uniform grid. The paper I referred to use this least-squares approximation to find out the 2nd-derivative of a given dataset.

TurbJet · May 12, 2020, 21:49

Quote:

Originally Posted by sbaffini

The general idea is that you write a 1st order taylor series approximation of a variable phi in the neighborhood n of the point c where you need the gradient (it is used for gradients in general) and assume that:

phi_n = phi_c + dphi/dx_i * (x_n-x_c)_i

Write this for all the available neighbors of a given point c and you get a system of equations that, in most cases, needs to be solved by a least squares method because you have more neighbors than gradient components.

However, the exact least squares solution of the equations is available and you don't typically solve it by a numerical procedure.

There is abundant literature on this. I suggest the book by Blazek as textbook reference because it is the one that covers it best (he and Haselbacher actually developed one of the available closed solutions to the problem).

EDIT: a different version of the exact solution (basically, you have the one based on solving the normal equations and the one based on QR factorization) can also be found in several of the early works of Prof. Z. J. Wang but, unfortunately, each one of them has a slight little error, and you need to look at all of them together to catch it.

Let me also highlight that the work of Blazek and Haselbacher is more general and has, indeed, been extended to higher than first order approximations (used for k-exact rexonstructions) as well as to filters for unstructured grids.

Doru Caraeni also put forward some developments/derivations for the easy application for 3rd order codes

Finally, Nishikawa and coworkers have also made several studies recently

I did some digging after I posted this thread, most of the contents are applied to CFD (at least from what I can tell). However, I am looking for a general approach that can be simply applied to evaluate differentiations of a given 1D data with non-uniform grid; and that's it, I don't need fancy algorithms that can solve complex fluid problems. So I think my need is just a simple numerical analysis of 1-D derivative. Do you think the approach you mentioned can be used for this purpose?

Eifoehn4 · May 13, 2020, 02:19

Quote:

Originally Posted by TurbJet

Is this applicable to 1D case? I am just looking for an approach to differentiate a function on a non-uniform grid. The paper I referred to use this least-squares approximation to find out the 2nd-derivative of a given dataset.

Sure you can! In 1D with equidistant distributed data this will result in a standard 2nd order FD stencil. However you can use it also for non-equidistant data. For a second order derivative simply apply the operator twice.

Quote:

Originally Posted by TurbJet

Greetings,

Edit: To be more specific, my purpose is to evaluate the following 2nd-order derivative

$\frac{d^2F(\kappa)}{d(\log\kappa)^2}$

where $\kappa$ is just the integer wavenumber, and $F(\kappa)$ is a given function in spectral space.

I think you make the problem harder than it is. Note that

$\frac{dF}{d(\log\kappa)} = \frac{1}{\frac{d(\log\kappa)}{dF}} = \frac{1}{\frac{d(\log\kappa)}{dF}\frac{d \kappa}{d\kappa}}= \frac{1}{\frac{d(\log\kappa)}{d \kappa }\frac{d \kappa}{dF}} = \frac{1}{\frac{1}{\kappa }\frac{d \kappa}{dF}} = \kappa \frac{dF}{d \kappa}$

and therefore

$\frac{d^2 F}{d(\log\kappa)^2} = \kappa \frac{dF}{d \kappa} + \kappa^2 \frac{d^2 F}{d \kappa^2}$ .

This is more easy to discretisize.

Regards

sbaffini · May 13, 2020, 06:10

Quote:

Originally Posted by Eifoehn4

Sure you can! In 1D with equidistant distributed data this will result in a standard 2nd order FD stencil. However you can use it also for non-equidistant data. For a second order derivative simply apply the operator twice.

I think you make the problem harder than it is. Note that

$\frac{dF}{d(\log\kappa)} = \frac{1}{\frac{d(\log\kappa)}{dF}} = \frac{1}{\frac{d(\log\kappa)}{dF}\frac{d \kappa}{d\kappa}}= \frac{1}{\frac{d(\log\kappa)}{d \kappa }\frac{d \kappa}{dF}} = \frac{1}{\frac{1}{\kappa }\frac{d \kappa}{dF}} = \kappa \frac{dF}{d \kappa}$

and therefore

$\frac{d^2 F}{d(\log\kappa)^2} = \kappa \frac{dF}{d \kappa} + \kappa^2 \frac{d^2 F}{d \kappa^2}$ .

This is more easy to discretisize.

Regards

Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form $F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}$ , representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.

Eifoehn4 · May 13, 2020, 07:02

Quote:

Originally Posted by sbaffini

Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form $F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}$ , representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.

That's true. I think he has to test, which representation works better.

TurbJet · May 13, 2020, 15:29

Quote:

Originally Posted by Eifoehn4

That's true. I think he has to test, which representation works better.

Thanks for your previous reply. So I'll probably try both approaches.

TurbJet · May 13, 2020, 15:37

Quote:

Originally Posted by sbaffini

Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form $F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}$ , representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.

Could you explain more on this specific case that you gave? Why this spreading peak would affect the accuracy? Is it because it can't be accurately represented on a uniform grid?

sbaffini · May 13, 2020, 16:18

Well, the function I used is just an example of how the energy spectra behave at low wave numbers (I used k^4, but k^2 would have been another possibility), where they tipically have a peak.

Now, what approach is more suited obviously depends from what the final goal is, but I would point out the following:

1) The formula with log k is just a second derivative, while the formula with k has two derivatives (first and second) multiplied by k and k^2 respectively. More work and typically more error.

2) The steps in k are much larger than those in log k, so this is necessarily linked to higher error

3) Now that I think more about it, this might simply be a restatement of 2 but, if you plot the function above as function of k and of log k, the latter is much more smoother around the peak, which means that any given formula would give less error.

When I wrote the message this morning, I actually tried both formulas and they indeed differ at low wavenumbers for the function I mentioned. I deleted the script I used, but I suggest you to make a similar test to better understand them both.

TurbJet · May 13, 2020, 18:08

Quote:

Originally Posted by sbaffini

Well, the function I used is just an example of how the energy spectra behave at low wave numbers (I used k^4, but k^2 would have been another possibility), where they tipically have a peak.

Now, what approach is more suited obviously depends from what the final goal is, but I would point out the following:

1) The formula with log k is just a second derivative, while the formula with k has two derivatives (first and second) multiplied by k and k^2 respectively. More work and typically more error.

2) The steps in k are much larger than those in log k, so this is necessarily linked to higher error

3) Now that I think more about it, this might simply be a restatement of 2 but, if you plot the function above as function of k and of log k, the latter is much more smoother around the peak, which means that any given formula would give less error.

When I wrote the message this morning, I actually tried both formulas and they indeed differ at low wavenumbers for the function I mentioned. I deleted the script I used, but I suggest you to make a similar test to better understand them both.

I see. Great explanation and thanks a lot!

May 12, 2020, 00:05	Least approximation of derivative	#1
TurbJet Senior Member Join Date: Oct 2017 Location: United States Posts: 233 Blog Entries: 1 Rep Power: 9	Greetings, Recently, I've read some papers in which they are using least-square to approximate the 2bd-order derivatives of a given 1-D data. I couldn't figure out how to do this, and I couldn't find any helpful references online. Can anyone give me some hint, or recommend some references? Appreciate it! Edit: To be more specific, my purpose is to evaluate the following 2nd-order derivative $\frac{d^2F(\kappa)}{d(\log\kappa)^2}$ where $\kappa$ is just the integer wavenumber, and $F(\kappa)$ is a given function in spectral space. Last edited by TurbJet; May 12, 2020 at 22:04.

May 12, 2020, 04:32		#3
sbaffini Senior Member Paolo Lampitella Join Date: Mar 2009 Location: Italy Posts: 2,152 Blog Entries: 29 Rep Power: 39	The general idea is that you write a 1st order taylor series approximation of a variable phi in the neighborhood n of the point c where you need the gradient (it is used for gradients in general) and assume that: phi_n = phi_c + dphi/dx_i * (x_n-x_c)_i Write this for all the available neighbors of a given point c and you get a system of equations that, in most cases, needs to be solved by a least squares method because you have more neighbors than gradient components. However, the exact least squares solution of the equations is available and you don't typically solve it by a numerical procedure. There is abundant literature on this. I suggest the book by Blazek as textbook reference because it is the one that covers it best (he and Haselbacher actually developed one of the available closed solutions to the problem). EDIT: a different version of the exact solution (basically, you have the one based on solving the normal equations and the one based on QR factorization) can also be found in several of the early works of Prof. Z. J. Wang but, unfortunately, each one of them has a slight little error, and you need to look at all of them together to catch it. Let me also highlight that the work of Blazek and Haselbacher is more general and has, indeed, been extended to higher than first order approximations (used for k-exact rexonstructions) as well as to filters for unstructured grids. Doru Caraeni also put forward some developments/derivations for the easy application for 3rd order codes Finally, Nishikawa and coworkers have also made several studies recently LuckyTran likes this. Last edited by sbaffini; May 12, 2020 at 08:23.

May 12, 2020, 13:20		#4
Eifoehn4 Senior Member - Join Date: Jul 2012 Location: Germany Posts: 184 Rep Power: 13	Or you simply use this slide: edit: By the way it is called Least Squares! sbaffini likes this.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error when calculating the second derivative using the derivative function d2dy2	mona.li	Tecplot	0	July 8, 2018 14:11
Time derivative of pressure in low mach number approximation	hnemati	Main CFD Forum	1	June 16, 2016 07:05
Tangential derivative on curved boundary	Simbelmynë	Main CFD Forum	40	October 14, 2014 06:57
discontinuous second derivative	new_at_this	Main CFD Forum	9	April 28, 2012 16:55
Different one sided derivative approximation?	zonexo	Main CFD Forum	2	January 11, 2006 04:40

May 13, 2020, 16:18		#13
sbaffini Senior Member Paolo Lampitella Join Date: Mar 2009 Location: Italy Posts: 2,152 Blog Entries: 29 Rep Power: 39	Well, the function I used is just an example of how the energy spectra behave at low wave numbers (I used k^4, but k^2 would have been another possibility), where they tipically have a peak. Now, what approach is more suited obviously depends from what the final goal is, but I would point out the following: 1) The formula with log k is just a second derivative, while the formula with k has two derivatives (first and second) multiplied by k and k^2 respectively. More work and typically more error. 2) The steps in k are much larger than those in log k, so this is necessarily linked to higher error 3) Now that I think more about it, this might simply be a restatement of 2 but, if you plot the function above as function of k and of log k, the latter is much more smoother around the peak, which means that any given formula would give less error. When I wrote the message this morning, I actually tried both formulas and they indeed differ at low wavenumbers for the function I mentioned. I deleted the script I used, but I suggest you to make a similar test to better understand them both.