1

I would like to estimate the partial derivatives of a dependent variable $y$ against a number of independent variables $x_i$, $i \in [0,n]$.

I can easily differentiate the dependent and independent variables and perform a simple linear regression to approximate the first order partial derivatives.

However it is not clear to me how to approximate the second (and higher order derivatives). Should I differentiate again all the variables and perform a new linear regression? Does this make any sense at all? Should I look into an alternate method? Which one?

ilanman
  • 4,503
  • 1
  • 22
  • 46
amiando
  • 43
  • 7
  • 2
    To get higher order derivatives you need to fit a higher order model. In the polynomial case this is, e.g. the basis of [moving least squares](https://en.wikipedia.org/wiki/Savitzky%E2%80%93Golay_filter) derivative estimators. Note that fitting a higher order polynomial is still [linear regression](http://stats.stackexchange.com/questions/92065/why-is-polynomial-regression-considered-a-special-case-of-multiple-linear-regres), but the problem can be [unstable](http://stats.stackexchange.com/a/234197/127790) for polynomials beyond say cubic or so. – GeoMatt22 Jan 03 '17 at 22:12
  • Are you suggesting to say fit y to x^2 in order to obtain the second derivative of y with respect to x? – amiando Jan 03 '17 at 22:24
  • That's one approach. It's problematic, though, because polynomials are too *global*: small changes in data values at one end of your "point cloud" of data can have profound effects on the fit everywhere. Instead people often rely on *local smoothers,* such as cubic splines. A lot depends on your assumptions about the degree of differentiability of your model and on how rapidly varying its derivatives might be. In just three lines, @GeoMatt gave you excellent advice. – whuber Jan 03 '17 at 22:28
  • @amiando: Assuming that you fit a local linear smoother the higher order terms will correspond the higher order derivatives. – usεr11852 Jan 03 '17 at 22:29
  • Thanks for the clarifications, I am only interested in second order derivatives for now so it seems that I can get away with polynomials for the time being. – amiando Jan 03 '17 at 22:33
  • Just be careful that: 1. the derivative estimates especially at the edges of your sample's support can vary massively and be effectively useless, 2. You also need to fit lower order terms too (constant, linear, etc.), 3. I read somewhere in the past that to use the $n$-th term from a local smoother I need to use $n+1$-th terms during local linear fitting. When I did simulations on this, it turned out to be worse so I didn't follow it up... You might want to consider it though. (If I get the ref. I will post it) 4. The denseness of your reading along $x$ is crucial. – usεr11852 Jan 03 '17 at 22:42

0 Answers0