1

Suppose the beta coefficients of a weighted multiple regression model is given by the matrix formulation:

${\boldsymbol \beta} = ({\bf X}^T{\bf W}{\bf X})^{-1}{\bf X}^T{\bf Wy}$

Suppose that the dependent variable was correlated to one of the independent variables exponentially by its square (eg. $v_d=v_i^2$). Is it possible to incorporate such a non-linear relationship into the model using this formulation, or does it require a whole different method?

Jackson Capper
  • 125
  • 4
  • Could you clarify what you mean by "the dependent variable was correlated to one of the independent variables exponentially by its square"? Also $f(v_d) = v_i^2$ is notation unexplained. Is $v_d$ dependent variable and $f(.)$ is what function and $v_i ^2$ what is that? Do you simply have a model $y = .... \lambda x^2 ....$ where one independent variable is assumed squared on the RHS in the true model? – Jesper for President Oct 18 '19 at 11:46
  • Sorry for that; my mistake. I just meant the dependent variable was correlated exponentially to the independent as an example of a non-linear relationship. Yes I meant that its assumed, in that the analyst has eye-balled a scatter plot and seen a squared relationship and wants to incorporate that into the model. – Jackson Capper Oct 18 '19 at 11:50

1 Answers1

7

If this is the correct form, you could fit a linear regression with $E(Y|X) = b_0 + b_1 X_1 + b_2 X_2$ where $X_2$ is $X_1 \times X_1$.

This would present a relationship where $Y$ is quadratic in $X_1$ but linear in the beta parameters.

If you're not sure if a quadratic term is appropriate, and if you don't care much about interpreting the beta estimates easily, you could fit a restricted cubic spline of $X_1$ and model

$E(Y|X) = b_0 + \text{rcs}(X1).$

This is more flexible and requires less assumptions that may create out of sample stability.

Edited: I wrote "instability" above, but I believe RCS actually is more stable out of sample for predictions when the rcs is formulated in a reasonable manner. So, I changed it to "stability".

The overall short answer to your question: yes, because you can call $Z$ the new independent variable which is just a quadratic (or other) transformation of some $X_j$, then plug $Z$ into your beta calculation. The key would be linearity in the beta parameters.

LSC
  • 734
  • 3
  • 10
  • 1
    Great answer; thank you. To clarify, for a quadratic relationship, you would include $x$ itself and an artificial IV $x^2$ in the $X$ matrix and it would work out? The cubic would need $x$, $x^2$, and $x^3$ into the matrix? – Jackson Capper Oct 18 '19 at 12:49
  • [Yes](https://stats.stackexchange.com/a/28741/176202) – Frans Rodenburg Oct 18 '19 at 13:23
  • 1
    Exactly. Recall that linearity refers to the beta parameters and a transformed Xi such as X-squared is just a regular Xi for computational purposes. You know that it's derived from applying some transformation to an original independent variable, so that's useful more so for interpretation. In general, do not omit the lower order terms, as you noted. Finally the independent variable isn't really "artificial" as you called it. It's a legitimate term/variable but it just happens to be derived from another IV. – LSC Oct 19 '19 at 13:09