7

I am experiencing some cognitive dissonance about what 'linear in the parameters' means. For example, here and here.

For example, my understanding is $y_i = \beta_0 + \beta_1\beta_2x_1 + \exp(\beta_3)(x_2)^2 + \epsilon$ is not linear in the parameters, because it has two parameter variables multiplied together (namely ${\beta_1, \beta_2}$).

If $\beta_1$ (say) was replaced with $\gamma_1$, a constant, it would be.

Appreciate if someone could clarify this point.

Ben S
  • 73
  • 4
  • While you Are correct the function is not linear in the parameters, it can be made so with a log transformation – Repmat Mar 18 '16 at 08:42
  • @Repmat How so? How will log transformation help here? – Nick Cox Mar 18 '16 at 13:28
  • I don't see that **anything**, linear or nonlinear, will make $\beta_1, \beta_2$ separately estimable here. More positively, watch that GLM in different contexts means general linear models and generalized linear models, which overlap but are by no means identical classes. – Nick Cox Mar 18 '16 at 13:29
  • Removing the product and the exp – Repmat Mar 18 '16 at 13:31
  • 1
    OK, so you are reparameterising. That's not transformation (of variables), which was what I was inferring. – Nick Cox Mar 18 '16 at 13:33
  • @Repmat What are you taking the log of, exactly? I don't see how that works. – Glen_b Mar 18 '16 at 17:57
  • @Nickcox exactly what I meant. But I suppose that $\beta_1$ would not be identified. I see the comment is unclear – Repmat Mar 18 '16 at 18:04

3 Answers3

6

Your example model can be reëxpressed to be linear in the parameters $\alpha=\beta_1\beta_2$ & $\zeta=\exp\beta_3$:

$$g(\operatorname{E} Y) = \beta_0 + \alpha x_1 + \zeta x_2^2$$

(Clearly $\beta_1$ & $\beta_2$ aren't separately estimable; a non-linear model wouldn't help there. And note that $\hat\zeta$ must be constrained to be positive.) Some models can't be so reëxpressed:

$$g(\operatorname{E} Y) = \beta_0 + \beta_1 x_1 + x_2^{\beta_2}$$

Some can be though it's not obvious at first: https://stats.stackexchange.com/a/60504/17230.

There's a very thorough discussion of different meanings of "linear" at How to tell the difference between linear and non-linear regression models?.

Community
  • 1
Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
5

Linear in the parameters means that you can write your prediction as

$$\beta_0+\sum_{j=1}^px_{ij}\beta_j $$

For some definition of $x_{ij} $. But these x's need not be linear functions of your data. For example, ploynomial fitting of a time series has $x_{ij}=t_i^j $ where $t_i $ is the time associated with data point $i $. The prediction is a non linear function of time, but it is linear in the betas.

UPDATE

In response to the comment, the answer is "sort of". If $\beta_2$ was constant, then the predictor is linear in $\beta_0,\beta_1,\exp (\beta_3) $. It is not linear in $\beta_3$, but a transformation of $\beta_3$. In terms of least squares estimates it doesn't make much difference here.

probabilityislogic
  • 22,555
  • 4
  • 76
  • 97
  • Thanks so much for replying. Perhaps my question isn't clarified by including a transformation of the x's. I am asking about the beta's (parameters), not transforms of x's. Perhaps if you could comment on my specific example above. – Ben S Mar 18 '16 at 06:35
0

I think it's better for you to understand the three components of the GLM. Esp, you need understand how link function is defined.

You can refer to the page 7 in the slides below. 'linear in the parameters' is true after being transformed by the link function.

enter link description here

Shijia Bian
  • 503
  • 3
  • 8