2

I'm beginning with the regression analysis and I'm quite confused with the generalized linear regression.

I understand, that the ordinary linear models can be described with a formula

$$ y_i = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i $$

and when using generalized linear regression I'd expect it to be something like this

$$ g\left(y_i\right) = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i, $$ $g$ being the link function.

But, instead I came across this formula

$$ g\left(E[\mathbf{y}]\right) = \mathbf{x'}\mathbf{\beta} $$

and I don't get, what's the expected value in the left doing there.

Could you, please, explain it a little?

Eenoku
  • 443
  • 8
  • 17
  • Even in regular regression, the idea is to fit the model in such a way, that $y_i$ is the expected value of $\bf y$ given $x_i$. – January Mar 31 '17 at 08:46
  • Following on from January's point there, note that GLMs don't transform the *data*; they transform a linear function of the predictors. See http://stats.stackexchange.com/questions/67626/understanding-of-glm and http://stats.stackexchange.com/questions/262019/logit-link-in-glm-and-inverse-logit ... See also point 2. in the answer here: http://stats.stackexchange.com/questions/182076/interpreting-glm-regression-analysis-result/182132#182132 – Glen_b Mar 31 '17 at 09:05

1 Answers1

3

You are correct that regression model is

$$ y_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i $$

that can be written differently as

$$ \mu_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n \\ y_i = \mu_i + \epsilon_i $$

or saying it differently

$$ y_i \sim \mathcal{N}(\mu_i, \sigma^2) $$

i.e. we predict conditional mean of $Y$ given $X$ and assume residuals to be normally distributed around it.

and when using generalized linear regression I'd expect it to be something like this

$$ g\left(y_i\right) = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i, $$

$g$ being the link function.

But, instead I came across this formula

$$ g\left(E[\mathbf{y}]\right) = \mathbf{x'}\mathbf{\beta} $$

and I don't get, what's the expected value in the left doing there.

The above definition is correct. GLM's are generalizations of regression to distributions other then normal and link functions other then identity link. GLM is defined in terms of linear predictor

$$ \eta_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n $$

that is passed through inverse of the link function $g$ to obtain mean

$$ E(Y|X) = \mu_i = g^{-1}(\eta_i) $$

and $\mu_i$ is the conditional mean of $Y$ given $X$ of some probability distribution of choice (e.g. normal, Poisson, Bernoulli etc.).

Tim
  • 108,699
  • 20
  • 212
  • 390
  • Thank you very much. Just two little questions: 1) I suppose, that linear predictor in GLM should be $\mu$ and not $\eta$. Shouldn't it? 2) And is it possible to write it "my way" too or is it incorrect? – Eenoku Mar 31 '17 at 09:01
  • 2
    @Eenoku linear predictor is $\eta$ you transform it using inverse link to mean $\mu$. It is not possible to write GLM in terms of mean + noise since in cases other then normal (linear regression) it would be hard to define the "noise", e.g. in logistic regression you predict mean of Bernoulli distribution, so mean is real number in unit interval and $Y$ is 0's and 1's -- the "noise" would have nothing to do with Bernoulli distribution etc. Actually defining the model in terms of noise is pretty confusing and it is better to think of it in terms of predicting mean. – Tim Mar 31 '17 at 09:59