5

As we all know , the GLM has the structure: $G(EY)=X^{T}\beta$, in which $G(.)$ is a known link function. What confuse me is that some people say that it's a semiparametric model. But in my opinion, it's a parametric model, because there is no nonparametric part and all we don't know in the previuos structure is $\beta$.

Could anybody tell me the reason why someone call it a semiparametric model. Thanks!

shijing SI
  • 593
  • 1
  • 6
  • 14
  • 1
    Perhaps I am missing something, but I think that if $G$ is fixed, then it is called a 'parameteric model'. In particular, if the model allows linear predictors $\beta$ and arbitrary link, then it is called 'semi-parametric model'. –  May 05 '12 at 11:25
  • Maybe if one estimates the link function, it could be seen as semi-parametric. Also, could it be that "the people" mean Generalized Additive Models? – Momo May 05 '12 at 12:03
  • Semiparametric is usually used to describe a problem where some parameters are estimated but a particular unknown distribution that is part of the model does not have a specific parametric form. A good example is the Cox proportional hazard model. The regression parameters are estimated but the baseline hazard function is arbitrary and is removed from the problem by applying partial likelihood. Do you have a form of GLM that allows you to use an arbitrary error term distribution? That ia the only way I can think of to make it semiparametric. – Michael R. Chernick May 05 '12 at 12:34
  • @Michael: With "arbitrary error distribution", do you mean the error distribution of the latent variable that gets discretized? That would then correspond to using different link functions as procrastinator said. – Momo May 05 '12 at 12:42
  • @Procrastinator So in your opinion, if $G$ is a known function, then the generalized linear model is a parametric model. Right? – shijing SI May 05 '12 at 12:55
  • @shijingSI Yes. In this [paper](http://www.jstor.org/stable/2291390?seq=1) this is slightly mentioned. –  May 05 '12 at 12:57
  • @Momo I know that there are a number of differences between generalized additive model and generalized linear model. I just don't understand why some people claim that generalized linear model is a semi-parametric model. – shijing SI May 05 '12 at 13:00
  • 2
    Who claims that? It would help to have a reference, because in the reference there's likely to be an explanation. – jbowman May 05 '12 at 13:58
  • I was just grasping at straws trying to find something that would include a nonparametric component. I wasn't thinking of a latent variable just the noise term that is often assumed to be Gaussian with mean 0. – Michael R. Chernick May 05 '12 at 20:12

1 Answers1

8

A GLM isn't a semi-parametric model, but the output from typical use of GLMs can be justified with only semi-parametric assumptions.

If one only assumes that the observations $Y_1, Y_2, ... Y_n$ are independent and that $$ g(\mathbb{E}[\,Y_i|X_i=x_i\,]) = x_i^T\beta $$ then, under mild regularity conditions, solving the equations $$ \sum_i\frac{\partial g^{-1}(x_i^T\beta)}{\partial \beta}w(g^{-1}(x_i^T\beta))(Y_i - g^{-1}(x_i^T\beta)) = \mathbf{0} $$ provides consistent estimates for parameter $\beta$. The weighting term $w$ is arbitrary, but it determines the efficiency of this approach, and the best option is to use weights inversely proportional to the variance of $Y_i$, if you know this.

How does this connect to GLMs? Well, the estimating equation above is just the score equation (i.e. the one that determines the MLE), under the assumption of a GLM. A particularly simple case of thise is when we use the "canonical" link function, chose so that part of the derivative term cancels with the inverse-variance weights, and we get $$ \sum_i x_i(Y_i - g^{-1}(x_i^T\beta)) = \mathbf{0}, $$ which should look familiar to anyone who's studied linear regression, or logistic regression, or Poisson regression.

In general, we can view the point estimates from GLMs as MLEs under a particular fully parametric model for $Y$, or as consistent & efficient estimates resulting from assumptions on only the first and second moments of $Y$ - i.e. a semi-parametric model.

Similar arguments apply to the confidence intervals these methods provide; see e.g. McCullagh and Nelder's book for the details.

guest
  • 2,381
  • 14
  • 11
  • +1. Would you agree that if $g^-{1}$ would not be treated as known but would be estimated too, this would become completely semi-parametric? – Momo May 05 '12 at 19:25
  • Well, it is completely semi-parametric, so I'm not sure what you mean. There are completely non-parametric interpretations of the point estimates, but in these the link function is still assumed; e.g. using linear regression it is assumed that one is it interested in a linear trend, summarizing the population from which the data comes. – guest May 05 '12 at 22:01
  • Because you said "A GLM isn't a semi-parametric model...". I read your answer as if you say that the model is parametric, but can also be seen as semi-parametric if you just view it as estimating mean and/or variance. With "completely semi-parametric" I meant with no parametric interpretation. It might also depend on the definition of semi- or non-parametric for that matter, because I don't see what the space $\mathcal{F}$ is in e.g. logistic regression when we see the parameter space as $\Theta \subset \mathbb{R}^k \times \mathcal{F}$. Care to help me out? – Momo May 05 '12 at 22:35
  • So when you said "this would become" initially, you meant "this model" and not "this interpretation of the output"? I was talking about the latter. If one makes the choice of $g$ very flexible, then yes, with care you can view the limit of a parametric procedure as a semi-parametric one. But it requires more caution about the regularity than it seems wise to get into here. – guest May 05 '12 at 23:27
  • 2
    My main point is that the output we get from GLMs, which its easy to justify via parametric arguments, can also be interpreted as the output from a semi-parametric analysis. This is important because the latter analysis, while numerically identical, requires many fewer assumptions. – guest May 05 '12 at 23:33
  • 1
    Yes, I meant the model becomes semi-parametric. When the OP asked, I just wondered how to make the GLM semi-parametric i.e. with a combined finite and infinite parameter space. And all I could think of was something along the lines of dx.doi.org/10.2307%2F2951556 But I think I now understand what you mean: The model that is used is parametric, but the result relates to the first two moments only and therefore the infinite parameter space would be a family of distributions with the same two moments, right? Thanks for the nice new insights. – Momo May 05 '12 at 23:48
  • Yes, that's right. And you're welcome. – guest May 06 '12 at 23:45