8

I recently see a term "semi-parametric" in the answers of my question but not really understand what is this term means.

Wikipedia says

In statistics, a semiparametric model is a statistical model that has parametric and nonparametric components.

And gives Cox proportional hazards model as an example.

I see Cox proportional hazards model and logistic regression are very similar, why we say one is semi-parametric but not say another?

BTW I found this answer, says GLM is not a semi-parametric model.

Community
  • 1
Haitao Du
  • 32,885
  • 17
  • 118
  • 213

1 Answers1

15

The logistic regression is not "semi-parametric". It has only parametric component. For parametric model, the number of parameters is fixed and does not depend on the number of training data, but only depends on the model itself. This is true for logistic regression since if you have $n$ variables $X_1,\ldots,X_n$ you have $n+1$ parameters $w_0,\ldots,w_n$ to define the logistic regression model, and the number of these parameters does not increase or decrease based on the number of training data. Note that for non-parametric models you also have parameters, but the number of parameters is not fixed and depends on the number of training examples.

Hossein
  • 3,170
  • 1
  • 16
  • 32
  • 1
    Thanks can i use "if number of parameters are depending on number of row in data matrix" to see if a model is non parametric? – Haitao Du Mar 19 '17 at 03:59
  • 2
    Yes. Read this paragraph of the *Kevin Murphy's Machine Learning book* for more clarification: "does the model have a fixed number of parameters, or does the number of parameters grow with the amount of training data? The former is called a parametric model, and the latter is called a non- parametric model." – Hossein Mar 19 '17 at 04:20
  • Then what would a "semi-parametric" model be? Having both a fixed number and a data-dependent number of parameters? – R.M. Mar 19 '17 at 23:11
  • Yes! Partially linear regression and the proportional hazards model are two examples. – Hossein Mar 20 '17 at 09:19
  • @Hossein thanks for your help. But I am still not convinced we can use "if number of parameters are depending on number of row in data matrix" to check if it is a parametric method (although it is written in Murphy's book). For example, neural network, if network structure is fixed, then the number parameters will not depend on number of data. But it should be a non-parametric right? – Haitao Du Mar 20 '17 at 13:24
  • 1
    Why do you think neural networks are non-parametric? A neural net with a fixed structure is parametric. – Hossein Mar 21 '17 at 02:16