18

When was the word "bias" coined to mean $\mathbb{E}[\hat{\theta}-\theta]$?

The reason why I'm thinking about this right now is because I seem to recall Jaynes, in his Probability Theory text, criticizing the use of the word "bias" used to describe this formula, and suggesting an alternative.

From Jaynes' Probability Theory, section 17.2 "Unbiased Estimators:"

Why do orthodoxians put such exaggerated emphasis on bias? We suspect that the main reason is simply that they are caught in a psychosemantic trap of their own making. When we call the quantity $(\langle\beta\rangle-\alpha)$ the 'bias', that makes it sound like something awfully reprehensible, which we must get rid of at all costs. If it had been called instead the 'component of error orthogonal to the variance', as suggested by the Pythagorean form of (17.2), it would have been clear that these two contributions to the error are on an equal footing; it is folly to decrease one at the expense of increasing the other. This is just the price one pays for choosing a technical terminology that carries an emotional load, implying value judgments; orthodoxy falls constantly into this tactical error.

amoeba
  • 93,463
  • 28
  • 275
  • 317
Clarinetist
  • 3,761
  • 3
  • 25
  • 70
  • https://stats.stackexchange.com/questions/207760/when-is-a-biased-estimator-preferable-to-unbiased-one#207764 You might want to check the comments here. Also, my hunch would be that Jaynes criticized it because he was a bayesian – martn Dec 01 '17 at 13:22
  • 2
    According to Favid's list of "First (?) occurrence of common terms in mathematical statistics" it was first used in 1897 although the idea goes back further than that. I suspect that is not what you wanted to know which is why is is only a comment. – mdewey Dec 01 '17 at 13:28
  • 4
    @Clarinetist it is perfectly on-topic, this is what for we have the `[etymology]` tag. – Tim Dec 01 '17 at 13:47
  • Looking briefly around, I found that the etymology of the noun bias may be the Latin [bifax])https://fr.wiktionary.org/wiki/bifax#la), meaning bi-face, or the Latin biaxis, meaning two axes. – Xi'an Dec 01 '17 at 21:29
  • 2
    This is a beautiful quote from Jaynes. On a different matter, are you strictly interested in the etymology of the word "bias", or in the broader question of its historical appearance/use in statistics --- the later seems more interesting, I would say, than the strict etymology. – Carlos Cinelli Dec 02 '17 at 23:37
  • 1
    @CarlosCinelli Yeah, you're right. I mean the historical appearance - i.e., when was it coined in stats. – Clarinetist Dec 02 '17 at 23:40
  • The question is problematic as it is not clear how much hinges on your notation, including precisely what it means. Does the hat mean any estimator? Are you seeking a source using similar notation? Using similar concepts of a expectation across a distribution of estimates? Bowley in 1897 often appears in searches for bias, but he was talking about errors, and was some way off more modern ideas of estimators. – Nick Cox Dec 11 '17 at 12:42

1 Answers1

8

Apparently, the concept of mean bias was coined by:

Neyman, J., & Pearson, E. S. (1936). Contributions to the theory of testing statistical hypotheses. Statistical Research Memoirs, 1, 1-37.

acccording to:

Lehmann, E. L. "A General Concept of Unbiasedness" The Annals of Mathematical Statistics, vol. 22, no. 4 (Dec., 1951), pp. 587–592.

which contains a more extensive discussion on the history of this concept.

It is worth noticing that mean bias is just a type of bias, and there also exists the concept of median bias (which cannot be straightforwardly extended to the multivariate case, which may explain why it is not that popular).

MrPresident
  • 106
  • 2
  • 5
    Slight correction to the above: it appears that Neyman and Pearson defined "unbiased" in the context of hypothesis testing. It appears that, with regard to a point estimate, this concept was defined in David F. N., & Neyman, J. "Extension of the Markoff theorem on least squares," Statistical Research Memoirs, pp. 105-116. This citation is also found in the Lehmann article. – Clarinetist Dec 04 '17 at 15:56