1

Say we monitor a ski jumper's consecutive jumps (distance jumped), and we assume it is a combination (sum) of the skier's skills, + luck (modelled by a random variable). We also assume that the distance jumped at trial$_N$ is independent of that jumped at trial$_{N-1}$.

By some accounts (e.g. Daniel Kahneman, in his book Thinking Fast and Slow, p.178), after a very good first jump (i.e. above the skier's base-rate average, assuming luck=neutral), the regression to the mean principle gives us reason to believe the second is going to be worse (below average), simply because on average we have to get closer to the mean.

But this seems to contradict the principle whereby consecutive samples (with replacement) are independent, i.e. same base-rate probability for each sampling. For instance, if we roll a dice and it lands on 5, there is (I think) no reason to believe that the second roll should be below average (i.e. 1, 2 or 3) just because the first roll was above-average.

Is this in fact underlied/formalised by a broader statistical principle that I am missing here?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
z8080
  • 1,598
  • 1
  • 19
  • 38

1 Answers1

1

You both mis-state and misinterpret what Kahneman says. You should have offered a quote rather than described it from memory -- and indeed doing so may have resolved your difficulty before you even posted the question.

Firstly, the story about the ski jumping is about commentators' misunderstanding of regression to the mean (all but the first half sentence of this is on p179 of my copy). An actual explanation of regression to the mean is immediately before the ski jumping story and is in terms of golf:

  • the golfer who did well on day 1 is likely to be successful on day 2, but less than on the first, because the unusual luck he probably enjoyed on day 1 is unlikely to hold.
  • The golfer who did poorly on day 1 will probably be below average on day 2, but will improve, because his probable streak of bad luck is not likely to continue.

then shortly after he says:

My students were always surprised to hear that the best predicted performance on day 2 is more moderate, closer to the average than the evidence on which it is based (the score on day 1). This is why the pattern is called regression to the mean.

By contrast you say:

after a very good first jump ... the regression to the mean principles gives us reason to believe the second is going to be worse...

correct so far

... (below average)

No!

The point is that you expect it to be below the unusually good jump (worse than the jump you just did), not below the skiers own average.

But this seems to contradict the principle whereby consecutive samples (with replacement) are independent,

Kahneman does not assume independence. If you read the golf discussion carefully, he assumes consecutive scores are positively related (you'll score better than average on the second day, too, just closer to the mean than the good score you just had).

For instance, if we roll a dice and it lands on 5, there is (I think) no reason to believe that the second roll should be below average (i.e. 1, 2 or 3) just because the first roll was above-average.

That's not what regression to the mean says.

Your example with the die is slightly problematic because of issues with "closer to" when combined with only a few discrete outcomes (it also throws out the positive dependence Kahneman actually discusses -- and which typical is in real regression to the mean scenarios -- but let's leave that aside to begin).

First let's modify it by taking the sum of two dice and assume the first roll was unusually high -- you get a total of 10. Regression to the mean would suggest the next roll will be more likely to be less extremely high -- typically nearer to 7 -- than even more extreme (e.g. you will be more likely to get 7,8 or 9 than to get 11 or 12).

Secondly, let's now reintroduce the positive dependence. Imagine we have two dice, one red and one green. We roll 10 again (and for simplicity imagine we rolled two 5's). We re-roll the green one but keep the red one. Now the total is still likely to exceed the long term average (7), but will still tend to be closer to 7 rather than further from it than the 10 was. That is what regression to the mean is about, and that's what Kahneman describes.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thanks. It may be that Kahneman was expressing the worse performance on jump #2 in terms of being lower than jump #1, not than the skier's own average. To me it did not read this way, and this is because the performance is modelled as starting from a "baseline" represented by the skier's objective skills. – z8080 Oct 24 '16 at 11:17
  • To me, this (at least apparent) contradiction between regression to the mean and the independence of consecutive samples is still very interesting/challenging as a stats problem; and one that, I think, be reconciled by simply assuming that scores in the former are (as is it usually assumed) positively (and tightly) correlated. If we can't assume this, then performance on jump #1 should not at all influence our expected value of jump #2, for which the base-rate is still the best guess. – z8080 Oct 24 '16 at 11:18
  • To follow your sum-of-two-dice example, keeping the first dice and only re-rolling the second is a way of making the consecutive samples be correlated. But if we instead re-roll *both* dices, then am I correct that regression to the mean no longer applies in the same way, i.e. if roll #1 was 5+5=10, we still expect roll #2 to have a sum of 7 rather than being lower than roll #1? And that, in Bayesian terms, the presence of the assumption lets us continuously update the prior into a posterior, whereas without it (no correlation between scores), we are stuck with the base-rate prior? – z8080 Oct 24 '16 at 11:18
  • You still have that if you roll high the expected value of the next roll will be lower, but it would not longer be "surprising" in a way that would make us call it "regression". (The original examples from which the name originated related to inheritance of characteristics.) – Glen_b Oct 24 '16 at 11:39
  • But I really don't get what the expected value of event N+1 in any way depends on the value of event N, no matter how extreme that may have been. I mean, if we assume there is no positive correlation, then they're just independent events (samples), no?! In other words, P(R2) = P(R2|R1)! – z8080 Oct 24 '16 at 13:39
  • It's one thing to say expectation for roll2 changes as a function of roll1; and quite another to say we expect the value of *any* event to be close to the expected mean regardless of previous events. In the case of an extreme 1st roll, the expected value of the 2nd roll will be less extreme than roll1, but that's just bcs the expected value is less extreme than roll1. – z8080 Oct 24 '16 at 13:39
  • "regression to the mean" is NOT about E(X2|X1). It's about P(X2X1|X1 low). We've moved well beyond clarification of above answer into 1-on-1 tutorial (one where I seem to be mostly repeating information that's already in the answer). That's not the purpose of the site. – Glen_b Oct 24 '16 at 21:04
  • Haha that's a bit harsh given that was the first time you actually mention any conditional probabilities in your answer, but yes, I'm clear on my question now, so thank you. – z8080 Oct 25 '16 at 23:01
  • It's the first time I wrote it in mathematical notation, but it's not remotely the first mention of it It's plainly described in words throughout my original answer. P(X2 – Glen_b Oct 26 '16 at 02:40