5

This is no doubt a most elementary question. Perhaps context is relevant.

Specifically someone said there is a $70$% chance the US stock market will take out the March lows in the next six months.

I can understand a statement such as the probability of having a particular disease given a positive test result when the sensitivity of the test, false positive rate, and presumed proportion of the population effected are taken into account.

But I am not sure how to interpret a stand-alone remark such as I am referring to.

I am not interested in what the person meant. Rather, what should someone who is cognizant of probability and statistics intend to imply when they make such a statement.

I assure you I am not looking for investment advice or any corroboration or refutation of the remark. I only mention the context as it is not like tomorrow's weather (maybe based on today's) or something one can extrapolate.

Thanks

user663837
  • 187
  • 4

3 Answers3

9

It's not an elementary or trivial question. It's a philosophical one. It touches the deepest foundations of statistics. It's almost a religious question if Bayesians overhear it. There are many ways to answer it. For instance, it could mean that if our Universe forked this moment and we observed infinite number of evolutions of these universes then in 70% of cases in next 6 months US stock market will drop below March lows. Surely, this may sound extravagant, but I came to accept it as a possibility.

Another answer is that it's about the subjective gauge that helps you decide whether or not to invest in US stock market. We have only one realization of reality, so 70% of chance has no means of being tested in terms of frequencies of occurrence of this event.

Yet another way to answer is to assume an ergodic hypothesis which states that although we can't fork the Universe and see across all different realizations of future; we can, nevertheless, observe one Universe for long time, and that when you observe it for a long enough period it's the same as observing a set of forked universes. In practical terms it means that if you follow the forecast of whoever gave you the "70% prediction" then in 70% of the cases the advice will pay off. Over long period of time.

Aksakal
  • 55,939
  • 5
  • 90
  • 176
5

When you're talking about a repeatable event, the interpretation is pretty easy. A probability of 70% means that when you observe the event, the prediction should be borne out (in the long run) 7 times out of 10.

When you're talking about a non-repeatable event, then the best interpretation is as a level of confidence in the prediction. The idea is that if you take all the times a forecaster makes a prediction of 70%, about 7/10 of those predictions should be borne out. Thus, you can have an idea, based on a forecaster's track record, of how likely the prediction is to be correct, even though you can't replay the individual events to see empirically how often the counterfactuals happened.

In fact, this is exactly how the Good Judgement Project scored its forecasting tournament. The scoring metric was designed so that being well calibrated was as important as being right most of the time. In other words, if your 70%-forecasts are right 60% of the time, that's bad (you are overconfident), but it's also bad if they're right 80% of the time (you are underconfident). Notwithstanding the rather cynical take offered by some of the other answers, the main finding of GJP was that with suitable training people can be taught to make well-calibrated forecasts.

Nobody
  • 1,142
  • 7
  • 8
  • @Aksakal A non-repeatable event is something where you can't just run the experiment many times to see how often it turns out a certain way. Once you know how the event turned out, it doesn't make sense to ascribe a probability, but prospectively, before you know the result, it absolutely makes sense to talk about it probabilistically. The probability is a statement about our state of knowledge and how much that knowledge allows us to predict the outcome of the event. – Nobody Jun 10 '20 at 19:55
  • Try _Data Analysis: A Bayesian Tutorial_. It's not quite the same (the author talks about why it makes sense to have a probability distribution for a measurable quantity that has a single fixed value), but it's a similar argument. – Nobody Jun 10 '20 at 20:11
  • I understand now that you were talking about experiments, and in economics it's rarely possible to run them, so we deal with *observational* studies and *natural experiments*. However, OP is only asking about forecasts and probabilities associated with them. It's not important how they came up with them in the context of the question, it's only important what they mean for future and forecasts – Aksakal Jun 10 '20 at 20:34
2

First of all, without any kind of demonstration (transparency) this value means nothing.

On the other hand, my guess about this specific statement should mean this:

Prediction Guy: Me and my team are using a model that fits and predicts the stock market (Specifically the Dow Jones, for example). Now, we made the model run several times giving us X possible predictions, from that set of runs, we've seen that in 70% of cases we've seen that stock market has took out march lows in six months.

The thing is, it's impossible to have an specific percentage value of a success about stock market, because these indices are not dices or coins. There's no probability of success A or B. What it's really possible to do is to make predictions based in a model. Nonetheless, any estimation should be followed by a significance level and/or a confidence interval.

My conclusion: What the guy said may be just a guess (he's quite sure that it will happen but he can't be sure, and 70% is an atractive number).

My suggestion: Check (or demand) references about how this value has been extracted.

Dave
  • 203
  • 1
  • 6