2

(Sorry if this is answered elsewhere, but I can't seem to find.)

Something about Bayesian updates that has always confused me - does the "current" posterior n-size matter? And if so, why not?

For instance, according to this (page 3), we have

$$ \tilde\pi_{n+1}(\theta) \propto \pi_{n}(\theta)f(x_{n+1}|\boldsymbol{x_n},\theta) $$

Seems to me, $n$ could be 10 or 10,000, and the "impact" of $x_{n+1}$ on $\pi$ would be the same. Shouldn't we give the "existing posterior" greater weight if there are more "existing" observations?

I'm sure I'm missing something, but can't seem to put my finger on it.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
dashnick
  • 159
  • 8
  • What do you mean by *current" posterior n-size*? Care to clarify? – kjetil b halvorsen Dec 11 '20 at 15:40
  • 1
    If $n$ is larger, $\pi_n(\theta)$ is likely to be a narrower distribution, which may correspond to your idea of "greater weight" – Henry Dec 11 '20 at 15:43
  • I mean you have seen n observations ("current") and want to update based on the n+1 observation. Where in the math does it account for the magnitude of n? – dashnick Dec 11 '20 at 15:43
  • @Henry is that true? Why does size of n have to do with the shape of the distribution? – dashnick Dec 11 '20 at 15:44
  • @dashnick Try an example yourself, for example a Bernoulli (or binomial) likelihood and a Beta prior and posterior distribution – Henry Dec 11 '20 at 15:50
  • @Henry makes sense.. so it's sort of implicitly embedded – dashnick Dec 11 '20 at 15:53

1 Answers1

0

The point of Bayesian sequential updating is that no mater if you update prior with likelihood over all of your data at once, or if you do this sequentially, the result would be the same. This wouldn't hold if different observations had different weights. If for some reason you need to assign different weights to different samples (e.g. survey weights), that this would work the same in sequential and all-at-once updating. However if you don't have good reason for doing so, usually you do not want different samples to affect the result to different degree.

Tim
  • 108,699
  • 20
  • 212
  • 390