2

Say I want to know whether my roommate is baking a pie.

When she bakes, she tends to hum songs.

When she bakes, I tend to smell it happening.

When she bakes, I tend to see her less in the living room.

I am sitting in my armchair observing the smell of pie, hearing humming, and my roommate is not there. Forget the details like conditional probabilities: Broadly, I want to combine these pieces of evidence together to get some kind of unified belief about whether she is baking.

Are my observations independent in the probabilistic sense?

My instinct is "no", because they are all consequences of the same generative process, but then again my sight, smell, and hearing are pretty independent systems. And I could always be hearing someone else, smelling someone else's pie, or my roommate could be missing for some other reason, so it's possible different processes are generating the evidence.

Pavel Komarov
  • 1,097
  • 9
  • 19

2 Answers2

3

That's an interesting question, and one that might illustrate a difficulty inherent in translating statistical models into the real world.

The observations can be independent even though they are all consequences or correlates of a single generative state (baking).

They would fail independence if the probability of you detecting humming was affected by the presence of the smell of baking, or if your ability to smell was influenced by the presence or absence of your roommate.

You example is analogous to the more familiar example of drawing a sample of independent observations from a single population. Those independent observations have in common the population and so they are not independent in that aspect, but they are nonetheless independent from the point of view of the observer. The smell is one observation, humming is another, etc.

Michael Lew
  • 10,995
  • 2
  • 29
  • 47
0

This question is related to the notion of D-Separation in a Bayes Net.

Here we have a variable Baking with arrows/arcs drawn to Smell, Humming, and Roommate Present. The three child variables in the net are D-Separated only if I know the state of the parent variable, Baking. https://www.coursera.org/lecture/probabilistic-graphical-models/conditional-independence-PTXfn, https://www.coursera.org/lecture/probabilistic-graphical-models/independencies-in-bayesian-networks-JRkCU

So if I don't know whether baking is happening, then my child variables are NOT independent, and my observation of one will affect my expectation for the others.

Pavel Komarov
  • 1,097
  • 9
  • 19