7

I and my friends just had a little discussion whether the events are independent or dependent if they have no outcome in common. I thought that they have to be independent. When two events are independent, then $P(A)=P(A\mid B)$. Is the information given in the question enough to establish this?

If you look at it as a Venn diagram, then if there is no overlap between A and B, then they are independent. But my friend objected and said that this depends on the sample space.

So let's say we have two events: $P(A)=6/12$, $P(A\mid B)=2/4$, and $P(B)=4/12$, then obviously $P(A\mid B)$ equals $P(A)$. But for me, something smells fishy here.

I know this is more a stochastic question that a statistic question, but maybe someone can help.

Macro
  • 40,561
  • 8
  • 143
  • 148
Adam
  • 211
  • 2
  • 6
  • What do you mean by "no result in common"? – Peter Flom Mar 19 '12 at 11:21
  • Adam, I made minor edits to your post. Please, check that I didn't alter the meaning of the 3rd paragraph. – chl Mar 19 '12 at 12:07
  • 7
    From your description, it seems like you might be confusing the idea between independent and mutually exclusive. – King Mar 19 '12 at 12:38
  • 1
    if i got it right, no result in common=mutually exclusive. But the question was formulated a little sloppy. So maybe that is what makes the answering of this question a little complicated.It boils down to the fact that mutually exclusive does not say anything about the probabilites occuring. So it can be both.THANKS FOR THE GREAT ANSWERS BTW! – Adam Mar 19 '12 at 17:57
  • @Adam: You got it right. – Neil G Mar 20 '12 at 06:47

6 Answers6

12

No, events with no result in common are not independent if the events come from the same sample space.

An example: Throw a single fair die. Let event A be 'throw is a 1', and event B be 'throw is a 2'. Then $P(A) = P(B) = 1/6$, but $P(A|B) = P(B|A) = 0$, as the die throw can't be both 1 and 2.

Macro
  • 40,561
  • 8
  • 143
  • 148
onestop
  • 16,816
  • 2
  • 53
  • 83
  • 5
    You are correct for nonsingular countable spaces, but it would be good to take a little care with the generality of your first statement. After all, many sample spaces are infinite and many nonempty events have zero probability. They provide counterexamples: such events may have no outcome in common, yet they are independent. – whuber Mar 19 '12 at 17:22
  • 1
    @whuber: maybe you should add a precise answer? – Neil G Mar 20 '12 at 01:14
  • 1
    @whuber - You could increase the generality of the statement by noting that dependence between any two propositions can only occur when both are marginally possible and at least one is not marginally certain. That is $A$ and $B$ can only be dependent if both $p(A)>0$ and $p(B)>0$ and $p(A)+p(B)<2$. This is just the logical result of "a false proposition implies all propositions, true and false" and "a true proposition implies all true propositions". – probabilityislogic Mar 20 '12 at 11:58
10

Independence can be thought of as "If event A occurs it tells you nothing about the probability of event B occurring".

However, if two events share nothing in common then clearly if I know that one event occurred then I know that the other event COULDN'T have occurred. So I am getting information about the other event so there the two events can't be independent.

Note that this was an intuitive explanation and it's possible for B to be empty in which case A and B can be mutually exclusive and still be independent as explained in some of the other answers.

Dason
  • 2,092
  • 20
  • 19
  • That's true as long as neither of the events contains no outcomes. Such an event would be both disjoint and independent of all other events. – Neil G Mar 20 '12 at 01:13
  • That's very true and a good point. I was going for a more intuitive reason why mutually exclusive events aren't independent but you're right and I should note that in my answer. – Dason Mar 20 '12 at 03:58
  • Okay, the first paragraph is good, but the second paragraph isn't right. – Neil G Mar 20 '12 at 06:15
  • @NeilG - the second paragraph is correct, because if $A$ and $B$ are mutually exclusive, then we have the logical truths $A\implies\overline{B}$ and $B\implies\overline{A}$. In probabilistic notation this means $p(\overline{A}|B)=p(\overline{B}|A)=1$. This means that as long as $A$ is not impossible when $B$ is false and/or $B$ is not impossible when when $A$ is false we always have dependence between mutually exclusive events. – probabilityislogic Mar 20 '12 at 11:24
  • 1
    @probabilityislogic: No :) See my answer. – Neil G Mar 20 '12 at 11:49
5

It is true that if $P(A)=P(A\mid B)$ then the two events are intependent.

Now we know that: $P(A\mid B)=\frac{P(A\cap B)}{P(B)}$. Note the intersection in the numerator. If the two events didn't have "anything in common" then we would have $P(A\cap B)=0$ since the set $A\cap B$ would be empty. However independence works in the space of probabilities. Two events are called independent if $P(A\cap B)=P(A)P(B)$, ie the probability of having both events occuring is equal to the probability of the first one occuring times the probability of the second one. The fact that event $A$ occured doesn't tell us anything about event $B$.

If on the other hand $P(A\cap B)=0$ this simply says that the two events cannot happen at the same time: they are disjoint.

Example 1. What is the probability of throwing a die and getting both 1 and 3 at the same time? Since the two events are disjoint, we have that this probability is zero.

Example 2. What is the probability of throwing a die two times and getting 1 in the first time and 2 in the second? The two events are not disjoint; the fact that one happened doesn't exclude the other. However they are independent: the fact that one happened tells us nothing about the other. The probability of getting 1 in the first throw is 1/6, the probability of getting 2 in the second throw is 1/6. Then (abusing notation a bit): $P(2\mid 1)= \frac{P(1 \cap 2)}{P(1)} = \frac{ \frac{1}{36}}{\frac{1}{6}}=\frac{1}{6}=P(2)$.

3

You are confusing dependence with mutual-exclusivity. Two events $A$ and $B$ are independent if and only if $P(A \cap B) = P(A)P(B)$. They are disjoint if and only if $A \cap B = \emptyset$.

Consider a normally-distributed random variable $X$ with mean 0 and variance 1. Here is a pair of events for each of the four categories:

  1. the events $0 < X < 1$ and $1 < X < 2$ are dependent and disjoint
  2. the events $X = 0$ and $1 < X < 2$ are independent and disjoint
  3. the events $0 < X < 1$ and $0 < X < 2$ are dependent and not disjoint, and
  4. the events $X < 0$ and $\lvert X \rvert < 3$ are independent and not disjoint.

If events $A$ and $B$ are independent and disjoint then \begin{align} 0 &= P(A \cap B) \\\\ &= P(A)P(B) \\\\ &\implies P(A) = 0 \vee P(B) = 0. \end{align}

Neil G
  • 13,633
  • 3
  • 41
  • 84
  • I think that this is a poor illustration of the idea because the only cases where independence occurs is where at least one event is an event of probability $0$ or $1$. An event of probability $0$ is independent of all other events, and similarly an event of probability $1$ is independent of all other events. In 4. above, _both_ events are of probability $1$ which is overkill. – Dilip Sarwate Mar 20 '12 at 11:24
  • @DilipSarwate: Good point; edited. – Neil G Mar 20 '12 at 11:58
  • Hm, how concepts can be orthogonal? Their scalar product is zero? – mpiktas Mar 20 '12 at 12:04
  • @NeilG - note that you cannot produce an example for case 2 unless you take at least one marginally impossible event. Your example has $p(X=0)=0$. Similarly, you cannot produce an example for case 4 unless you take two certain events. Also I'm not so sure that case 2 is actually an example, because $p(1 – probabilityislogic Mar 20 '12 at 12:07
  • 1
    @probabilityislogic: You don't need any limits. The standard definition of an event space always includes at least one event with no outcomes because sigma algebras must be closed under intersection. I have added an event for case 4 where neither event is certain. The definition of independence is $P(A)P(B) = P(A \cap B)$. – Neil G Mar 20 '12 at 12:09
  • If you mean that independent non-disjoint events must _both_ have probability $1$, that is incorrect. Indeed, $\{X=5\}$ and $\{0 < X < 10\}$ are independent events for your particular example. More generally (and getting away from your particular example), there are any number of good illustrations of the idea that independent events with probabilities restricted to $(0,1)$ cannot be mutually exclusive. I urge you to revise your answer. As it stands, with statements like "dependence and mutual exclusivity are orthogonal concepts" it is deserving of a down vote. – Dilip Sarwate Mar 20 '12 at 12:17
  • @Dilip: Did you see my edits? However, note that for point 2, there is no way to escape one of the events having probability 0. I will add a proof. – Neil G Mar 20 '12 at 12:19
  • @probabilityislogic: Do you agree that disjoint events can be dependent? The definition of an event includes events that have probability zero. (I would still describe some of these events as uncertain since they can occur.) – Neil G Mar 20 '12 at 13:50
  • My point was that all disjoint events are dependent, except for the trivial case when one of the events is false. – probabilityislogic Mar 20 '12 at 14:01
  • note that probability is always conditional on something, at the very least it is conditional on the support chosen to define the probabilities. And it depends on what you mean by "occur" for $X=0$ for standard normal, as we have described the infinite number of decimal places required to be a "value". you mean $|X|0$ we will still have dependence. – probabilityislogic Mar 20 '12 at 14:14
  • 1
    @probabilityislogic: But the event X=0 is not "false." And, it's not the result of a limit. It's a proper event in the Borel algebra on the reals. – Neil G Mar 20 '12 at 14:19
  • @neilg - the event $X=0$ is false under the assumption of a normal distribution (or any other continuous variable). Note that if we did have a random variable from a normal distribution, we could never truly "observe" it even once, we could only observe that it lied between two limits $a – probabilityislogic Mar 20 '12 at 15:29
  • 1
    @probabilityislogic: Of course you can observe it. Just because its probability is zero, it doesn't mean that it's impossible. http://en.wikipedia.org/wiki/Almost_surely – Neil G Mar 20 '12 at 15:44
  • The problem with defining $X=0$ as an event, is that we are then trying to calculate directly on an infinite set of propositions, which is generally not 100% safe territory for probability calculations (e.g. nothing can happen, but something must happen...). Better to go with CDFs because here the propositions are exhaustive but finite, such as $X\leq b$ and $X>b$. In any event, my comments are not very related to your original answer anymore - still, I've got something to mull over though. – probabilityislogic Mar 20 '12 at 16:34
  • 1
    @probabilityislogic: What you're doing is rederiving the concept of a sigma algebra. What you'll discover is that if you have events $X \le b$ and $X > b$, then you must have events like $X=b$ since the space of events must be closed under countable intersection since we want to be able to ask the probability of a conjunction of events. Above you wrote that "But note that so long as e>0 we will still have dependence"; the problem with this argument is that properties that hold for all terms in an infinite sequence do not necessarily hold at the limit of the sequence. – Neil G Mar 21 '12 at 05:07
  • 1
    @NeilG Yes, your post is greatly improved and I am up-voting it. One minor comment, though. $A$ and $B$ being disjoint implies that $P(A \cap B) = 0$, but the converse is not true, and so your "if and only if" assertion (third sentence of first paragraph) is not correct. I have posted a separate answer too. – Dilip Sarwate Mar 21 '12 at 13:45
2

Let $A$ and $B$ denote two events defined on a sample space $\Omega$. The formal definition of independent events is as follows.

Definition: $A$ and $B$ are said to be (stochastically) mutually independent events if

$$P(A\cap B) = P(A)P(B).$$

It is easily shown that any one of the four relations shown below implies the other three:

$$\begin{align*} P(A\cap B) &= P(A)P(B)\\ P(A^c\cap B) &= P(A^c)P(B)\\ P(A\cap B^c) &= P(A)P(B^c)\\ P(A^c\cap B^c) &= P(A^c)P(B^c) \end{align*}$$ and so if $A$ and $B$ are mutually independent events, then so are $A^c$ and $B$ mutually independent events, as are $A$ and $B^c$, and $A^c$ and $B^c$.

Now, if $P(B) > 0$ so that we can write $P(A \mid B)$ as $P(A\cap B)/P(B)$, then $P(A\mid B)$ equals $P(A)$, and this is often taken as the colloquial meaning (or definition) of independence. $A$ and $B$ are independent events if knowing that $B$ has occurred does not change our estimate of the probability of $A$. Put another way, the posterior probability $P(A\mid B)$ is the same as the prior probability $P(A)$. The asymmetry in the colloquial definition even leads people to say $A$ is independent of $B$ (which can make beginners wonder whether $B$ is independent of $A$ or not), but the formal definition makes it clear that independence is a mutual property: one cannot have $A$ independent of $B$ but $B$ dependent on $A$.


Turning to the OP's question, if $0 < P(A), P(B) < 1$, then mutual independence and mutual exclusion are mutually exclusive properties. If one property holds, the other cannot. Of course, the most commonly encountered case is that neither property holds. Said out loud and clear

  • If $A$ and $B$ are mutually independent events, then they cannot be mutually exclusive events.

  • If $A$ and $B$ are mutually exclusive events, then they cannot be mutually independent events.

In the first case, note that mutual independence implies that $P(A\cap B) = P(A)P(B) > 0$ and so the intersection of $A$ and $B$ has positive probability. In the second case, $P(A\cap B) = 0$ cannot equal $P(A)P(B)$ since neither $P(A)$ nor $P(B)$ is $0$ by assumption and so their product is a positive number.

As a corollary, note that $A$ and $A$ cannot be a pair of mutually independent events. and nor can $A$ and $A^c$ be mutually independent events.


Much of the discussion in the comments has centered on the rare cases when $P(A)$ or $P(B)$ happen to equal $0$ or $1$. First note that since $$P(A \cap \Omega) = P(A) = P(A)P(\Omega)$$ and so $A$ and the certain event $\Omega$ are independent events for all choices of $A$. Similarly, since $$P(A \cap \emptyset) = P(\emptyset) = 0 = P(A)P(\emptyset),$$ $A$ and the impossible event $\emptyset$ are independent events for all choices of $A$. More generally, if $B$ is an event of probability $0$ (not necessarily the impossible event), then since $A\cap B$ is a subset of $B$ and hence also has probability $0$, we can generalize to $P(A \cap B) = 0 = P(A)P(B)$ and so

  • Any event of probability $0$ is independent of all events (including itself and its complement). If $B$ is an event of probability $0$, then $B$ and $B^c$ are independent events that are mutually exclusive.

If $B$ is an event of probability $0$, then $B^c$ is an event of probability $1$. Since $B$ and $A$ are independent events for all choices of $A$, so also are $B^c$ and $A$ independent events for all choices of $A$. Thus, we have

  • Any event of probability $1$ is independent of all events (including itself and its complement). If $A$ is an event of probability $1$, then $A$ and $A^c$ are independent events that are mutually exclusive.

Note that, as @NeilG has pointed out in his answer, if $A$ and $B$ are independent events that are mutually exclusive, then at least one of $A$ and $B$ must be an event of probability $0$.

We also have an anticorollary: $A$ and $A$ are mutually independent events if and only if $P(A)$ equals either $0$ or $1$. $A$ and $A^c$ are mutually independent events if and only if one of $P(A)$ and $P(A^c)$ equals $0$ (and the other equals $1$.)

Dilip Sarwate
  • 41,202
  • 4
  • 94
  • 200
-1

mutually exclusive events.. natural blonde hair, and black skin. If I know someone has black skin, I know they will not have naturally blonde hair. Therefore, the characteristic of hair color and skin color are dependent.. Knowing that someone has black skin tells me information about what their hair will be like.. Their hair will NOT be blonde.. These things are dependent. Mutually exclusive implies dependence!

independent events... IQ and shoe size.. If you thought someone was a size 8 shoe, would it change your opinion about their IQ if I told you their shoe size was actually a 10? No.. Shoe size and IQ (likely) have no relationship. (Assume it's true.) Shoe size and IQ are independent of one another.

I hope that helps.

Adam
  • 813
  • 10
  • 16