Probability

STAT1003 – Statistical Techniques

Dr. Emi Tanaka

Australian National University

These slides are best viewed on a modern browser like Google Chrome on a desktop or laptop. Some interactive components may require some time to fully load.


Acknowledgement

This lecture was partiallyadapted from the previous STAT1003 lecturers. Thank you folks!

Basic Probability Concepts

A random experiment

  • A random (or stochastic) experiment is a process for which the outcome cannot be predicted with certainty.
  • In constrast, an experiment with a pre-determined outcome is a deterministic process.
  • The word trial is used to refer to a single performance of a random experiment.

Is the experiment random or deterministic?

  1. Observe the face of the die on top when rolling a die ,
  2. Observe the number of emails you receive in a day ,
  3. Observe the maximum temperature tomorrow ,
  4. Observe the sun rising tomorrow .

Sample space

The sample space, denoted by \(S\) (or \(\Omega\)), of a random experiment is a set of all the possible outcomes.

  • For example, suppose we observe the number when rolling a fair six-sided die, then the possible outcomes are \(S = \{1, 2, 3, 4, 5, 6\}\).
  • The outcomes in a sample space must be mutually exclusive: no two outcomes can both occur at the same time on any single trial.
  • What is the sample space of rolling two dice?
  • What is the sample space of flipping ten coins?

Probability of an event

An event, denoted as say \(E\), is a set or collection of outcomes in the sample space, \(E \subseteq S\), including the possibility of no outcomes, denoted as \(\emptyset\).

The probability of an event \(E\), denoted as \(P(E)\), is a number between \(0\) and \(1\) (inclusive) that describes the likelihood of the event occurring.

  • \(P(S) = 1\).
  • \(P(E)\) is equal to the sum of the probabilities of the outcomes that make up \(E\).
  • For equally likely outcomes with finite sample space: \[P(E)=\dfrac{\text { number of outcomes in } E}{\text { number of outcomes in } S} = \dfrac{|E|}{|S|}.\]
  • Suppose we roll a fair die and observe the number on top.
  • Let \(E_1\) be the event that the number is even.
  • Then \(E_1= \{2, 4, 6\}\). So \[\begin{align*} P(E_1) &= P(\{2\}) + P(\{4\}) + P(\{6\}) \\ &= \frac{1}{6} + \frac{1}{6} + \frac{1}{6}= \frac{3}{6} = \frac{1}{2} \\ \end{align*}\]
  • Let \(E_2\) be the event that the number is greater than 7.
  • Then \(E_2 = \emptyset\) and \(P(E_2) = 0\).

Challenge Rolling a die

Suppose that we roll a fair die. What is the chance of:

  1. getting 1 when rolling a die?
  2. getting a 1 or 2 in the next roll?
  3. getting either 1, 2, 3, 4, 5, or 6 on the next roll?
  4. not rolling a 2?

Simulating random experiments in R

For finite outcomes with known probabilities, we can simulate random events in R using the sample() function.

  • We use set.seed() for reproduciblity (the input can be anything).

Law of large numbers

The law of large numbers states that as a random experiment is repeated many times, the sample average (or relative frequency) gets closer to the true expected value (or probability).

  • What is the probability of getting a 1 when rolling a fair die? Note \(\frac{1}{6} \approx 0.167\).
  • Alternatively, you can use the replicate() function to repeat the experiment a number of times and calculate the average.

Combining events

  • Suppose we roll a conventional die and let \(X\) be the number on top.
  • Now suppose we have two events:
    • \(A\): \(X\) is an even number and
    • \(B\): \(X > 3\).

The intersection of \(A\) and \(B\), denoted \(A \cap B\), is the event that happens when both \(A\) and \(B\) occur.

The union of \(A\) and \(B\), denoted \(A \cup B\), is the event that happens when either \(A\), \(B\) or both occur.

Venn diagram

  • \(A \cap B = \{4, 6\}\), thus \(P(A \cap B) = \dfrac{2}{6} = \dfrac{1}{3}\).
  • \(A \cup B = \{2, 4, 5, 6\}\), thus \(P(A \cup B) = \dfrac{4}{6} = \dfrac{2}{3}\).

Addition rule for two events

For two events \(A\) and \(B\), the addition rule states that the probability of the union event is \[P(A\cup B) = P(A) + P(B) - P(A\cap B)\]

Venn diagram

Example: drawing a card

There are 52 cards in a normal deck of cards.

2♣ 3♣ 4♣ 5♣ 6♣ 7♣ 8♣ 9♣ 10♣ J♣ Q♣ K♣ A♣
 2♦ 3♦ 4♦ 5♦ 6♦ 7♦ 8♦ 9♦ 10♦ J♦ Q♦ K♦ A♦
 2♥ 3♥ 4♥ 5♥ 6♥ 7♥ 8♥ 9♥ 10♥ J♥ Q♥ K♥ A♥
 2♠ 3♠ 4♠ 5♠ 6♠ 7♠ 8♠ 9♠ 10♠ J♠ Q♠ K♠ A♠

Consider a regular deck of card and let events:

  • \(A\): a randomly selected card is a diamond.
  • \(B\): a randomly selected card is a face card.

What is \(P(A \cup B)?\)

Solution

  • The intersection event \(A\cap B\): diamond and face card, which gives only three cards: \(J\diamondsuit , Q\diamondsuit, K\diamondsuit\).
  • By addition rule, we have

\[\begin{align*} P(A\cup B) &= P(A) + P(B) - P(A\cap B) \\ &= P(\diamondsuit) + P(J\text{ or }Q\text{ or }K) - P(J\diamondsuit\text{ or }Q\diamondsuit\text{ or }K\diamondsuit)\\ &= \dfrac{13}{52} + \dfrac{12}{52} - \dfrac{3}{52}\\ &= \dfrac{11}{26} \end{align*}\]

Addition rule for three events

For three events \(A\), \(B\) and \(C\), the addition rule states that the probability of the union event is

\[P(A\cup B \cup C) = P(A) + P(B) + P(C) - P(A\cap B) - P(A\cap C) - P(B\cap C) + P(A\cap B \cap C)\]

Addition rule for disjoint events

  • Two events \(A\) and \(B\) are disjoint or mutually exclusive if they cannot both happen or \(A \cap B = \emptyset\).
  • Note \(P(A \cap B) = P(\emptyset) = 0\).

For two disjoint events \(A\) and \(B\) the addition rule simplifies to \[P(A\cup B) = P(A) + P(B)\]

Suppose the events:

  • \(A\): a dice roll is even, and
  • \(C\): a dice roll is 5.

Then, \[P(A\cup C) = P(A) + P(C) = \frac{1}{2} + \frac{1}{6} = \frac{4}{6} = \frac{2}{3}.\]

Complement rule

The complement of \(A\), denoted \(A^c\), is the event where \(A\) does not occur. Note: \(A \cup A^c = S\).

For an event \(A\), the complement rule states that

\[P(A) = 1-P(A^c)\]

  • If event \(A\) is rolling an even number on a die, then \(A^c\) is rolling an odd number.
  • Thus, \(P(A) = 1 - P(A^c) = 1 - \dfrac{3}{6} = \dfrac{3}{6} = \dfrac{1}{2}\).

Algebra of sets

You can use Venn diagrams to confirm the following properties of sets:

  • Commutative property:
    • \(A\cup B = B\cup A\)
    • \(A\cap B = B\cap A\)
  • Associative property:
    • \((A \cup B) \cup C=A \cup(B \cup C)\)
    • \((A \cap B) \cap C=A \cap(B \cap C)\)
  • Distributive property:
    • \(A \cup(B \cap C)=(A \cup B) \cap(A \cup C)\)
    • \(A \cap(B \cup C)=(A \cap B) \cup(A \cap C)\)
  • De Morgan’s laws:
    • \((A \cup B)^c=A^c \cap B^c\)
    • \((A \cap B)^c=A^c \cup B^c\)
  • Double complement:
    • \(\left(A^c\right)^c=A\)

Summary

  • Sample space, \(S\), is the set of all possible outcomes of a random experiment.
  • An event, \(E\), is a set of outcomes in the sample space.
  • The probability of an event is a number between 0 and 1 that describes the likelihood of the event occurring.
  • If event \(E = \{O_1, O_2, \ldots\}\), then \(P(E) = P(O_1) + P(O_2) + \cdots\).
  • Law of large numbers: as a random experiment is repeated many times, the sample average gets closer to the true expected value.
  • Addtion rule: \(P(A\cup B) = P(A) + P(B) - P(A\cap B)\).
  • Complement rule: \(P(A) = 1 - P(A^c)\)
  • Venn diagrams are useful for visualising events and their relationships.

Combinatorics

Challenge Rolling two dice

If we roll two fair dice, what is the probability that the sum of the two numbers is 5?

Challenge Rolling ten dice

What is the probability that the sum of the numbers when rolling ten fair dice is 30?

  • To answer this question, there are a number of different approaches that can be taken:
    1. List all possible outcomes and count those that sum to 30. But there \(6^{10} = 60,466,176\) possible outcomes!
    2. Use combinatorial methods to count the number of outcomes that sum to 30.
    3. Use simulation to estimate the probability.
1 1 1 1 1 1 6 6 6 6 
1 1 1 1 1 2 5 6 6 6 
1 1 1 1 1 3 4 6 6 6 
1 1 1 1 1 3 5 5 6 6 
1 1 1 1 1 4 4 5 6 6 
1 1 1 1 1 4 5 5 5 6 
1 1 1 1 1 5 5 5 5 5 
1 1 1 1 2 2 4 6 6 6 
1 1 1 1 2 2 5 5 6 6 
1 1 1 1 2 3 3 6 6 6 
1 1 1 1 2 3 4 5 6 6 
1 1 1 1 2 3 5 5 5 6 
1 1 1 1 2 4 4 4 6 6 
1 1 1 1 2 4 4 5 5 6 
1 1 1 1 2 4 5 5 5 5 
1 1 1 1 3 3 3 5 6 6 
1 1 1 1 3 3 4 4 6 6 
1 1 1 1 3 3 4 5 5 6 
1 1 1 1 3 3 5 5 5 5 
1 1 1 1 3 4 4 4 5 6 
1 1 1 1 3 4 4 5 5 5 
1 1 1 1 4 4 4 4 4 6 
1 1 1 1 4 4 4 4 5 5 
1 1 1 2 2 2 3 6 6 6 
1 1 1 2 2 2 4 5 6 6 
1 1 1 2 2 2 5 5 5 6 
1 1 1 2 2 3 3 5 6 6 
1 1 1 2 2 3 4 4 6 6 
1 1 1 2 2 3 4 5 5 6 
1 1 1 2 2 3 5 5 5 5 
1 1 1 2 2 4 4 4 5 6 
1 1 1 2 2 4 4 5 5 5 
1 1 1 2 3 3 3 4 6 6 
1 1 1 2 3 3 3 5 5 6 
1 1 1 2 3 3 4 4 5 6 
1 1 1 2 3 3 4 5 5 5 
1 1 1 2 3 4 4 4 4 6 
1 1 1 2 3 4 4 4 5 5 
1 1 1 2 4 4 4 4 4 5 
1 1 1 3 3 3 3 3 6 6 
1 1 1 3 3 3 3 4 5 6 
1 1 1 3 3 3 3 5 5 5 
1 1 1 3 3 3 4 4 4 6 
1 1 1 3 3 3 4 4 5 5 
1 1 1 3 3 4 4 4 4 5 
1 1 1 3 4 4 4 4 4 4 
1 1 2 2 2 2 2 6 6 6 
1 1 2 2 2 2 3 5 6 6 
1 1 2 2 2 2 4 4 6 6 
1 1 2 2 2 2 4 5 5 6 
1 1 2 2 2 2 5 5 5 5 
1 1 2 2 2 3 3 4 6 6 
1 1 2 2 2 3 3 5 5 6 
1 1 2 2 2 3 4 4 5 6 
1 1 2 2 2 3 4 5 5 5 
1 1 2 2 2 4 4 4 4 6 
1 1 2 2 2 4 4 4 5 5 
1 1 2 2 3 3 3 3 6 6 
1 1 2 2 3 3 3 4 5 6 
1 1 2 2 3 3 3 5 5 5 
1 1 2 2 3 3 4 4 4 6 
1 1 2 2 3 3 4 4 5 5 
1 1 2 2 3 4 4 4 4 5 
1 1 2 2 4 4 4 4 4 4 
1 1 2 3 3 3 3 3 5 6 
1 1 2 3 3 3 3 4 4 6 
1 1 2 3 3 3 3 4 5 5 
1 1 2 3 3 3 4 4 4 5 
1 1 2 3 3 4 4 4 4 4 
1 1 3 3 3 3 3 3 4 6 
1 1 3 3 3 3 3 3 5 5 
1 1 3 3 3 3 3 4 4 5 
1 1 3 3 3 3 4 4 4 4 
1 2 2 2 2 2 2 5 6 6 
1 2 2 2 2 2 3 4 6 6 
1 2 2 2 2 2 3 5 5 6 
1 2 2 2 2 2 4 4 5 6 
1 2 2 2 2 2 4 5 5 5 
1 2 2 2 2 3 3 3 6 6 
1 2 2 2 2 3 3 4 5 6 
1 2 2 2 2 3 3 5 5 5 
1 2 2 2 2 3 4 4 4 6 
1 2 2 2 2 3 4 4 5 5 
1 2 2 2 2 4 4 4 4 5 
1 2 2 2 3 3 3 3 5 6 
1 2 2 2 3 3 3 4 4 6 
1 2 2 2 3 3 3 4 5 5 
1 2 2 2 3 3 4 4 4 5 
1 2 2 2 3 4 4 4 4 4 
1 2 2 3 3 3 3 3 4 6 
1 2 2 3 3 3 3 3 5 5 
1 2 2 3 3 3 3 4 4 5 
1 2 2 3 3 3 4 4 4 4 
1 2 3 3 3 3 3 3 3 6 
1 2 3 3 3 3 3 3 4 5 
1 2 3 3 3 3 3 4 4 4 
1 3 3 3 3 3 3 3 3 5 
1 3 3 3 3 3 3 3 4 4 
2 2 2 2 2 2 2 4 6 6 
2 2 2 2 2 2 2 5 5 6 
2 2 2 2 2 2 3 3 6 6 
2 2 2 2 2 2 3 4 5 6 
2 2 2 2 2 2 3 5 5 5 
2 2 2 2 2 2 4 4 4 6 
2 2 2 2 2 2 4 4 5 5 
2 2 2 2 2 3 3 3 5 6 
2 2 2 2 2 3 3 4 4 6 
2 2 2 2 2 3 3 4 5 5 
2 2 2 2 2 3 4 4 4 5 
2 2 2 2 2 4 4 4 4 4 
2 2 2 2 3 3 3 3 4 6 
2 2 2 2 3 3 3 3 5 5 
2 2 2 2 3 3 3 4 4 5 
2 2 2 2 3 3 4 4 4 4 
2 2 2 3 3 3 3 3 3 6 
2 2 2 3 3 3 3 3 4 5 
2 2 2 3 3 3 3 4 4 4 
2 2 3 3 3 3 3 3 3 5 
2 2 3 3 3 3 3 3 4 4 
2 3 3 3 3 3 3 3 3 4 
3 3 3 3 3 3 3 3 3 3 
Total combinations (without taking into account the order): 121

How long would it take to list all possible outcomes?

Brute force analytical solution

What is the probability that the sum of the numbers when rolling ten fair dice is 30?

Factorial

The factorial of a non–negative integer \(n\), denoted by \(n!\), is the product of all positive integers less than or equal to \(n\).

  • E.g \(5! = 5\times 4\times 3\times 2\times 1 = 120\).
  • Note: we define \(0!=1\) by convention.
  • In R, there are several ways to compute factorials:

Combination

A combination is a selection of objects from a collection where the order of selection does not matter. The number of combinations of \(r\) objects from a collection of \(n\) objects is given by the formula:

\[{n \choose r}=\frac{n !}{(n-r) ! r !}.\]

  • This is read as “\(n\) choose \(r\)”.
  • E.g. choosing a committee of 3 people from 10 candidates: \[{10 \choose 3} = \dfrac{10!}{(10-3)!3!} = 120\] combinations.

Permutation

A permutation is an arrangement of items from a collection where the order matters. The number of permutations of \(r\) objects from a collection of \(n\) objects is given as:

\[(n)_r = \frac{n!}{(n-r)!}\]

  • E.g. arranging 3 books from a shelf of 5 books: \[(5)_3 = \dfrac{5!}{(5-3)!} = 60\] permutations.

Challenge

A password is created by selecting 3 unique letters from the word “PASSWORD” and arranging them in any order. How many possible passwords are there?

A committee of 5 people is selected from a group of 8 men and 6 women. What is the probability that the committee contains exactly 3 women?

How many ways can 8 people sit around a circular table?

Summary

  • Factorials are used to count the number of ways to arrange objects.
  • Combinations count the number of ways to choose objects when order does not matter.
  • Permutations count the number of ways to arrange objects when order matters.

Conditional Probability

SMS spam classification

  • A large language model (LLM) was used to classify if a SMS message is spam or ham (legitimate).

Example spam SMS

Last chance 2 claim ur £150 worth of discount vouchers-Text YES to 85023 now!SavaMob-member offers mobile T Cs 08717898035. £3.00 Sub. 16 . Remove txt X or STOP

Example SMS (not spam)

I dun believe u. I thk u told him.

  • The results are summarized in the contingency table (also referred to as confusion matrix in this case) below.
Prediction
Truth
Total
ham spam
ham 2,280 368 2,648
spam 1,843 282 2,125
Total 4,123 650 4,773
  • What the probability that if I select a random SMS message that LLM classified it as spam?

Relative frequency table

  • Relative to what?


Table 1
Prediction
Truth
Total
ham spam
ham 0.861 0.139 1.000
spam 0.867 0.133 1.000
Table 2
Prediction
Truth
ham spam
ham 0.553 0.566
spam 0.447 0.434
Total 1.000 1.000
Table 3
Prediction
Truth
Total
ham spam
ham 0.478 0.077 0.555
spam 0.386 0.059 0.445
Total 0.864 0.136 1.000

Joint distribution table

A joint probability of events \(A\) and \(B\) is the probability that both events occur together, denoted by \(P(A \cap B)\).

  • Suppose that we randomly select a SMS message from the dataset.
  • What are the probabilities:
    • \(P(\text{SMS is predicted as ham} \cap \text{SMS is a ham})\)
    • \(P(\text{SMS is predicted as ham} \cap \text{SMS is a spam})\)
    • \(P(\text{SMS is predicted as spam} \cap \text{SMS is a ham})\)
    • \(P(\text{SMS is predicted as spam} \cap \text{SMS is a spam})\)
Prediction
Truth
Total
ham spam
ham 0.478 0.077 0.555
spam 0.386 0.059 0.445
Total 0.864 0.136 1.000

Marginal probability

A marginal probability is the probability of a single event occurring, regardless of the outcomes of other variables.

  • Marginal probabilities are calculated by either:
    • adding across the rows of the table, or
    • adding down the columns of the table.
Prediction
Truth
Total
ham spam
ham 0.478 0.077 0.555
spam 0.386 0.059 0.445
Total 0.864 0.136 1.000

\[P(\underbrace{\text{predicted as ham}}_{\large A})=P(A \cap \underbrace{\text{is a ham}}_{\large B})+P(A \cap \underbrace{\text{is a spam}}_{\large B^c})\]

  • Note here \(B \cup B^c = S\) (sample space).

Law of total probability

Suppose that the events \(B_{1}, B_{2}, \ldots, B_{n}\) are a partition of the sample space. That is:

  • \(B_{1}, B_{2}, \ldots, B_{n}\) are mutually exclusive
  • \(B_{1} \cup B_{2} \cup \ldots \cup B_{n}=S\).

Then for any event \(A\), the following is true: \[P(A)=\sum_{i=1}^{n} P\left(A \cap B_{i}\right).\]

This is referred to as the law of total probability.

Conditional probability

For two events \(A\) and \(B\), the conditional probability of \(A\) given that \(B\) has occurred, denoted by \(P(A \mid B)\), is defined to be:

\[P(A \mid B)=\frac{P(A \cap B)}{P(B)}\]

\[P(\text{Truth} \mid \text{Prediction})\]

Prediction
Truth
Total
ham spam
ham 0.861 0.139 1.000
spam 0.867 0.133 1.000

\[P(\text{Prediction} \mid \text{Truth})\]

Prediction
Truth
ham spam
ham 0.553 0.566
spam 0.447 0.434
Total 1.000 1.000

Note that \(P(A \mid B) \ne P(B \mid A)\) in general.

Challenge

If a SMS is predicted to be spam, what is the probability that the prediction is correct?

You are rolling a die. You are told you rolled an even number.

  • What is the probability that you rolled a 2?
  • What is the probability that you rolled a big number (4,5,6)?
Prediction
Truth
Total
ham spam
ham 0.478 0.077 0.555
spam 0.386 0.059 0.445
Total 0.864 0.136 1.000

Solution:

  • \(P(\text{is a spam} \mid \text{predicted as spam}) = \dfrac{P(\text{is a spam} \cap \text{predicted as spam})}{P(\text{predicted as spam})} = \dfrac{0.059}{0.445} \approx 0.133.\)
  • \(P(\text{is a 2} \mid \text{is even}) = \dfrac{P(\text{is a 2} \cap \text{is even})}{P(\text{is even})} = \dfrac{\frac{1}{6}}{\frac{1}{2}} = \dfrac{1}{3}.\)
  • \(P(\text{is a big number} \mid \text{is even}) = \dfrac{P(\text{is a big number} \cap \text{is even})}{P(\text{is even})} = \dfrac{\frac{2}{6}}{\frac{1}{2}} = \dfrac{2}{3}.\)

Independent events

Two events \(A\) and \(B\) are independent if and only if:

\[P(A \cap B)=P(A) \times P(B).\]

  • Notice if \(A\) and \(B\) are independent: \[\begin{equation*} \begin{array}{l} P(A \mid B)=\frac{P(A \cap B)}{P(B)}=\frac{P(A) \times P(B)}{P(B)}=P(A) \\ P(B \mid A)=\frac{P(B \cap A)}{P(A)}=\frac{P(B) \times P(A)}{P(A)}=P(B) \end{array} \end{equation*}\]

  • That is, two events are independent if the probability of one event occurring is not affected by the occurrence of the other event.

Challenge

Consider rolling two dice, what is the probability of getting two 1s?

If we shuffle up a deck of cards and draw one, is the event that the card is a heart independent of the event that the card is an ace?

A streaming service reports that 25% of its users watch documentaries and 60% of its users watch movies. If 15% of users watch both documentaries and movies, are the events “a user watches documentaries” and “a user watches movies” independent?

Multiplication rule

  • Using the definition of conditional probability we have \[P(A \mid B)=\frac{P(A \cap B)}{P(B)}\]

For two events \(A\) and \(B\), the multiplication rule states that \[\begin{align*} P(A\cap B) &= P(A\mid B)\times P(B)\\ &= P(B\mid A)\times P(A) \end{align*}\]

  • When \(A\) and \(B\) are independent, the multiplication rule is simply \[P(A\cap B) = P(A) \times P(B).\]

Smallpox in Boston, 1721

  • The smallpox data set provides a sample of 6,224 individuals from the year 1721 who were exposed to smallpox in Boston.
  • Doctors at the time believed that inoculation, which involve exposing a person to the disease in a controlled form, could reduce the likelihood of death.
  • The counts of result (died or lived) from inoculated (yes and no) are shown in the contingency table below.
Result
Inoculated
Total
no yes
died 844 6 850
lived 5136 238 5374
Total 5980 244 6224

What is the probability that a randomly selected person who was not inoculated died from smallpox?

\[P(\text{died}\mid \text{not inoculated}) = \frac{P(\text{died}\cap \text{not inoculated})}{P(\text{not inoculated})} = \frac{844/6224}{5980/6224} = 0.1411.\]

What is the probability that an inoculated person died from smallpox?

\[P(\text{died}\mid \text{inoculated}) = \frac{P(\text{died}\cap \text{inoculated})}{P(\text{inoculated})} = \frac{6/6224}{244/6224} = 0.0246.\]

Sum of conditional probabilities

Let \(A_{1}, \ldots, A_{n}\) represent a set of disjoint and exhaustive events, i.e.,

  • \(A_{1} \cup A_{2} \cup \ldots \cup A_{n}=S\), and
  • \(A_i \cap A_j = \emptyset\) for \(i\neq j\).

Then for event \(B\),

\[ P\left(A_{1} \mid B\right)+\cdots+P\left(A_{n} \mid B\right) = 1 \]

  • One special case: \[ P(A \mid B)=1-P\left(A^{c} \mid B\right) \]

Bayes’ theorem

For two events \(A\) and \(B\), the Bayes’ theorem states \[P(A\mid B) = \frac{P(B|A)P(A)}{P(B)}.\]

Heroin test example

  • Joe is a randomly chosen member of a large population in which 3% are heroin users.
  • Joe tests positive for heroin in a drug test that correctly identifies users 95% of the time and correctly identifies non-users 90% of the time.
  • Determine the probability that Joe uses heroin given the positive test result.
  • First write what we want in probabilities:
    • \(P(\text{User}) = 0.03\)
    • \(P(\text{Positive Test}\mid \text{User}) = 0.95\)
    • \(P(\text{Negative Test}\mid \text{User}^c) = 0.9\)
    • \(P(\text{User} \mid \text{Positive Test})\)
  • By Bayes’ theorem, \(P(\text{User}\mid \text{Positive Test})\)

\[\begin{align*} &= \frac{P(\text{User}\cap \text{Positive Test})}{P(\text{Positive Test})}\\ &= \frac{P(\text{Positive Test}\mid \text{User})P(\text{User})}{P(\text{Positive Test}\mid \text{User})P(\text{User}) + P(\text{Positive Test}\mid \text{User}^c)P(\text{User}^c)}\\ &= \frac{0.95\times 0.03}{0.95\times 0.03+(1-0.9)\times (1-0.03)}\\ &= 0.2271 \end{align*}\]

Summary

  • Joint probability, \(P(A\cap B)\), describes the probability of two events, \(A\) and \(B\), occurring together.
  • Marginal probability, \(P(A)\), describes the probability of \(A\) occurring, regardless of \(B\).
  • Conditional probability, \(P(A|B) = \dfrac{P(A\cap B)}{P(B)}\) describes the probability of an event occurring given that another event has occurred.
  • Events \(A\) and \(B\) are independent if and only if \(P(A\cap B) = P(A)P(B)\).
  • Law of total probability: if \(B_1, B_2, \ldots, B_n\) are a partition of the sample space, then \(P(A) = \sum_{i=1}^n P(A\cap B_i)\).
  • Bayes’ theorem: \[P(A\mid B) = \frac{P(B|A)P(A)}{P(B)}\]