Basic probability concepts

STAT1003 – Statistical Techniques

Dr. Emi Tanaka

Australian National University

These slides are best viewed on a modern browser like Google Chrome on a desktop or laptop. Some interactive components may require some time to fully load.

Table of contents

• A random experiment
• Sample space
• Probability of an event
• Challenge Rolling a die
• Simulating random experiments in R
• Law of large numbers
• Combining events
• Addition rule for two events
• Example: drawing a card
• Addition rule for three events
• Addition rule for disjoint events
• Complement rule
• Algebra of sets
• Summary

A random experiment

• A random (or stochastic) experiment is a process for which the outcome cannot be predicted with certainty.
• In constrast, an experiment with a pre-determined outcome is a deterministic process.
• The word trial is used to refer to a single performance of a random experiment.

Is the experiment random or deterministic?

1. Observe the face of the die on top when rolling a die ,
2. Observe the number of emails you receive in a day ,
3. Observe the maximum temperature tomorrow ,
4. Observe the sun rising tomorrow .

Sample space

The sample space, denoted by \(S\) (or \(\Omega\)), of a random experiment is a set of all the possible outcomes.

• For example, suppose we observe the number when rolling a fair six-sided die, then the possible outcomes are \(S = \{1, 2, 3, 4, 5, 6\}\).
• The outcomes in a sample space must be mutually exclusive: no two outcomes can both occur at the same time on any single trial.

• What is the sample space of rolling two dice?
• What is the sample space of flipping ten coins?

Probability of an event

An event, denoted as say \(E\), is a set or collection of outcomes in the sample space, \(E \subseteq S\), including the possibility of no outcomes, denoted as \(\emptyset\).

The probability of an event \(E\), denoted as \(P(E)\), is a number between \(0\) and \(1\) (inclusive) that describes the likelihood of the event occurring.

• \(P(S) = 1\).
• \(P(E)\) is equal to the sum of the probabilities of the outcomes that make up \(E\).
• For equally likely outcomes with finite sample space: \[P(E)=\dfrac{\text { number of outcomes in } E}{\text { number of outcomes in } S} = \dfrac{|E|}{|S|}.\]

• Suppose we roll a fair die and observe the number on top.
• Let \(E_1\) be the event that the number is even.
• Then \(E_1= \{2, 4, 6\}\). So \[\begin{align*} P(E_1) &= P(\{2\}) + P(\{4\}) + P(\{6\}) \\ &= \frac{1}{6} + \frac{1}{6} + \frac{1}{6}= \frac{3}{6} = \frac{1}{2} \\ \end{align*}\]

• Let \(E_2\) be the event that the number is greater than 7.
• Then \(E_2 = \emptyset\) and \(P(E_2) = 0\).

Challenge Rolling a die

Suppose that we roll a fair die. What is the chance of:

1. getting 1 when rolling a die?
2. getting a 1 or 2 in the next roll?
3. getting either 1, 2, 3, 4, 5, or 6 on the next roll?
4. not rolling a 2?

Simulating random experiments in R

For finite outcomes with known probabilities, we can simulate random events in R using the sample() function.

• We use set.seed() for reproduciblity (the input can be anything).

Law of large numbers

The law of large numbers states that as a random experiment is repeated many times, the sample average (or relative frequency) gets closer to the true expected value (or probability).

• What is the probability of getting a 1 when rolling a fair die? Note \(\frac{1}{6} \approx 0.167\).

• Alternatively, you can use the replicate() function to repeat the experiment a number of times and calculate the average.

Combining events

• Suppose we roll a conventional die and let \(X\) be the number on top.
• Now suppose we have two events:
  • \(A\): \(X\) is an even number and
  • \(B\): \(X > 3\).

The intersection of \(A\) and \(B\), denoted \(A \cap B\), is the event that happens when both \(A\) and \(B\) occur.

The union of \(A\) and \(B\), denoted \(A \cup B\), is the event that happens when either \(A\), \(B\) or both occur.

Venn diagram

• \(A \cap B = \{4, 6\}\), thus \(P(A \cap B) = \dfrac{2}{6} = \dfrac{1}{3}\).
• \(A \cup B = \{2, 4, 5, 6\}\), thus \(P(A \cup B) = \dfrac{4}{6} = \dfrac{2}{3}\).

Addition rule for two events

For two events \(A\) and \(B\), the addition rule states that the probability of the union event is \[P(A\cup B) = P(A) + P(B) - P(A\cap B)\]

Venn diagram

Example: drawing a card

There are 52 cards in a normal deck of cards.

2♣ 3♣ 4♣ 5♣ 6♣ 7♣ 8♣ 9♣ 10♣ J♣ Q♣ K♣ A♣
2♦ 3♦ 4♦ 5♦ 6♦ 7♦ 8♦ 9♦ 10♦ J♦ Q♦ K♦ A♦
2♥ 3♥ 4♥ 5♥ 6♥ 7♥ 8♥ 9♥ 10♥ J♥ Q♥ K♥ A♥
2♠ 3♠ 4♠ 5♠ 6♠ 7♠ 8♠ 9♠ 10♠ J♠ Q♠ K♠ A♠

Consider a regular deck of card and let events:

• \(A\): a randomly selected card is a diamond.
• \(B\): a randomly selected card is a face card.

What is \(P(A \cup B)?\)

Solution

• The intersection event \(A\cap B\): diamond and face card, which gives only three cards: \(J\diamondsuit , Q\diamondsuit, K\diamondsuit\).
• By addition rule, we have

\[\begin{align*} P(A\cup B) &= P(A) + P(B) - P(A\cap B) \\ &= P(\diamondsuit) + P(J\text{ or }Q\text{ or }K) - P(J\diamondsuit\text{ or }Q\diamondsuit\text{ or }K\diamondsuit)\\ &= \dfrac{13}{52} + \dfrac{12}{52} - \dfrac{3}{52}\\ &= \dfrac{11}{26} \end{align*}\]

Addition rule for three events

For three events \(A\), \(B\) and \(C\), the addition rule states that the probability of the union event is

\[P(A\cup B \cup C) = P(A) + P(B) + P(C) - P(A\cap B) - P(A\cap C) - P(B\cap C) + P(A\cap B \cap C)\]

Addition rule for disjoint events

• Two events \(A\) and \(B\) are disjoint or mutually exclusive if they cannot both happen or \(A \cap B = \emptyset\).
• Note \(P(A \cap B) = P(\emptyset) = 0\).

For two disjoint events \(A\) and \(B\) the addition rule simplifies to \[P(A\cup B) = P(A) + P(B)\]

Suppose the events:

• \(A\): a dice roll is even, and
• \(C\): a dice roll is 5.

Then, \[P(A\cup C) = P(A) + P(C) = \frac{1}{2} + \frac{1}{6} = \frac{4}{6} = \frac{2}{3}.\]

Complement rule

The complement of \(A\), denoted \(A^c\), is the event where \(A\) does not occur. Note: \(A \cup A^c = S\).

For an event \(A\), the complement rule states that

\[P(A) = 1-P(A^c)\]

• If event \(A\) is rolling an even number on a die, then \(A^c\) is rolling an odd number.
• Thus, \(P(A) = 1 - P(A^c) = 1 - \dfrac{3}{6} = \dfrac{3}{6} = \dfrac{1}{2}\).

Algebra of sets

You can use Venn diagrams to confirm the following properties of sets:

• Commutative property:
  • \(A\cup B = B\cup A\)
  • \(A\cap B = B\cap A\)
• Associative property:
  • \((A \cup B) \cup C=A \cup(B \cup C)\)
  • \((A \cap B) \cap C=A \cap(B \cap C)\)

• Distributive property:
  • \(A \cup(B \cap C)=(A \cup B) \cap(A \cup C)\)
  • \(A \cap(B \cup C)=(A \cap B) \cup(A \cap C)\)
• De Morgan’s laws:
  • \((A \cup B)^c=A^c \cap B^c\)
  • \((A \cap B)^c=A^c \cup B^c\)
• Double complement:
  • \(\left(A^c\right)^c=A\)

Summary

• Sample space, \(S\), is the set of all possible outcomes of a random experiment.
• An event, \(E\), is a set of outcomes in the sample space.
• The probability of an event is a number between 0 and 1 that describes the likelihood of the event occurring.
• If event \(E = \{O_1, O_2, \ldots\}\), then \(P(E) = P(O_1) + P(O_2) + \cdots\).
• Law of large numbers: as a random experiment is repeated many times, the sample average gets closer to the true expected value.
• Addtion rule: \(P(A\cup B) = P(A) + P(B) - P(A\cap B)\).
• Complement rule: \(P(A) = 1 - P(A^c)\)
• Venn diagrams are useful for visualising events and their relationships.