Bernoulli and Binomial Distribution

STAT1003 – Statistical Techniques

Dr. Emi Tanaka

Australian National University

These slides are best viewed on a modern browser like Google Chrome on a desktop or laptop. Some interactive components may require some time to fully load.

Table of contents

• Binary events in the wild
• Bernoulli distribution
• Binomial distribution
• A Binomial random variable
• Binomial probability and distribution function
• Simulating Binomial random variables
• Use in-silico experiments to understand statistics
• Summary

Binary events in the wild

• Possible outcomes:
  (A) tail or (B) head
• For an unbiased coin, probability for each outcome is 0.5.

• Possible outcomes:
  (A) a baby girl 👧 or (B) a baby boy 👦 ignoring miscarriages, irregularities, intersex, etc
• Probability for each outcome is 0.5.

• Possible outcomes:
  (A) Labor party 🔴 or (B) Coalition party 🔵 (ignoring other parties and formation of majority or minority government)
• Probability for Labor party winning??

• Possible outcomes:
  (A) Win 🏆 or (B) Lose ❌
• Probability of winning depends on the skill level of the players.

Bernoulli distribution

• If \(X\) is a random variable representing the outcome of a single trial with two possible outcomes (0 = failure or 1 = success), we can model \(X\) using a Bernoulli distribution.
• We write \(X \sim \mathrm{Bernoulli}(p)\) where \(p\) is the probability of success.
• Probability mass function is given by: \(P(X = 1) = p\) and \(P(X = 0) = 1 - p\).
• Expected value: \(E(X) = p\).
• Variance: \(\mathrm{Var}(X) = p(1-p)\).

Binomial distribution

• The number of “successes” out of \(n\) independent Bernoulli trials with probability of success, \(p\), follows a binomial distribution with parameters \(n\) and \(p\).

• Or we can simulate the sum directly:

A Binomial random variable

\[X = X_1 + X_2 + \cdots + X_n \sim B(n, p)\]

• \(X_i \sim \text{Bernoulli}(p)\) where \(p\) is the probability of success,
• \(X_i = 1\) if \(i\)-th trial is a success, otherwise \(X_i = 0\),
• all the trials are independent and \(p\) is constant for all trials,
• \(X \in \{0, 1, \ldots, n\}\) is the number of successes out of \(n\) trials.
• Bernoulli random variable is a special case of Binomial random variable when \(n = 1\).

• Expected value: \(E(X) = np\)
• Variance: \(\text{Var}(X) = np(1-p)\)
• Standard deviation: \(\text{SD}(X) = \sqrt{np(1-p)}\)

Probability mass function:

\[P(X = x) = \binom{n}{x} p^x (1-p)^{n-x}\]

Binomial probability and distribution function

Suppose I flip an unbiased coin 10 times (so \(n = 10, p = 0.5\)).

• What is the probability that exactly 3 are heads?

• What is the probability that there are 3 or less heads?

• What is the probability that there are 3 or more heads?

Simulating Binomial random variables

Simulating coin flips and count the number of heads

Use in-silico experiments to understand statistics

• Computer-based simulations are “cheap”.
• Understand how statistics behave under known data-generating process.

viewof n_exp = Inputs.number([5, Infinity], {step: 1, label: "n_exp", value: 5})
viewof n_trials = Inputs.number([1, Infinity], {step: 1, label: "n_trials", value: 10})
viewof prob = Inputs.range([0, 1], {step: 0.05, label: "prob", value: 0.5})

Summary

\[X \sim B(n, p)\]

• \(X\) is the number of successes in \(n\) independent Bernoulli trials with probability of success, \(p\).
• Expected value: \(E(X) = np\).
• Variance: \(\text{Var}(X) = np(1-p)\).
• Probability mass function: \(P(X = x) = \binom{n}{x} p^x (1-p)^{n-x}\) for \(x = 0, 1, \ldots, n\).