Basic Probability

This chapter establishes the fundamental concepts of probability theory, introducing sample spaces, events, and their associated probabilities. A solid grasp of these foundational principles is essential for tackling advanced topics in statistical modeling and machine learning, making this material critically important for the CMI examination.

---

Chapter Contents

|

| Topic |

|---|-------| | 1 | Sample Spaces and Events | | 2 | Finite Probability Spaces |

---

We begin with Sample Spaces and Events.

Part 1: Sample Spaces and Events

This section covers the fundamental concepts of sample spaces and events, which form the bedrock of probability theory. It focuses on defining the set of all possible outcomes and subsets representing specific occurrences.

---

Formula Sheet

---

Key Definitions

---

Must Remember

Define $\Omega$ first: Always start by clearly defining the sample space for any probability problem. This forms the universal set for all events.

Events are Subsets: Every event is a subset of the sample space.

Visualisation: Venn diagrams are invaluable tools for understanding and manipulating events and their relationships (union, intersection, complement).

Mutually Exclusive vs. Independent: Do not confuse mutually exclusive events with independent events. Mutually exclusive events cannot happen together ($P(E_1 \cap E_2) = 0$), while independent events mean the occurrence of one does not affect the probability of the other ($P(E_1 \cap E_2) = P(E_1)P(E_2)$).

Discrete vs. Continuous: The nature of the sample space (discrete or continuous) dictates the method of assigning probabilities (summation for discrete, integration for continuous).

---

Common Mistakes

---

Previous Year Questions

type="MSQ" question="[2016] Varsha lives alone and dislikes cooking, so she goes out for dinner every evening. She has two favourite restaurants, Dosa Paradise and Kababs Unlimited, which she travels by local train. The train to Dosa Paradise runs every $10$ minutes, at $0$, $10$, $20$, $30$, $40$ and $50$ minutes past the hour. The train to Kababs Unlimited runs every $20$ minutes, at $8$, $28$ and $48$ minutes past the hour. She reaches the station at a random time between $7{:}15$ pm and $8{:}15$ pm and chooses between the two restaurants based on the next available train. What is the probability that she ends up in Kababs Unlimited?" options=["$\frac{1}{5}$","$\frac{1}{3}$","$\frac{2}{5}$","$\frac{1}{2}$"] answer="$\frac{2}{5}$" hint="Consider the arrival time as a continuous random variable over the 60-minute interval. For each point in time, determine which restaurant's train departs sooner." solution="Let the arrival time $T$ be uniformly distributed in the interval $[15, 75)$ minutes past 7:00 pm. The total duration is $75 - 15 = 60$ minutes.

Train departure times (minutes past 7:00 pm) within or immediately after the arrival interval:
* Dosa Paradise (D): $D_{times} = \{20, 30, 40, 50, 60, 70\}$
* Kababs Unlimited (K): $K_{times} = \{28, 48, 68\}$ (the next K train after 68 would be 88, which is past 8:15pm).

Varsha chooses Kababs Unlimited if the next available Kababs train departs at or before the next available Dosa train. Let $D(T)$ be the departure time of the next Dosa train $\ge T$, and $K(T)$ be the departure time of the next Kababs train $\ge T$. We want to find the length of $T \in [15, 75)$ such that $K(T) \le D(T)$.

We analyze the intervals for $T$:

$T \in [15, 20]$: $D(T)=20$, $K(T)=28$. $20 < 28 \implies$ Dosa. Length: $5$.

$T \in (20, 28]$: $D(T)=30$, $K(T)=28$. $28 < 30 \implies$ Kababs. Length: $8$.

$T \in (28, 30]$: $D(T)=30$, $K(T)=48$. $30 < 48 \implies$ Dosa. Length: $2$.

$T \in (30, 40]$: $D(T)=40$, $K(T)=48$. $40 < 48 \implies$ Dosa. Length: $10$.

$T \in (40, 48]$: $D(T)=50$, $K(T)=48$. $48 < 50 \implies$ Kababs. Length: $8$.

$T \in (48, 50]$: $D(T)=50$, $K(T)=68$. $50 < 68 \implies$ Dosa. Length: $2$.

$T \in (50, 60]$: $D(T)=60$, $K(T)=68$. $60 < 68 \implies$ Dosa. Length: $10$.

$T \in (60, 68]$: $D(T)=70$, $K(T)=68$. $68 < 70 \implies$ Kababs. Length: $8$.

$T \in (68, 70]$: $D(T)=70$, $K(T)=88$. $70 < 88 \implies$ Dosa. Length: $2$.

$T \in (70, 75)$: $D(T)=80$, $K(T)=88$. $80 < 88 \implies$ Dosa. Length: $5$.

Total length of intervals where Varsha goes to Kababs Unlimited = $8 + 8 + 8 = 24$ minutes.
Total length of arrival interval = $60$ minutes.
The probability is $\frac{24}{60} = \frac{2}{5}$.
"

---

Quick Practice

type="MCQ" question="A fair six-sided die is rolled. Which of the following defines the event 'rolling an even number or a number greater than 4'?" options=["$\{2, 4, 6\}$","$\{2, 4, 5, 6\}$","$\{4, 6\}$","$\{2, 5, 6\}$"] answer="$\{2, 4, 5, 6\}$" hint="Identify the sample space, then list outcomes for each individual condition ('even number', 'greater than 4'), and finally combine them using the 'or' operator (union)." solution="The sample space for a six-sided die roll is $\Omega = \{1, 2, 3, 4, 5, 6\}$.
Let $A$ be the event 'rolling an even number'. $A = \{2, 4, 6\}$.
Let $B$ be the event 'rolling a number greater than 4'. $B = \{5, 6\}$.
The event 'rolling an even number or a number greater than 4' is the union of $A$ and $B$: $A \cup B = \{2, 4, 6\} \cup \{5, 6\} = \{2, 4, 5, 6\}$."

type="MCQ" question="Consider an experiment where two fair coins are tossed. Let $E_1$ be the event 'at least one head' and $E_2$ be the event 'exactly two tails'. Are $E_1$ and $E_2$ mutually exclusive?" options=["Yes, because $P(E_1 \cap E_2) = 0$","No, because $P(E_1 \cap E_2) \neq 0$","Yes, because $E_1 \cup E_2 = \Omega$","No, because $E_1 \cup E_2 \neq \Omega$"] answer="Yes, because $P(E_1 \cap E_2) = 0$" hint="First, define the sample space. Then list the outcomes for $E_1$ and $E_2$. Check their intersection." solution="The sample space for tossing two fair coins is $\Omega = \{HH, HT, TH, TT\}$.
Event $E_1$: 'at least one head' = $\{HH, HT, TH\}$.
Event $E_2$: 'exactly two tails' = $\{TT\}$.
The intersection of $E_1$ and $E_2$ is $E_1 \cap E_2 = \{HH, HT, TH\} \cap \{TT\} = \emptyset$.
Since their intersection is the empty set, $E_1$ and $E_2$ are mutually exclusive. This implies $P(E_1 \cap E_2) = P(\emptyset) = 0$."

---

Remember

> A clear and precise definition of the sample space and events is the most critical first step for solving any probability problem.

See full notes for detailed explanations and worked examples!

---

---

Part 2: Finite Probability Spaces

This section provides a concise overview of finite probability spaces, focusing on essential formulas, definitions, and problem-solving techniques. It's designed for rapid recall of fundamental concepts for the CMI exam.

---

Formula Sheet

|

| Concept | Formula | When to Use |
|---|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| 1 | Probability Space | $(\Omega, \mathcal{F}, P)$ | Foundation of probability theory: sample space, event space, probability measure. |
| 2 | Probability Measure Axioms | 1. $P(E) \ge 0$ for all $E \in \mathcal{F}$
2. $P(\Omega) = 1$
3. $P(\cup_{i=1}^n E_i) = \sum_{i=1}^n P(E_i)$ if $E_i$ are mutually exclusive | Defines a valid probability function. For finite spaces, finite additivity is sufficient. |
| 3 | Equally Likely Outcomes | $P(E) = \frac{|E|}{|\Omega|}$ | When all outcomes in a finite sample space are equally likely. |
| 4 | Complement Rule | $P(E^c) = 1 - P(E)$ | To find the probability of an event not occurring. |
| 5 | Union of Two Events | $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ | To find the probability of at least one of two events occurring. |
| 6 | Union of Mutually Exclusive Events | $P(A \cup B) = P(A) + P(B)$ | If $A$ and $B$ cannot occur simultaneously ($A \cap B = \emptyset$). |
| 7 | Conditional Probability | $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$, for $P(B) > 0$ | To find the probability of event $A$ given that event $B$ has occurred. |
| 8 | Multiplication Rule | $P(A \cap B) = P(A \mid B)P(B)$ | To find the probability of both $A$ and $B$ occurring. |
| 9 | Event Independence | $P(A \cap B) = P(A)P(B)$ | If the occurrence of $A$ does not affect the probability of $B$. |
| 10 | Permutations | $P(n,k) = \frac{n!}{(n-k)!}$ | Number of ways to arrange $k$ items from $n$ distinct items (order matters). |
| 11 | Combinations | $\binom{n}{k} = \frac{n!}{k!(n-k)!}$ | Number of ways to choose $k$ items from $n$ distinct items (order does not matter). |
| 12 | Inclusion-Exclusion (3 Events) | $P(A \cup B \cup C) = \sum P(A) - \sum P(A \cap B) + P(A \cap B \cap C)$ | For finding the probability of the union of three events. |

---

Key Definitions

---

Must Remember

Define $\Omega$ clearly: The first step in any probability problem is to correctly identify the sample space and its size ($|\Omega|$).

Distinguish Permutations vs. Combinations: Order matters for permutations ($P(n,k)$), order does not matter for combinations ($\binom{n}{k}$).

Axioms are Fundamental: All probability calculations must adhere to the three basic axioms.

Mutually Exclusive vs. Independent: Do not confuse these concepts. Mutually exclusive implies $P(A \cap B)=0$ (if $A,B$ non-empty), while independent implies $P(A \cap B) = P(A)P(B)$.

---

Common Mistakes

---

Previous Year Questions

type="MCQ" question="[Year: 2022] The Telvio mobile service provider allows each customer to choose a part of their 10-digit mobile number when they get a new connection. The first two digits of the number are fixed by the company based on the customer region. The customer can choose the last four digits that they wish. The company chooses each of the remaining four digits uniformly at random and without replacement from the digits $\{0,1,2,\ldots,9\}$. Note that this means that the digits in positions 3, 4, 5 and 6 in a Telvio number are all different. What is the probability that, in the mobile number assigned to a new customer by Telvio, the digits in positions 3, 4, 5 and 6 appear in increasing order when read from left to right?" options=["$\frac{1}{4}$","$\frac{1}{16}$","$\frac{1}{24}$","$\frac{1}{32}$"] answer="$\frac{1}{24}$" hint="Consider the total number of ways to choose and arrange 4 distinct digits vs. the number of ways to choose 4 distinct digits and arrange them in increasing order." solution="Let the four chosen digits be $d_3, d_4, d_5, d_6$.
Total number of ways to choose 4 distinct digits from 10 and arrange them in specific positions (3, 4, 5, 6) is a permutation: $P(10,4) = 10 \times 9 \times 8 \times 7 = 5040$. This is the size of our sample space $|\Omega|$.
For the digits to appear in increasing order, once 4 distinct digits are chosen, there is only one way to arrange them in increasing order. For example, if digits $\{1, 5, 2, 8\}$ are chosen, only $\{1, 2, 5, 8\}$ satisfies the increasing order condition.
The number of ways to choose 4 distinct digits from 10 is a combination: $\binom{10}{4} = \frac{10 \times 9 \times 8 \times 7}{4 \times 3 \times 2 \times 1} = 10 \times 3 \times 7 = 210$. This is the number of favorable outcomes $|E|$.
The probability is $\frac{|E|}{|\Omega|} = \frac{\binom{10}{4}}{P(10,4)} = \frac{210}{5040} = \frac{1}{24}$.
Alternatively, for any set of 4 distinct digits, there are $4!$ ways to arrange them. Only 1 of these ways is in increasing order. So the probability is $\frac{1}{4!} = \frac{1}{24}$."

type="MCQ" question="[Year: 2021] In the chamber containing the Philosopher's stone, Harry sees a deck of $5$ cards, each with a distinct number from $1$ to $5$. Harry removes two cards from the deck, one at a time. What is the probability that the two cards selected are such that the first card's number is exactly one more than the number on the second card?" options=["$1/5$","$4/25$","$1/4$","$2/5$"] answer="$1/5$" hint="The order of drawing cards matters. List all possible pairs and then count favorable pairs." solution="The cards are drawn one at a time, so the order matters.
Total number of ways to remove two cards one at a time from 5 distinct cards is a permutation: $P(5,2) = 5 \times 4 = 20$. This is the size of our sample space $|\Omega|$.
We are looking for pairs $(C_1, C_2)$ where $C_1$ is the first card and $C_2$ is the second card, such that $C_1 = C_2 + 1$.
The favorable pairs are:
(2, 1)
(3, 2)
(4, 3)
(5, 4)
There are 4 such favorable outcomes $|E|$.
The probability is $\frac{|E|}{|\Omega|} = \frac{4}{20} = \frac{1}{5}$."

type="MCQ" question="[Year: 2020] A fair coin is repeatedly tossed. Each time a head appears, $1$ rupee is added to the first bag. Each time a tail appears, $2$ rupees are put in the second bag. What is the probability that both the bags have the same amount of money after $6$ coin tosses?" options=["$\frac{1}{2^6}$","$\frac{6!}{2!\,4!\,2^6}$","$\frac{2^2}{2^6}$","$\frac{6!}{2^6}$"] answer="$\frac{6!}{2!\,4!\,2^6}$" hint="Let H be the number of heads and T be the number of tails. Set up an equation for the amounts in the bags." solution="Let $H$ be the number of heads and $T$ be the number of tails after 6 tosses.
We know $H+T=6$.
Amount in the first bag = $H \times 1 = H$ rupees.
Amount in the second bag = $T \times 2 = 2T$ rupees.
We want the amounts to be equal, so $H = 2T$.
Substitute $T = 6-H$ into the equation: $H = 2(6-H) \Rightarrow H = 12 - 2H \Rightarrow 3H = 12 \Rightarrow H = 4$.
If $H=4$, then $T = 6-4=2$.
So, we need exactly 4 heads and 2 tails in 6 tosses.
The number of ways to get 4 heads in 6 tosses is given by the binomial coefficient $\binom{6}{4} = \frac{6!}{4!2!}$.
The total number of possible outcomes for 6 coin tosses is $2^6$.
The probability is $\frac{\binom{6}{4}}{2^6} = \frac{\frac{6!}{4!2!}}{2^6} = \frac{6!}{2!\,4!\,2^6}$."

type="MCQ" question="[Year: 2018] Let $C_n$ be the number of strings $w$ consisting of $n$ X's and $n$ Y's such that no initial segment of $w$ has more Y's than X's. Now consider the following problem. A person stands in the middle of a swimming pool holding a bag of $n$ red and $n$ blue balls. He draws a ball out one at a time and discards it. If he draws a blue ball, he takes one step back, if he draws a red ball, he moves one step forward. What is the probability that the person remains dry?" options=["$\frac{C_n}{2^{2n}}$","$\frac{C_n}{\binom{2n}{n}}$","$\frac{n \cdot C_n}{(2n)!}$","$\frac{n \cdot C_n}{\binom{2n}{n}}$"] answer="$\frac{C_n}{\binom{2n}{n}}$" hint="The problem describes a path where 'red' is forward and 'blue' is backward. The condition 'remains dry' means the path never goes below the starting point. This is directly related to Catalan numbers." solution="The person draws $n$ red balls and $n$ blue balls, for a total of $2n$ draws.
Let drawing a red ball (R) be a step forward (+1) and drawing a blue ball (B) be a step backward (-1).
The person starts at position 0. To 'remain dry' means the person's position never goes below 0. This is a classic problem related to Dyck paths.
The total number of ways to arrange $n$ red balls and $n$ blue balls is $\binom{2n}{n}$, which is the total size of our sample space $|\Omega|$.
The number of paths from $(0,0)$ to $(2n,0)$ that do not go below the x-axis (i.e., never having more blue balls than red balls in any initial segment) is given by the $n$-th Catalan number, $C_n = \frac{1}{n+1}\binom{2n}{n}$.
The problem statement defines $C_n$ as 'the number of strings $w$ consisting of $n$ X's and $n$ Y's such that no initial segment of $w$ has more Y's than X's'. If X corresponds to a step forward (Red) and Y to a step backward (Blue), then $C_n$ is exactly the number of favorable outcomes $|E|$ where the person remains dry.
Therefore, the probability is $\frac{|E|}{|\Omega|} = \frac{C_n}{\binom{2n}{n}}$."

type="MCQ" question="[Year: 2017] An FM radio channel has a repository of $10$ songs. Each day, the channel plays $3$ distinct songs that are chosen randomly from the repository. Mary decides to tune in to the radio channel on the weekend after her exams. What is the probability that no song gets repeated during these $2$ days?" options=["$\binom{10}{3}^2 \cdot \binom{10}{6}^{-1}$","$\binom{10}{6}\cdot \binom{10}{3}^{-2}$","$\binom{10}{3}\cdot \binom{7}{3}\cdot \binom{10}{3}^{-2}$","$\binom{10}{3}\cdot \binom{7}{3}\cdot \binom{10}{6}^{-1}$"] answer="$\binom{10}{3}\cdot \binom{7}{3}\cdot \binom{10}{3}^{-2}$" hint="Calculate the total possible song selections over two days, and then the number of selections where no song is repeated." solution="On Day 1, the channel chooses 3 distinct songs from 10. The number of ways is $\binom{10}{3}$.
On Day 2, the channel also chooses 3 distinct songs from 10. The number of ways is $\binom{10}{3}$.
The total number of ways the channel can play songs over two days (allowing repetition between days) is $\binom{10}{3} \times \binom{10}{3} = \left(\binom{10}{3}\right)^2$. This is our sample space $|\Omega|$.
For no song to be repeated during these 2 days, the 3 songs chosen on Day 2 must be distinct from the 3 songs chosen on Day 1.
Number of ways to choose 3 songs on Day 1: $\binom{10}{3}$.
After Day 1, 3 songs have been played. So, for Day 2, there are $10-3=7$ songs remaining in the repository from which to choose.
Number of ways to choose 3 distinct songs from the remaining 7 songs for Day 2: $\binom{7}{3}$.
The number of favorable outcomes (no song repeated) is $\binom{10}{3} \times \binom{7}{3}$. This is $|E|$.
The probability is $\frac{|E|}{|\Omega|} = \frac{\binom{10}{3} \times \binom{7}{3}}{\left(\binom{10}{3}\right)^2}$.
This expression matches option 3: $\binom{10}{3}\cdot \binom{7}{3}\cdot \binom{10}{3}^{-2}$."

---

Quick Practice

A bag contains 5 red and 3 blue balls. If two balls are drawn without replacement, what is the probability that both are red?

In a class of 30 students, 18 play football, 15 play cricket, and 7 play both. What is the probability that a randomly chosen student plays neither football nor cricket?

A fair die is rolled twice. What is the probability that the sum of the outcomes is 7, given that the first roll was an odd number?

---

Remember

> Master the art of defining the sample space and identifying favorable outcomes for accurate probability calculations.

See full notes for detailed explanations and worked examples!

---

Chapter Summary

* Sample Space ($\Omega$): The set of all possible outcomes of a random experiment. Outcomes must be mutually exclusive and exhaustive.
* Events ($E$): Any subset of the sample space $\Omega$. The power set $\mathcal{P}(\Omega)$ forms the set of all possible events for a finite sample space.
* Axioms of Probability: For any event $E$, $0 \le P(E) \le 1$. $P(\Omega) = 1$. For mutually exclusive events $E_1, E_2, \dots$, $P(\bigcup_{i=1}^{\infty} E_i) = \sum_{i=1}^{\infty} P(E_i)$. For finite spaces, this simplifies to finite additivity.
* Finite Probability Space: Defined by $(\Omega, \mathcal{F}, P)$, where $\Omega$ is a finite set, $\mathcal{F}$ is the power set of $\Omega$, and $P$ is a probability measure.
* Equally Likely Outcomes: If all outcomes in a finite sample space are equally likely, then the probability of an event $E$ is given by $P(E) = \frac{|E|}{|\Omega|}$.
* Basic Probability Rules:
* $P(E^c) = 1 - P(E)$
* $P(\emptyset) = 0$
* Inclusion-Exclusion Principle: For any two events $A$ and $B$, $P(A \cup B) = P(A) + P(B) - P(A \cap B)$.

---

Chapter Review Questions

type="MCQ" question="Two fair six-sided dice are rolled. What is the probability that the sum of the numbers shown on the dice is 7?" options=["1/12","1/6","1/9","5/36"] answer="1/6" hint="Enumerate all possible outcomes for the sum of 7 and count the total possible outcomes." solution="The total number of possible outcomes when rolling two fair dice is $6 \times 6 = 36$. The outcomes where the sum is 7 are: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1). There are 6 such outcomes. Thus, the probability is $\frac{6}{36} = \frac{1}{6}$."

type="NAT" question="In a survey of 200 students, 120 take Mathematics, 90 take Physics, and 40 take both. How many students take neither Mathematics nor Physics?" answer="30" hint="Use the Inclusion-Exclusion Principle to find the number of students taking at least one subject." solution="Let $M$ be the event that a student takes Mathematics and $P$ be the event that a student takes Physics.
We are given:
$|M| = 120$
$|P| = 90$
$|M \cap P| = 40$

Using the Inclusion-Exclusion Principle for counts:
$|M \cup P| = |M| + |P| - |M \cap P|$
$|M \cup P| = 120 + 90 - 40 = 210 - 40 = 170$

The number of students who take at least one subject is 170.
The total number of students surveyed is 200.
The number of students who take neither subject is:
$200 - |M \cup P| = 200 - 170 = 30$."

type="NAT" question="A fair coin is tossed 5 times. What is the probability of observing at least one head? Express your answer as a decimal rounded to four decimal places." answer="0.9688" hint="Consider the complement event: 'no heads'." solution="The total number of possible outcomes when tossing a fair coin 5 times is $2^5 = 32$.
The event 'at least one head' is the complement of the event 'no heads' (i.e., all tails).
There is only 1 outcome where no heads appear (TTTTT).
So, $P(\text{no heads}) = \frac{1}{32}$.
Therefore, $P(\text{at least one head}) = 1 - P(\text{no heads}) = 1 - \frac{1}{32} = \frac{31}{32}$.
As a decimal, $\frac{31}{32} = 0.96875$. Rounded to four decimal places, this is 0.9688."

---

What's Next?