0.2 — Mathematical Prerequisites

📅 January 15, 2025

0.2 — Mathematical Prerequisites#

Before diving into probability theory, we need to establish the mathematical foundations. This lesson reviews essential concepts from set theory and combinatorics that we’ll use throughout the series.

Set Theory Basics#

A set is a collection of distinct objects. Sets are fundamental to probability because events are sets of outcomes. Learn more: Set Theory

Set Notation#

Elements: Objects in a set are called elements
Membership: $a \in A$ means “a is an element of set A”
Empty Set: $\emptyset$ or ${}$ is the set with no elements
Universal Set: $Ω$ (omega) is the set of all possible outcomes

Set Operations#

Union ( $A \cup B$ ): All elements in A or B (or both)

${1, 2, 3} \cup {3, 4, 5} = {1, 2, 3, 4, 5}$

Intersection ( $A \cap B$ ): Elements in both A and B

${1, 2, 3} \cap {3, 4, 5} = {3}$

Complement ( $A^{c}$ or $\overset{ˉ}{A}$ ): All elements not in A (relative to universal set)

If $Ω = {1, 2, 3, 4, 5}$ and $A = {1, 2}$ , then:

$A^{c} = {3, 4, 5}$

Difference ( $A ∖ B$ ): Elements in A but not in B

${1, 2, 3} ∖ {3, 4} = {1, 2}$

Set Properties#

Disjoint Sets: $A \cap B = \emptyset$ (no common elements)
Subset: $A \subseteq B$ means all elements of A are in B
Cardinality: $∣ A ∣$ is the number of elements in set A

Combinatorics#

Combinatorics is the study of counting arrangements and selections. It’s essential for calculating probabilities in discrete settings. Learn more: Combinatorics

Multiplication Principle#

If you can do task A in $m$ ways and task B in $n$ ways, you can do both in $m \times n$ ways.

Example: If you have 3 shirts and 4 pants, you have $3 \times 4 = 12$ outfits.

Permutations#

A permutation is an ordered arrangement of objects. Learn more: Permutation

Permutations of n distinct objects: $n! = n \times (n - 1) \times ... \times 2 \times 1$

$5! = 5 \times 4 \times 3 \times 2 \times 1 = 120$

Permutations of r objects from n: $P (n, r) = \frac{n !}{( n - r )!}$

$P (5, 3) = \frac{5 !}{2 !} = 60$

Combinations#

A combination is an unordered selection of objects (order doesn’t matter). Learn more: Combination

Combinations of r objects from n: $C (n, r) = (r n) = \frac{n !}{r ! ( n - r )!}$

$(3 10) = \frac{10 !}{3 ! 7 !} = 120$

Key Formula: $(r n) = (n - r n)$ (choosing r is same as excluding n-r)

Binomial Theorem#

$(a + b)^{n} = \sum_{k = 0}^{n} (k n) a^{n - k} b^{k}$

This will be important when we study the binomial distribution! Learn more: Binomial Theorem

Summation and Product Notation#

Summation ( $\sum$ ): $\sum_{i = 1}^{n} x_{i} = x_{1} + x_{2} + ... + x_{n}$

Product ( $\prod$ ): $\prod_{i = 1}^{n} x_{i} = x_{1} \times x_{2} \times ... \times x_{n}$

Key Formulas:

$\sum_{i = 1}^{n} i = \frac{n ( n + 1 )}{2}$
$\sum_{i = 1}^{n} i^{2} = \frac{n ( n + 1 ) ( 2 n + 1 )}{6}$

Functions and Limits#

Functions#

A function $f : A \to B$ maps each element of set A to an element of set B.

Domain: Set A (inputs)
Range/Codomain: Set B (outputs)

Limits#

The limit of $f (x)$ as $x$ approaches $a$ is $L$ : $lim_{x \to a} f (x) = L$ . Learn more: Limit (Mathematics)

Important for continuous probability distributions!

Continuity#

A function is continuous if small changes in input cause small changes in output. Probability density functions are continuous.

Basic Calculus#

Derivatives#

The derivative $f^{'} (x)$ measures the rate of change:

$f^{'} (x) = lim_{h \to 0} \frac{f ( x + h ) - f ( x )}{h}$

Learn more: Derivative

Common Derivatives:

$(x^{n})^{'} = n x^{n - 1}$
$(e^{x})^{'} = e^{x}$
$(ln x)^{'} = \frac{1}{x}$

Integrals#

The integral $\int_{a}^{b} f (x) d x$ measures the area under the curve. Learn more: Integral

Fundamental Theorem of Calculus: $\int_{a}^{b} f^{'} (x) d x = f (b) - f (a)$ . Learn more: Fundamental Theorem of Calculus

Common Integrals:

$\int x^{n} d x = \frac{x ^{n + 1}}{n + 1} + C where n \neq = - 1$

$\int e^{x} d x = e^{x} + C$

$\int \frac{1}{x} d x = ln ∣ x ∣ + C$

Series#

Geometric Series#

For $∣ r ∣ < 1$ : $\sum_{i = 0}^{\infty} r^{i} = \frac{1}{1 - r}$

For finite: $\sum_{i = 0}^{n} r^{i} = \frac{1 - r ^{n + 1}}{1 - r}$

Learn more: Geometric Series

Taylor Series#

$f (x) = \sum_{n = 0}^{\infty} \frac{f ^{(n)} ( a )}{n !} (x - a)^{n}$

Important for moment generating functions! Learn more: Taylor Series

Why This Matters#

These mathematical tools are essential because:

Set Theory: Events in probability are sets of outcomes
Combinatorics: Counts outcomes in discrete probability
Calculus: Needed for continuous distributions and expectations
Series: Used in moment generating functions and infinite sums

Next Steps#

In the next lesson, we’ll set up Python and the scientific computing libraries (NumPy, SciPy) that we’ll use throughout this series to implement probability concepts.

Quick Reference#

Concept	Notation	Example
Union	$A \cup B$	${1, 2} \cup {2, 3} = {1, 2, 3}$
Intersection	$A \cap B$	${1, 2} \cap {2, 3} = {2}$
Complement	$A^{c}$	If $Ω = {1, 2, 3}$ , then ${1}^{c} = {2, 3}$
Permutation	$P (n, r)$	$P (5, 3) = 60$
Combination	$(r n)$	$(3 10) = 120$
Factorial	$n!$	$5! = 120$