Most of this stuff is derived from Probability With Martingales by Williams. Maybe I'll add other sources too in the coming future.

Overview

I've never really had a good intuition for what a random variable really is. In EECS126, we define random variables as $X : \Omega \xrightarrow{} \mathcal{X}$ mappings from our probability space to some set of values $\mathcal{X}$, such that if we define a $\sigma$-algebra on $\mathcal{X}$, then the preimage of every $B \in \Sigma$ is in the $\sigma$-algebra $\mathcal{F}$ of our probability space. Then, the distribution of this random variable is the function $\mu = \mathbb{P} \circ X^{-1}$, which defines a probability measure on $\mathcal{X}$.

This definition makes sense (and indeed after reading Williams' book aligns with the measure-theoretic formalism of random variables as measurable functions) but is never really used in the class. Also, I have some outstanding questions:

• Why is the distribution function a valid probability measure?
• Given a distribution, can we construct a corresponding random variable (ie. in 126, we say some random variable is sampled from an exponential distribution, but it's unclear why that RV exists?)

Measurable Functions

Williams' mostly works with real-valued random variables, so I'll be doing the same, but the definitions extend to general measurable spaces. Given a measurable space $(S, \Sigma)$, we define

Definition: $\Sigma$-measurable function: Given $h : S \xrightarrow{} \mathbb{R}$, $h$ is $\Sigma$-measurable if $h^{-1}(A) \in \Sigma, \forall A \in \mathbb{B}$.

Recall $\mathcal{B}$ is $\mathcal{B}(\mathbb{R})$ the Borel sets on the usual topology on $\mathbb{R}$. Ok, this agrees with 126 definition.

Remark: It's interesting to see the parallels between topological definitions of continuous functions and the definition of measurable functions here. Instead of the preimage of open sets being open, we have the preimage of measurable sets being measurable. There are more cool parallels like this going forward!

With this definition, we can consider useful properties of measurable functions:

• If we have some class $\mathcal{C} \subseteq \mathcal{B}$ such that $\sigma(\mathcal{C}) = \mathcal{B}$ (that is, the class generates the Borel sets), then $h^{-1} : \mathcal{C} \xrightarrow{} \Sigma \implies h \in m\Sigma$. That is, measurability on this class implies measurability alltogether. (Quick proof: Take $\Epsilon$ as the class of $B \in \mathcal{B} : h^{-1}(B) \in \Sigma$. $h^{-1}$ preserves set operations, so $\Epsilon$ is a $\sigma$-algebra. By construction, $\Epsilon \supseteq \mathcal{C}$, so it must be that $\sigma(\mathcal{C}) = \mathcal{B} \subseteq \Epsilon \implies \Epsilon = \mathcal{B}$.)
• If $S$ is topological and $h : S \xrightarrow{} \mathbb{R}$ is continuous, then $h$ is Borel.

It turns out that $m\Sigma$ is also an algebra over $\mathbb{R}$, so $\forall \lambda \in \mathbb{R}, h, h_1, h_2 \in m\Sigma$, $h_1 + h_2 \in m\Sigma, h_1 h_2 \in m\Sigma, \lambda h \in m\Sigma$.

Moreover, the composition of measurable functions is measurable: $h \in m\Sigma, f \in m\mathcal{B} \implies f \circ h \in m\Sigma$. Consider the following picture:

So, a measurable set passed through the backwards $f^{-1}$ map is still measurable, which passed through the backwards $h^{-1}$ map is still measurable.

Useful Lemma: For any measurable $(S, \Sigma)$, $h : S \xrightarrow{} \mathbb{R}$ is $\Sigma$-measurable if $\{h \leq c\} \triangleq \{s \in S : h(s) \leq c\} \in \Sigma \forall c \in \mathbb{R}$. This directly follows from the fact that $(-\infty, c] \forall c$ is a $\pi$-system on $\mathbb{R}$ that generates $\mathcal{B}$, from which we can use our above result to say that $h$ is $\Sigma$-measurable.

Lastly, given a sequence $(h_n)_{n \in \mathbb{N}} \in m\Sigma$, $\inf h_n, \lim \inf h_n, \lim\sup h_n$ are all $\Sigma$-measurable (this time not technically into $\mathcal{B}$ but rather into $\mathcal{B}(\mathbb{R} \cup \{-\infty, \infty\})$) (Proof needed??!?!!?)

The definition of a random variable directly follows from looking at measurable function. Given some (measurable) porbability space $(\Sigma, \mathcal{F})$, a random variable is just some $h \in m\mathcal{F}$.

$\sigma$-algebra generated by collection of functions on $\Sigma$: If we are given $\Omega, (Y_\lambda)_{\lambda \in C}, Y_\lambda : \Omega \xrightarrow{} \mathbb{R}$, define $\mathcal{F} \triangleq \sigma(Y_\lambda : \lambda \in C)$ as the smallest $\sigma$-algebra $\mathcal{F}$ on $\Omega$ such that each map $Y_\lambda$ is $\mathcal{F}$-measurable. Clearly, $\sigma(Y_\lambda : \lambda \in C) = \sigma(\{Y_\lambda^{-1}(B) : \lambda \in C, B \in \mathcal{B}\})$, our $\sigma$-algebra is generated by (must contain at least) the preimage of every Borel set through every function in our collection.

With 202A fresh in my mind, this seems really similar to the initial topology given some set $X$ and a family of topological spaces with functions taking $X$ to those spaces. Instead of the coarsest topology that makes all the functions continuous, here we have the smallest $\sigma$-algebra that makes all the functions measurable with respect to that algebra.

A Coin Tossing Example

Lets take a countable sequence of coin tosses such that $\Omega = \{H, T\}^N$, $\omega = (\omega_1, \omega_2, \cdots), \omega_n \in \{H, T\}$.

Distributions

If we have a random variable $X : \Omega \xrightarrow{} \mathbb{R}$, we have

So we can immediately define the probability law of this RV as $L_X \triangleq P \circ X^{-1}$, which maps from our Borel sets to $[0, 1]$ -- this is consistent with the pushforward measure given in the 126 notes. However, note that $\pi(\mathbb{R}) = \{(-\infty, c] : c \in \mathbb{R})\}$ generates $\mathcal{B}$, so $L_X$ is fully determined by some $F_X$ on the $\pi$-system, which we can define as

But this is just the CDF of the random variable! So, the CDF fully determines our probability law of $X$ because of the nice property of $\pi$-systems, that two probability measures agreeing on a $\pi$-system agree on the $\sigma$-algebra generated by that $\pi$-system. There is one other thing that we actually should have checked earlier too:

$L_X$ is a valid probability measure on $\mathbb{R}, \mathcal{B}$: $L_X(\mathbb{R}) = P(\omega : X(\omega) \in \mathbb{R}) = 1$, so it assigns measure 1 to $\mathbb{R}$. Then, lets check countable additivity: Take disjoint Borel sets $\sqcup_i B_i$. Then,

It must be that $X(\omega) \in B_i$ for exactly one $B_i$, else they would not be pairwise disjoint. So, $\omega \in X^{-1}B_i \in \mathcal{F}$. Then, given disjoint $B_i, B_j$, $X^{-1}B_i, X^{-1}B_j$ must be disjoint. For if they werent, that would imply one $\omega$ satisfies both $X(\omega) \in B_i$ and $X(\omega) \in B_j$, which is a contradiction. So, our expression becomes

So countable additivity of $L_X$ follows from countable additivity of $P$.

Constructing Random Variables From Distributions

It turns out we can go the other way too. If we have a "nicely behaved" distribution function $F$, we can construct a probability measure $L$ such that $L(-\infty, x] = F(x) \forall x$. Then, if we take our probability triple to be $(\Omega, \mathcal{F}, P) = (\mathbb{R}, \mathcal{B}, \mathcal{L}), X(\omega) = \omega$, then $F_X(x) = F(x) \forall x$, that is, there exists a random variable whose distribution function equals our given one at all points. What does nicely behaved mean?

• $F : \mathbb{R} \xrightarrow{} [0, 1], x \leq y \implies F(x) \leq F(y)$
• $\lim_{x \xrightarrow{} \infty} F(x) = 1, \lim_{x \xrightarrow{} -\infty} F(x) = 0$
• $F$ is right continuous (see this is true for $F_X(c) = P(X \leq C)$: $P(X \leq x + \frac{1}{n}) \downarrow{} P(X \leq x)$ by monotone convergence,)