2  Random variables & modes of convergence

Example 2.2 (Example of random variables, and the technical condition required) Quick example: suppose that we roll \(n\) fair dice. Then we have finitely many possible outcomes and hence it is a simple experiment (in terms of Definition 1.31) for which it is easy to explicitly specify a probability space \((\Omega,\mathcal{F},\P)\) (as we also saw in Example 1.52 and Example 1.54 for instance):

• \(\Omega\) is the set of all possible length \(n\) sequences with entries from \(\{1,\ldots,6\}\) i.e. \[\omega=(\omega^{(1)},\omega^{(2)},\ldots,\omega^{(n)}) \quad \text{where} \quad \omega^{(i)} \in \{1,\ldots,6\}.\] Note that here we decide not to use our fancy symbols ⚀, ⚁, ⚂, ⚃, ⚄ and ⚅ that we have sometimes used in dice rolling examples in Chapter 1. The only reason for this is that the notation we use here makes it easier to write down expressions for our random variables. And that is perfectly ok: for any experiment there are many possible different ways to formulate \(\Omega\) (and hence the whole probability space \((\Omega,\mathcal{F},\P)\)) that are all functionally interchangeable!
• For \(\mathcal{F}\) we can simply take the powerset \(\mathcal{P}(\Omega)\) (cf. Definition 1.27).
• The probability measure \(\P: \mathcal{F} \to [0,1]\) is fully characterised by the fact that all outcomes are equally likely: \[\P(\{\omega\})=\frac{1}{6^n} \quad \text{for all } \omega \in \Omega\] (this is the “right one” in the sense that it reflects the “reality” of rolling fair dice).

Here are some examples of “candidate” random variables for this experiment/on this probability space:

• \(X\) gives the result of the second die. That is, \(X:\Omega \to \R\) is given by \[X(\omega)=\omega^{(2)}.\]
• \(Y\) gives the largest dice roll. That is, \(Y:\Omega \to \R\) is given by \[Y(\omega)=\max \left\{ \omega^{(1)},\omega^{(2)},\ldots,\omega^{(n)} \right\}.\]
• \(Z\) gives the average of the dice rolls. That is, \(Z:\Omega \to \R\) is given by \[Z(\omega)=\frac{1}{n} \sum_{i=1}^n \omega^{(i)}.\]

Ok, fine, that’s of course all stuff you’ve seen before. But why do we write “candidate” above? Well here’s the thing. We are very used, from earlier Prob courses, to be able to talk about probabilities via random variables. For instance the probability that \(X\) takes a value larger than \(4\). Or that \(Y\) takes the value \(6\). Etc. That’s a key feature of working with random variables that we definitely need!

But “\(X\) takes a value larger than \(4\)” is a statement about what happens to \(X\) in its co-domain i.e. on the real line in Figure 2.1. And probabilities are determined by our \(\P\), which acts on events \(A \in \mathcal{F}\) rather so that’s a completely different location in Figure 2.1! However there is a very natural way to bridge that gap, as follows.

Let \(A \subseteq \Omega\) be the set of outcomes which, when they get plugged into \(X\), result in a value for \(X\) larger than \(4\). That is, \[A=\{ \omega \in \Omega \, | \, X(\omega)>4 \}.\] If you look again at how \(X\) above is defined, it readily follows that we may also write \[A=\{ \omega \in \Omega \, | \, \omega^{(2)}>4 \}.\] The point is that we now have created an event \(A\) (event, because we’re using the powerset as our \(\sigma\)-algebra and hence any subset is an event) which by very def satisfies \[\omega \in A \iff X(\omega)>4\] i.e. if we run the experiment (cf. Definition 2.1) and observe outcome \(\omega \in \Omega\), then \(X(\omega)>4\) is true and hence “\(X\) takes a value larger than \(4\)” holds if and only if \(\omega \in A\) i.e. the event \(A\) happens! And so we can very naturally define/understand the probability that “\(X\) takes a value larger than \(4\)” happens as the probability that \(A\) happens i.e. as \(\P(A)\)! :).

Of course we can do this in more generality: if we take any subset \(B \subseteq \R\) (where \(\R\) is the real line in Figure 2.1) and wonder what the probability is that \(X\) takes a value in \(B\), then this is naturally defined/understood as that the probability that the event \(A\) happens, which contains exactly all the outcomes \(\omega \in \Omega\) for which \(X(\omega) \in B\) holds. That is, naturally defined/understood as \(\P(A)\) where the event \(A\) is given by \[A=\{ \omega \in \Omega \, | \, X(\omega) \in B \}. \tag{2.1}\]

Lovely — now we can talk about the probabilities that random variables take values in certain parts/subsets of the real line! There is just one technicality here. In this example we’re using for our \(\sigma\)-algebra \(\mathcal{F}\) the powerset, and hence any subset of \(\Omega\) is an event and is assigned a probablity by our \(\P: \mathcal{F} \to [0,1]\). In particular hence also Equation 2.1 for whatever \(B \subseteq \R\) we choose. However as we have discussed in Chapter 1, in the case of an advanced experiment (cf. Definition 1.31) we’re typically forced to use a \(\sigma\)-algebra \(\mathcal{F}\) that is smaller than the powerset. And since the probability measure \(\P: \mathcal{F} \to [0,1]\) then only assigns probabilities to some subsets of \(\Omega\) (namely the elements of \(\mathcal{F}\) i.e. our events, recall from Definition 1.50) but not all subsets, the above construction is only guaranteed to work if we know for sure that a subset of the form Equation 2.1 is actually an event i.e. element of \(\mathcal{F}\)!

Example 2.4 (Some examples of inverse images) Some quick examples of inverse images then — I’m sure you’ll remember them but they will be important for us, so good to make sure we’re all on board!

Firstly, if \(\Omega\) is finite (or countable) then the process of creating them can be nicely visualised as in Figure 2.2 for instance. But also otherwise it’s not very hard to derive them (if your function is not too annoying of course), and in the case \(\Omega=\R\) you could also use a visual method by drawing the graph of \(f\), finding the \(B\) you’re interested in on the vertical axis, and then simply checking which originals/values on the horizontal axis get mapped into that \(B\).

As an explicit example with \(\Omega=\R\), let’s take \(f(x)=x^2\). Then we have for instance (convince yourself!) that

• \(f^{-1}([1,4])=[-2,-1] \cup [1,2]\),
• \(f^{-1}((25,\infty))=(-\infty,-5) \cup (5,\infty)\),
• \(f^{-1}([-9,9])=[-3,3]\),
• \(f^{-1}([-10,-1])=\emptyset\).

Example 2.8 (Simple example of a measurable function) Whenever you’re investigating some new concept, it’s always good to first look at the simplest examples you can think of! In the case of a function \(f:\Omega \to \R\) and whether or not it is measurable according to Definition 2.6, that simplest case would obviously be a constant function i.e. \(f:\Omega \to \R\) given by \(f(\omega)=c\) for all \(\omega \in \Omega\), for some fixed \(c \in \R\).

Let’s look at what the inverse images (cf. Definition 2.3) of this guy tend to look like. It’s not very hard to convince yourself that that is quite easily done (for a visualisation, see Figure 2.3):

• If we take any Borel set \(B\) in the co-domain \(\R\) that does not contain \(c\), then there is no \(\omega \in \Omega\) that gets mapped into \(B\) at all, and hence \(f^{-1}(B)=\emptyset\).
• If on the other hand we take any Borel set \(B\) in the co-domain \(\R\) that does contain \(c\), then all \(\omega \in \Omega\) get mapped into \(B\), and hence \(f^{-1}(B)=\Omega\).

So that’s a very nice and tidy list of possibilities! :). Now, have another look at Definition 2.6: if we choose a \(\sigma\)-algebra \(\mathcal{F}\) on \(\Omega\), then our constant function is \(\mathcal{F}\)-measurable if it holds that \(f^{-1}(B) \in \mathcal{F}\) for all Borel sets \(B\) in the co-domain \(\R\). But given the above limited list, that requirement simplifies to: \(\mathcal{F}\) needs to contain \(\emptyset\) and \(\Omega\) as elements. And as you’ll probably remember (or see Definition 1.33), any \(\sigma\)-algebra contains \(\emptyset\) and \(\Omega\). Hence our conclusion is that the constant function is \(\mathcal{F}\)-measurable no matter what \(\sigma\)-algebra \(\mathcal{F}\) on \(\Omega\) we use!

As you can probably imagine, the constant function is the only one that is so accommodating of whatever \(\sigma\)-algebra we decide to use.

Example 2.10 (The sigma-algebra generated by a constant function) Let’s revisit our constant function from Example 2.8: \(f: \Omega \to \R\) given by \(f(\omega)=c\) for all \(\omega \in \Omega\), where \(c \in \R\) is fixed.

We saw in Example 2.10 already that any possible inverse image \(f^{-1}(B)\) for this function is rather trivially equal to either \(\emptyset\) or to \(\Omega\). So according to Proposition 2.9, the collection \[\sigma(f)=\{ \emptyset, \Omega \}\] is a \(\sigma\)-algebra on \(\Omega\) (indeed, we had independently already seen this in Example 1.34).

Further according to Proposition 2.9, for any \(\sigma\)-algebra \(\mathcal{F}\) on \(\Omega\), \(f\) is \(\mathcal{F}\)-measurable if and only if \(\sigma(f) \subseteq \mathcal{F}\). However, any \(\sigma\)-algebra \(\mathcal{F}\) on \(\Omega\) contains \(\emptyset\) and \(\Omega\) as elements (cf. Definition 1.33) and hence satisfies \(\sigma(f) \subseteq \mathcal{F}\). So this statement from Proposition 2.9 boils down to: \(f\) is \(\mathcal{F}\)-measurable for any \(\sigma\)-algebra \(\mathcal{F}\) on \(\Omega\). And that is indeed perfectly consistent with the conclusion we had earlier already drawn in Example 2.8!

Example 2.12 (Checking measurability of the constant function using the alternative strategy) Ok, one more time that constant function from Example 2.8 & Example 2.10 then: \(f: \Omega \to \R\) given by \(f(\omega)=c\) for all \(\omega \in \Omega\), where \(c \in \R\) is fixed. Let \(\mathcal{F}\) be some \(\sigma\)-algebra on \(\Omega\).

Let’s check, from scratch and using the in general easier method provided by Proposition 2.11 i rather than the definition from Definition 2.6, whether or not \(f\) is measurable. So following Proposition 2.11 i, we need to look at inverse images of the form \(f^{-1}((-\infty,a])\) for all \(a \in \R\). Recall from Definition 2.3 what this means again: \(f^{-1}((-\infty,a])\) is the subset consisting of all \(\omega \in \Omega\) that are mapped into the interval \((-\infty,a]\) by \(f\) or, more mathematically, \[f^{-1}((-\infty,a])=\{ \omega \in \Omega \, | \, f(\omega) \in (-\infty,a] \}.\]

If you feel even a tiny bit insecure about doing this correctly, then I’d always strongly recommend to go about this as follows. Imagine the co-domain of \(f\) i.e. the real line. Start with \(a\) “very close” to \(-\infty\) and then imagine it travelling over the real line all the way up to \(\infty\). For every value of \(a\) along that journey, investigate which \(\omega \in \Omega\) get mapped into the interval \((-\infty,a]\) and hence jointly make up the inverse image \(f^{-1}((-\infty,a])\). This gives a natural flow: as \(a\) grows and the interval \((-\infty,a]\) gets larger as well, you cover an increasingly large part of the real line and hence your inverse images \(f^{-1}((-\infty,a])\), as subsets of \(\Omega\), grow as well.

Executing that strategy in this particular example of a constant function, you quickly see the following (see Figure 2.4 for visuals):

• If \(a<c\), then the interval \((-\infty,a]\) does not contain the point \(c\) on the real line where all \(\omega \in \Omega\) get mapped to. Hence, no \(\omega \in \Omega\) get mapped into \((-\infty,a]\) i.e. \(f^{-1}((-\infty,a])=\emptyset\).
• If \(a \geq c\), then the interval \((-\infty,a]\) does contain the point \(c\) on the real line where all \(\omega \in \Omega\) get mapped to. Hence, all \(\omega \in \Omega\) get mapped into \((-\infty,a]\) i.e. \(f^{-1}((-\infty,a])=\Omega\).

So, if we now introduce a \(\sigma\)-algebra \(\mathcal{F}\) on the domain \(\Omega\) and apply the criterion from Proposition 2.11 i, then it follows that \(f\) is measurable if and only if \(\mathcal{F}\) contains both \(\emptyset\) and \(\Omega\) as its elements. And so, exactly as in Example 2.8 & Example 2.10, we can again conclude that \(f\) is measurable for any \(\mathcal{F}\), since any \(\sigma\)-algebra contains these two elements.

Example 2.14 (Constant random variables) As we consistely did in Section 2.2, let’s also look here at the simplest example we can think of: let \((\Omega,\mathcal{F},\P)\) be some probability space (cf. Definition 1.50), and consider the constant mapping \(X: \Omega \to \R\) given by \(X(\omega)=c\) for all \(\omega \in \Omega\), where \(c \in \R\) is fixed.

As we saw in Example 2.8 and Example 2.12 already, such a mapping is \(\mathcal{F}\)-measurable no matter what \(\mathcal{F}\) we’re using as part of our probability space, and hence by Definition 2.13 above it is indeed a random variable.

So in particular we’re able to talk about probabilities that \(X\) takes certain values:

• For any Borel set \(B\) in the co-domain \(\R\) that does not contain \(c\), we can compute \[\begin{align*} \text{The prob that $X$ takes a value in $B$} &= \P(X \in B) \\ &= \P \left( X^{-1}(B) \right) \\ &= \P(\emptyset) \\ &= 0, \end{align*} \] where we use Equation 2.4, for the third equation that \(X^{-1}(B)=\emptyset\) as already seen in Example 2.8, and the final one that \(\P\) is a probability measure and hence satisfies condition i of Definition 1.50.
• For any Borel set \(B\) in the co-domain \(\R\) that does contain \(c\), we can compute \[\begin{align*} \text{The prob that $X$ takes a value in $B$} &= \P(X \in B) \\ &= \P \left( X^{-1}(B) \right) \\ &= \P(\Omega) \\ &= 1, \end{align*} \] where we again use Equation 2.4, for the third equation that \(X^{-1}(B)=\Omega\) as already seen in Example 2.8, and then we use that \(\P(\Omega)=1\) by def of a probability space (cf. Definition 1.50).

These are of course exactly the values that you would expect to find right! Note that the second case includes also the special case \(B=\{c\}\), and in this case we commonly write the conclusion “\(\P(X^{-1}(\{c\}))=1\)” in the shorter form “\(\P(X=c)=1\)” as explained in Definition 2.13.

Example 2.15 (Another example of a random variable: the identity mapping) Here’s another one. Suppose that in one of your earlier Probability courses would encounter the phrase “Let \(X\) denote a real number arbitrarily chosen from the interval \([0,1]\)”. How would we, with all our fancy shizzle new concepts, now properly translate this probabilistic experiment (cf. Definition 2.1) into solid maths (rather than blindly believing that we can!)?

Well obviously what we need to do is fined a probability space and a random variable \(X\) defined on it, which capture the (intended) meaning of that phrase above. As we have mentioned before, there’s always a multitude of different perfectly suitabley choices we could make for this. One easy option, if we look back at Example 1.58, is to use the probability space \(([0,1],\mathcal{B}([0,1]),\lambda)\), where \(\lambda\) is the Lebesgue measure from Proposition 1.56. If we run the experiment for this probability space, then we observe as outcome an element from the outcome space \([0,1]\), and, as discussed in that example, the “reality” that \(\lambda\) imposes is that each number is (roughly speaking) equally likely to be chosen. Ok! So running this experiment is already giving us what we need, without including a random variable at all!

What if we nevertheless do desperately want a random variable? Well then we simply choose the mapping that doesn’t change anything about the outcome of the experiment i.e. the identity mapping \(X: [0,1] \to \R\) given by \(X(\omega)=\omega\). Would you agree that if we now run this experiment including \(X\) and look at the value that \(X\) takes (cf. Definition 2.1), then this is indeed giving us the “real number arbitrarily chosen from the interval \([0,1]\)” that we were looking for?

Of course, we still need to check that \(X\) is actually a random variable i.e. \(\mathcal{B}([0,1])\)-measurable mapping (cf. Definition 2.13). Let’s follow the recommended strategy for that, i.e. Proposition 2.11 i (the inverse images are all pretty straightforward to work out using their def from Definition 2.3 and the simple form that \(X\) has, nevertheless if you have any doubts then make visuals in the same vein as in Figure 2.4 where now \(\Omega=[0,1]\) and \(X\) is the identity mapping as defined above!):

• First let \(a<0\). Then \[X^{-1}((-\infty,a])=\{ \omega \in \Omega \, | \, X(\omega) \in (-\infty,a] \}=\emptyset\] and \(\emptyset \in \mathcal{B}([0,1])\) by Definition 1.33 i.
• Next let \(a \in [0,1]\). Then \[X^{-1}((-\infty,a])=\{ \omega \in \Omega \, | \, X(\omega) \in (-\infty,a] \}=[0,a]\] and \([0,a] \in \mathcal{B}([0,1])\) by Definition 1.41.
• Finally let \(a >1\). Then \[X^{-1}((-\infty,a])=\{ \omega \in \Omega \, | \, X(\omega) \in (-\infty,a] \}=[0,1]=\Omega\] and \([0,1]=\Omega \in \mathcal{B}([0,1])\) by Definition 1.33 i & ii.

Fine! But ehm, could we have done this final check any quicker? Well actually yes! If you take another good look at Proposition 2.11, and in particular at part ii, then that covers the situation we’re dealing with here, with \(I=[0,1]\)! Using that result, all we need to do is convince ourselves that \(X: [0,1] \to \R\) given by \(X(\omega)=\omega\) is continuous (with at most countably many discontinuities).

But obviously it is continuous! Don’t let the notation fool you here — this \(X\) is (for this purpose) “just” a function. Its name, \(X\) or otherwise, doesn’t matter — after all, what’s in a name people! Neither does it matter what symbol we use for its argument(s), recall e.g. Remark 1.5. I mean, if rather than “\(X: [0,1] \to \R\) given by \(X(\omega)=\omega\)” we would have written “\(f: [0,1] \to \R\) given by \(f(x)=x\)”, which is exactly the same function, just with a different name, then you would have immediately said “yep, that one’s continuous!” no?

Example 2.16 (Creating new random variables from existing ones) Just to quickly point out: since the defining requirement in Definition 2.13 for a mapping to be a random variable is measurability, the results from Proposition 2.11 can come in extremely handy to quickly build new random variables from existing ones! For instance, on some probability space \((\Omega,\mathcal{F},\P)\) we have all of the following (as you can and should readily check!):

• If \(X\) is a random variable and \(f:\R \to \R\) is Borel measurable, then it follows from Proposition 2.11 iii that \(Y: \Omega \to \R\) defined as \(Y:=f \circ X\) i.e. \(Y(\omega):=f(X(\omega))\) for all \(\omega \in \Omega\) is also a random variable. For example, any continuous \(f\) works (cf. Proposition 2.11 ii), so \(Y=5X^2\), \(Y=e^X\), \(Y=\cos(X)\) etc. are all examples of random variables given that \(X\) is one.
• If \(X\) and \(Y\) are random variables, then so are for instance \(Z=X+Y\), \(U=XY\) etc. (cf. Proposition 2.11 iv).
• Combining arguments from the two bullet points above, if \(X\) and \(Y\) are random variables then so is for instance \[W=XY\cos(X+Y) e^{X-Y}\] (to pick one somewhat impressive looking expression! ;)).

Note that in the above, we do “algebra with random variables” and the defining expressions for the new random variables are to be understood in pointwise mode, i.e. as equalities between functions/mappings. For instance the definition \(Y=5X^2\) is to be read as: \(Y(\omega)=5X(\omega)^2\) for all \(\omega \in \Omega\) (cf. Subsection 2.3.1).

Example 2.19 (Exploring the idea of distributions: the “behaviour” of random variables on the real line) Let’s create some feel for the situation. Take again the example visualised in Figure 2.1 (quickly review it!): we have that \(\Omega=\{\omega_1,\ldots,\omega_5\}\), which is finite and hence we are in the soothing setting of what we called a simple experiment in Definition 1.31, and as we’ve seen in Example 1.54 it is in this case very easy to make valid explicit choices for the other ingredients:

• For our \(\sigma\)-algebra \(\mathcal{F}\) we can simply take the powerset \(\mathcal{P}(\Omega)\) (cf. Definition 1.27).
• Our probability measure \(\P:\mathcal{F} \to [0,1]\) (cf. Definition 1.50) is fully determined by the probabilities for each of the individual outcomes, which can be any sequence of five non-negative numbers that add up to \(1\). Say that \(\P(\{\omega_1\})=\P(\{\omega_2\})=0.1\), \(\P(\{\omega_3\})=0.4\) and \(\P(\{\omega_4\})=\P(\{\omega_5\})=0.2\).
• Let’s also make explicit choices for the values that \(X: \Omega \to \R\) takes on the real line: say that \(X(\omega_1)=1\), \(X(\omega_2)=X(\omega_3)=3\), \(X(\omega_4)=5\) and \(X(\omega_5)=6\).

Ok, that completes the full setup of our probability space \((\Omega,\mathcal{F},\P)\) with a random variable \(X\) (of course, we should check that \(X\) satisfies the measurability condition in Definition 2.13 but that’s immediate in this case, cf. Exercise 2.4).

As final addition to the structure, as we discussed in Definition 2.13, cf. Equation 2.5 in particular, we can naturally talk about the probability that if we run the experiment, \(X\) takes its value in a Borel set \(B\) on the real line, via \(\P(X \in B)=\P(X^{-1}(B))\). In this example we can for instance walk through all the values that \(X\) takes on the real line, and compute for each the probability that \(X\) takes that particular value — check these comps for yourself to reinforce the mechanism via inverse images, and recall that singletons are Borel (cf. Definition 1.41): \[\P(X=1)=\P \left( X^{-1}(\{1\}) \right)=\P(\{\omega_1\})=0.1, \tag{2.7}\] and similarly \[\P(X=5)=0.2 \quad \text{and} \quad \P(X=6)=0.2. \tag{2.8}\] Further \[\begin{align*} \P(X=3) &= \P \left( X^{-1}(\{3\}) \right) \\ &= \P(\{\omega_2,\omega_3\}) \\ &= \P(\{\omega_2\})+\P(\{\omega_3\}) \\ &=0.1+0.4 \\ &=0.5, \end{align*} \tag{2.9}\] where the third equality uses countable additivity (cf. Definition 1.50).

Alrighty, that’s the whole structure discussed in detail — for a nice visual showing all of this, specialising the one from Figure 2.1, see Figure 2.6 (a).

Now let’s look at this: what would the expectation of this \(X\) i.e. \(\E[X]\) be (just based on old knowledge from earlier Prob courses)? Well you would probably recognise it as a discrete random variable, with range i.e. set of possible values (cf. Definition 2.3) \(\{1,3,5,6\}\), and you’ll probably remember that \(\E[X]\) can be computed as the sum of the possible values that \(X\) can take multiplied with the corresponding probabilities i.e. \[\begin{align*} \E[X] &= \sum_{k \in \{1,3,5,6\}} k \P(X=k) \\ &= 1 \cdot 0.1+ 3 \cdot 0.5 + 5 \cdot 0.2 + 6 \cdot 0.2 \\ &= 3.8. \end{align*} \] If we now stare at this for a bit, then we realise that \(\E[X]\) only depends on what happens in the bottom bit of Figure 2.6 (a), cf. Figure 2.6 (b): in words, how \(X\) “behaves on the real line”: what values \(X\) takes there and with what probabilities! And if you think a bit about it, then this is not only true for \(\E[X]\) but also for \(\E[g(X)]\) for a function \(g: \R \to \R\) via \[\E[g(X)] = \sum_{k \in \{1,3,5,6\}} g(k) \P(X=k)\] and this then also covers any distributional property of \(X\): variance, skewness, moments of any type, generating functions etc. etc.!

This “behaviour of \(X\) on the real line” is, albeit in a somewhat loose formulation, exactly what the distribution of \(X\) is! Note that even though all other parts in the structure, the bits that have been made almost fully transparent in Figure 2.6 (b), on the one hand do contribute to/determine how \(X\) behaves on the real line i.e. the distribution of \(X\), on the other hand, once we know what that behaviour/distribution of \(X\) is, that is all we need to do any “distributional” computations with \(X\).

A final point to make here, and that is actually the central claim: to fully know how \(X\) behaves on the real line, and hence what its distribution is, we don’t even need to be told what values \(X\) takes on the real line, we only need to know the following collection of probabilities: \[\P(X \in B) \quad \text{for all } B \in \mathcal{B}(\R) \tag{2.10}\] (i.e. for all Borel sets \(B\) on the real line/in the co-domain of \(X\), cf. Definition 2.13).

Why? Well if we have Equation 2.10, then we can in particular deduce the probabilities Equation 2.7–Equation 2.9 (just choose \(B=\{1\}\), \(B=\{3\}\), \(B=\{5\}\) and \(B=\{6\}\) in Equation 2.10), as well as that \(\P(\R \setminus \{1,3,5,6\})=0\). This fully reconstructs the picture in Figure 2.6 (b)!

Example 2.21 (Different random variables with the same distribution) In Example 2.19 and Definition 2.20 we go from probability space plus random variable to a (unique) distribution, and then an interesting question is whether we can also go the other way around: given some distribution, can we (uniquely) rebuild the probablity space plus random variable that generated it? The answer to that is a clear “no, not uniquely” — in fact it’s quite easy to come up with examples of quite different choices for a probability space plus a random variable that all lead to the same distribution!

For example, take \((\Omega,\mathcal{F},\P)\) where \(\Omega=\{\omega_1,\omega_2\}\), \(\mathcal{F}=\mathcal{P}(\Omega)\) and \(\P: \mathcal{F} \to [0,1]\) with \(\P(\{\omega_1\})=\P(\{\omega_2\})=1/2\) (recall that this is all guaranteed to be valid as \(\Omega\) is finite and hence it’s a “simple” experiment, recall from Example 1.54). Consider the two random variables \(X,Y:\Omega \to \R\) given by (as in Example 2.19, measurability is immediate from Exercise 2.4) \[X(\omega_1)=0, \quad X(\omega_2)=1; \quad \text{and} \quad Y(\omega_1)=1, \quad Y(\omega_2)=0\] (make a visual as in Figure 2.6 (a) for this and the below if you find that helpful!!!).

If we look at working out the distribution of \(X\) as defined in Definition 2.20, you’ll quickly realise that any Borel set \(B\) in the co-domain of \(X\) falls into one of four cases:

• if \(0 \not\in B\) and \(1 \not\in B\), then \(X^{-1}(B)=\emptyset\) and hence \[\mu_X(B)=\P(X \in B)=\P \left( X^{-1}(B) \right)=\P(\emptyset)=0;\]
• if \(0 \in B\) and \(1 \not\in B\), then \(X^{-1}(B)=\{\omega_1\}\) and hence \[\mu_X(B)=\P(X \in B)=\P \left( X^{-1}(B) \right)=\P(\{\omega_1\})=\frac{1}{2};\]
• if \(1 \in B\) and \(0 \not\in B\), then \(X^{-1}(B)=\{\omega_2\}\) and hence \[\mu_X(B)=\P(X \in B)=\P \left( X^{-1}(B) \right)=\P(\{\omega_2\})=\frac{1}{2};\]
• if \(1 \in B\) and \(0 \in B\), then \(X^{-1}(B)=\Omega\) and hence \[\mu_X(B)=\P(X \in B)=\P \left( X^{-1}(B) \right)=\P(\Omega)=1.\]

And if you now do work out the distribution of \(Y\) denoted by \(\mu_Y\), you’ll quickly realise that you get exactly the same cases with exactly the same values! That is, we have that \(\mu_X(B)=\mu_Y(B)\) for all Borel sets \(B\) i.e. \(X\) and \(Y\) have the same distribution!

And we can make things even more wild if you’d like: consider the probability space \((\Omega=[0,1],\mathcal{B}([0,1]),\P)\) where \(\P\) is the Lebesgue measure (cf. Proposition 1.56 and Example 2.15). Consider \(Z : \Omega \to \R\) given by \(Z(\omega)=0\) if \(\omega \in (1/4,3/4)\) and \(Z(\omega)=1\) if \(\omega \in [0,1/4] \cup [3/4,1]\). We would need to do a bit of work to show that \(Z\) is measurable and hence a random variable, but we have done a very similar case before in Example 2.15 so I’ll leave that up to you.

What do we get if we work out the distribution \(\mu_Z\) of \(Z\)? You guessed it, again the same one — \(Z\) has the same distribution as \(X\) and \(Y\)! Indeed, you can again split all Borel sets \(B\) in the same four cases (i.e. depending on whether or not \(0\) and \(1\) are elements), you’ll find that the inverse images \(Z^{-1}(B)\) is one of \(\emptyset\) (with \(\P(\emptyset)=0\) by Definition 1.50 i), \((1/4,3/4)\) (with \(\P((1/4,3/4))=1/2\) by Proposition 1.56 i), \([0,1/4] \cup [3/4,1]\) (with \(\P([0,1/4] \cup [3/4,1])=1/2\) by countable additivity, cf. Definition 1.50, and Equation 1.19), and \(\Omega\) (with \(\P(\Omega)=1\) either by Definition 1.50 since \(\P\) is a probability measure or since \(\Omega=[0,1]\) and Equation 1.19).

Final question then: do you recognise this distribution that \(X\), \(Y\) and \(Z\) all three share as one that you have seen before in an earlier Probability course? Maybe less obviously so from the maths we did above, but what if we describe the “behaviour” of all of these random variables on the real line in words: they take the value \(0\) with prob \(1/2\) and the value \(1\) with prob \(1/2\)? Well indeed, your very good old friend the Bernoulli distribution with parameter \(1/2\)! :).

Example 2.23 (Working out a cdf) Of course you’ve seen and worked with plenty of examples of cdf’s in your earlier Probability courses so I don’t want to dwell on this, but maybe nice to construct one together — also to hopefully hammer home that link with your existing knowledge!

Let’s take again the probability space and random variable \(X\) from Example 2.19, and as visualised in Figure 2.6. If we wanted to work out the distribution of \(X\), then we could of course go down the route of working out all probabilities Equation 2.11. However if you look back at Example 2.21, where we followed this route as well, then you’ll agree that that would become an annoyingly lengthy list of different categories of Borel sets \(B\) to consider! So let’s go down the cdf route rather — in which all essential information about the distribution is preserved (cf. Proposition 2.22) but the working is much quicker (fundamentally, as alluded to above Proposition 2.22, because it focusses on a “sufficient” yet relatively small collection of Borel sets only).

Ok, so, according to Equation 2.13 we need to work out the function \(F_X: \R \to [0,1]\) given by \[F_X(x)=\P(X \in (-\infty,x])=\P(X \leq x) \quad \text{for all } x \in \R.\] Recall, and never forget!, that probabilities involving a random variable are always (shorthand for) probabilities of inverse images (cf. Definition 2.13). In this case: \[F_X(x)=\P \left( X^{-1}((-\infty,x]) \right) \quad \text{for all } x \in \R. \tag{2.14}\]

Now, in order to work out Equation 2.14, in general I’d recommend following the same strategy as in Example 2.12: on the real line that is the co-domain of \(X\), start with \(x\) all the way on the left (i.e. “near” \(-\infty\)) and gradually let it walk to the right over your real line, towards \(\infty\). As you do so, wonder for every value of \(x\) what the inverse image \(X^{-1}((-\infty,x]) \in \mathcal{F}\) as subset of \(\Omega\) is, and what its probabilty and hence \(F(x)\) is.

However in this case we can shortcut that process a little bit given the work we already did in Example 2.19 and visualised in Figure 2.6 (b): we already know that the possible values that \(X\) takes on the real line (i.e. its range) is \(\{1,3,5,6\}\), and the probabilities that \(X\) ends up in each of these values we already worked out in Equation 2.7–Equation 2.9. Using these we can argue as follows, where we also use the standard properties of probability measures from Definition 1.50 and Theorem 1.51:

• for any \(x<1\), \(X^{-1}((-\infty,x])=\emptyset\) (there is no \(\omega \in \Omega\) that gets mapped into \((-\infty,x]\)) and hence \[F_X(x)=\P \left( X^{-1}((-\infty,x]) \right)=\P(\emptyset)=0\] (recall Definition 1.50 i).
• for any \(x \in [1,3)\), the event \(\{X \leq x\}\) (or: inverse image \(X^{-1}((-\infty,x])\)) equals the event \(\{X=1\}\) (or: inverse image \(X^{-1}(\{1\})\)) and hence \[F_X(x)=\P \left( X^{-1}((-\infty,x]) \right)=\P(X=1)=0.1.\] Note that there is no magic/handwaving here, just rock hard logic: if \(\omega \in \Omega\) is such that \(X(\omega)=1\) then also \(X(\omega) \leq x\), while if \(\omega \in \Omega\) is such that \(X(\omega) \in \{3,5,6\}\) then \(X(\omega) \not\leq x\)! Note also the subtlety here that \(x=1\) belongs to this case and not the previous one, because the interval \((-\infty,1]\) does contain \(1\) while \((-\infty,x]\) for any \(x<1\) does not contain \(1\)!
• for any \(x \in [3,5)\), we similarly get that the event \(\{X \leq x\}\) equals \(\{X=1\} \cup \{X=3\}\) and hence \[\begin{align*} F_X(x) &= \P \left( X^{-1}((-\infty,x]) \right) \\ &= \P(\{X=1\} \cup \{X=3\}) \\ &= \P(X=1)+\P(X=3) \\ &=0.6. \end{align*} \] Note that we also used here that \(\{X=1\}\) and \(\{X=3\}\) are disjoint (because obviously there is no \(\omega \in \Omega\) for which \(X(\omega)=1\) and \(X(\omega)=3\) both hold!) and hence we can use countable additivity (Definition 1.50 ii).
• similarly, we get \(F(x)=0.8\) for all \(x \in [5,6)\) and \(F(x)=1\) for all \(x \geq 6\). Check for yourself!!

In conclusion, the cdf \(F_X: \R \to [0,1]\) of \(X\) is given by \[F_X(x)=\begin{cases} 0 & \text{if } x<1 \\ 0.1 & \text{if } x \in [1,3) \\ 0.6 & \text{if } x \in [3,5) \\ 0.8 & \text{if } x \in [5,6) \\ 1 & \text{if } x \geq 6. \end{cases} \tag{2.15}\] See Figure 2.7 for a plot.

A final comment here: recall, from Definition 2.20, that the sole purpose in life of this cdf \(F_X\) is to (uniquely) represent the distribution of \(X\) i.e. its “behaviour” on the real line via the relationships laid out in Definition 2.20. It’s nice to observe how we can also visually see that here. Recall that we captured how \(X\) “behaves” on the real line earlier in Figure 2.6 (b). Now I’d like you to imagine to take the cdf plot from Figure 2.7 and place it on the real line in Figure 2.6 (b). What do we see then? Well indeed, \(F_X\) has its jumps/discontinuities exactly in the points that are also the possible values of \(X\), and furthermore the heights of these jumps/discontinuities are exactly equal to the probabilities that \(X\) ends up in each of these possible values! It’s not some massive revelation, you could easily make these observations from just the algebra as well, but nevertheless I think that it’s quite neat to observe this visual confirmation that the cdf of \(X\) indeed represents/“encodes” the distribution of \(X\)/how \(X\) “behaves” on the real line! :).

Example 2.29 (An example with three random variables) Consider the experiment of rolling three fair dice (a special case of Example 2.2). Representing each possible outcome as a triple of numbers from \(1\) to \(6\) i.e. \[ \Omega= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \, \right| \, \omega^{(i)} \in \{1,\ldots,6\} \right\} \] then \(\Omega\) is finite and hence we’re dealing with a “simple” experiment (cf. Definition 1.31), and as we know (from Example 1.54 e.g.), for setting up the rest of the probability space \((\Omega,\mathcal{F},\P)\) (cf. Definition 1.50) we can make straightforward choices: for the \(\sigma\)-algebra the powerset i.e. \(\mathcal{F}=\mathcal{P}(\Omega)\), and the probability measure \(\P: \mathcal{F} \to [0,1]\) is fully specified by the probabilities it gives to the individual outcomes i.e. \[\P(\{\omega\})=\frac{1}{6^3} \quad \text{for all } \omega \in \Omega \tag{2.29}\] (making all possible outcomes equally likely, properly reflecting the “reality” of fair dice).

Further let \(X\) give the smallest number of the first two dice, let \(Y\) give the largest number of the first two dice, and let \(Z\) give the average of the final two dice. That is, \(X,Y,Z: \Omega \to \R\) are defined as \[\begin{align*} X(\omega) &= \min\left\{ \omega^{(1)},\omega^{(2)} \right\}, \\ Y(\omega) &= \max \left\{ \omega^{(1)},\omega^{(2)} \right\}, \\ Z(\omega) &= \frac{\omega^{(2)}+\omega^{(3)}}{2}. \end{align*} \] Note that the choice for the powerset as our \(\sigma\)-algebra guarantees that all three are random variables, cf. Exercise 2.4. Further it is worth observing that the ranges/set of possible values are \(\{1,\ldots,6\}\), \(\{1,\ldots,6\}\) and \(\{1,1.5,2,2.5,\ldots,6\}\) respectively, so each of the three has a finite range and is therefore a discrete random variables (cf. Subsection 2.4.1). Of course, more generally we also know from Remark 2.24 that any random variable on this probability space is necessarily discrete!

So, just to fully touch base and go through the basics of running this experiment (cf. Definition 2.1): if you roll the three dice and observe for instance the outcome \(\omega=(3,4,6)\), then you observe that the three random variables take the following values on their respective real lines: \[\begin{align*} X \big( (3,4,6) \big) &= \min\{3,4\}=3, \\ Y \big( (3,4,6) \big) &= \max\{3,4\}=4, \\ Z \big( (3,4,6) \big) &= \frac{4+6}{2}=5. \end{align*} \] Or, in the alternative viewpoint of Figure 2.9 (d): the mapping \((X,Y,Z): \Omega \to \R^3\), \(\omega \mapsto (X(\omega),Y(\omega),Z(\omega))\) takes the value \((3,4,5) \in \R^3\).

inverse images and probs: let’s first look at some inverse images and probs, to reinforce the shorthand notation from Definition 2.13 and to stress one more time that \(\P\) by def only takes events i.e. subsets of \(\Omega\) that are elements of \(\mathcal{F}\) (so any, in this case) as input:

• Consider the probability that \(X\) takes a value larger than \(4\) i.e. \(\P(X>4)\). First let’s see what event we’re exactly dealing with (of course, this is only of several possible ways to write this out in such a way that you’re clear on which elements it contains): \[\begin{align*} \{ X>4 \} &= X^{-1} \big((4,\infty) \big) \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, X(\omega)>4 \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \min\left\{ \omega^{(1)},\omega^{(2)} \right\}>4 \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \omega^{(1)}>4 \text{ and } \omega^{(2)}>4 \right\}. \end{align*} \] You can use for instance one of your classic counting tricks to deduce that this event contains \(2 \cdot 2 \cdot 6=24\) elements/possible outcomes. By writing this event as the union of its elements as singletons, we get from countable additivity (cf. Definition 1.50) that the probability of this event equals the sum of the probabilities of its elements, and hence by Equation 2.29 that \[\P(X>4)=\frac{24}{6^3}=\frac{1}{9}.\]
• Next consider for instance the probability that \(Y\) takes the value \(6\) i.e. \(\P(Y=6)\). Again first dissecting the event in question a bit: \[\begin{align*} \{Y=6\} &= Y^{-1} \big(\{6\} \big) \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, Y(\omega)=6 \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \max\left\{ \omega^{(1)},\omega^{(2)} \right\}=6 \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \omega^{(1)}=6 \text{ or } \omega^{(2)}=6 \right\}, \end{align*} \] which contains \(1 \cdot 6 \cdot 6 + 6 \cdot 1 \cdot 6-6=66\) elements/possible outcomes. Hence similarly as in the previous bullet point we arrive at \[\P(Y=6)=\frac{66}{6^3}=\frac{66}{216}.\]
• Finally consider say the probability that \(Z\) is larger than \(X+Y\) (observe that \(\min\{a,b\}+\max\{a,b\}=a+b\)): \[\begin{align*} \{Z>X+Y\} &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, Z(\omega)>X(\omega)+Y(\omega) \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \frac{\omega^{(2)}+\omega^{(3)}}{2}>\omega^{(1)}+\omega^{(2)} \right\} \\ &= \left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \omega^{(3)}>2 \omega^{(1)}+\omega^{(2)} \right\}, \end{align*} \] which (if I’ve counted correctly!) contains \(7\) elements/possible outcomes. Hence again similar as in the first bullet point we arrive at \[\P(Z>X+Y)=\frac{7}{6^3}=\frac{7}{216}.\]

\(\sigma\)-algebras: as discussed above, for the “main” \(\sigma\)-algebra to make up our probability space \((\Omega,\mathcal{F},\P)\) we have chosen the powerset i.e. \(\mathcal{F}=\mathcal{P}(\Omega)\). But recall that even on a finite \(\Omega\) there exist many other \(\sigma\)-algebras! Of course, by very def of the powerset any other \(\sigma\)-algebra \(\mathcal{G}\) is always a sub-\(\sigma\)-algebra of \(\mathcal{F}\) i.e. \(\mathcal{G} \subseteq \mathcal{F}\).

Some examples are the \(\sigma\)-algebras generated by the three random variables: \(\sigma(X)\), \(\sigma(Y)\) and \(\sigma(Z)\) (cf. Definition 2.13 & Proposition 2.9). Further Definition 2.27 gives us four more \(\sigma\)-algebras: \(\sigma(X,Y)\), \(\sigma(X,Z)\), \(\sigma(Y,Z)\) and even \(\sigma(X,Y,Z)\).

Now, here is maybe a slightly reassuring message: going forward, we will normally not be digging into the nitty-gritty of the \(\sigma\)-algebras we encounter. They are mostly tools for us to enable us to do some lovely advanced Probability Theory! Key is to be able to work with them on a more general/intuitve level.

For instance, we can observe that we have the inclusions: \[\sigma(X) \subseteq \sigma(X,Y) \subseteq \sigma(X,Y,Z).\] To justify the first inclusion, from Proposition 2.9 we know that \(\sigma(X)\) is the smallest \(\sigma\)-algebra that makes \(X\) measurable i.e. that it is a sub-\(\sigma\)-algebra of any other \(\sigma\)-algebra that makes \(X\) measurable. Since \(X\) is \(\sigma(X,Y)\)-measurable (cf. Proposition 2.28 i), it follows that \(\sigma(X) \subseteq \sigma(X,Y)\). The other inclusion can similarly be justified by (again) making use of Proposition 2.28 i (convince yourself!). So as a general rule: if we look at \(\sigma\)-algebras generated by random variables i.e. of the form \(\sigma(\ldots)\), then the more random variables we include on these “\(\ldots\)” the larger (in a weak sense) the resulting \(\sigma\)-algebra is.

Further, the criterion from Proposition 2.28 ii is a great “common sense” rule. For instance, let’s look at \(\sigma(X,Y)\). If the experiment is done with outcome \(\omega \in \Omega\) and we know \(X(\omega)\) and \(Y(\omega)\) i.e. the minimum and the maximum of the first two dice, then you’ll agree (after maybe a little bit of thinking) that this means that we in fact know the result of the first and second die. Then Proposition 2.28 ii tells us that any subset/event (the same thing here, since \(\mathcal{F}=\mathcal{P}(\Omega)\)!) \(A \subseteq \Omega\) is an element of \(\sigma(X,Y)\) if and only if we can decide whether or not \(A\) happens based on knowing the result of the first and second die. So for instance the event that both the first and second die yield an even number \[\left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \omega^{(1)},\omega^{(2)} \in \{2,4,6\} \right\}\] is an element of \(\sigma(X,Y)\), while the event that all three dice yield an even number i.e. \[\left\{ \left. \omega=\left( \omega^{(1)},\omega^{(2)},\omega^{(3)} \right) \in \Omega \, \right| \, \omega^{(1)},\omega^{(2)},\omega^{(3)} \in \{2,4,6\} \right\}\] is not an element of \(\sigma(X,Y)\), because if we know that the first and second die are even both possibilities are still open: \(A\) may or may not happen depending on whether that unknown third die is even and so we can’t decide whether or not \(A\) happens!

Proposition 2.28 iii can be used with the same higher level thinking. It impies for instance that \(Z\) (the one in this example) is not \(\sigma(X,Y)\)-measurable, because (the same arguments as in the previious paragraph): knowing \(X(\omega)\) and \(Y(\omega)\) and hence the result of the first and second die does not tell us anything about the third die, while the result of the third die is necessary to determine \(Z(\omega)\).

As a final comment, to counterbalance our regular focus on situations with sub-\(\sigma\)-algebras a bit: if you take any two \(\sigma\)-algebras, say \(\mathcal{G}_1\) and \(\mathcal{G}_2\), then it is not always true that one of \(\mathcal{G}_1 \subseteq \mathcal{G}_2\) or \(\mathcal{G}_2 \subseteq \mathcal{G}_1\) holds! Consider for instance \(\sigma(X,Y)\) and \(\sigma(X,Z)\). Then (all three bullet points can be justified using Proposition 2.28 ii, check for yourself! — and I’m writing down the events only in shorthand notation which you can extend yourself if you find that helpful):

• Any event depending only on \(X\), say \(\{X>4\}\), is an element of both \(\sigma(X,Y)\) and \(\sigma(X,Z)\). So \(\sigma(X,Y) \cap \sigma(X,Z) \not= \emptyset\).
• There exist events, for example \(\{Y=1\}\), that are elements of \(\sigma(X,Y)\) but not of \(\sigma(X,Z)\). So \(\sigma(X,Y) \not\subseteq \sigma(X,Z)\).
• And similarly there exist events, for example \(\{Z \leq 2\}\), that are elements of \(\sigma(X,Z)\) but not of \(\sigma(X,Y)\). So \(\sigma(X,Z) \not\subseteq \sigma(X,Y)\).

So indeed, it’s not as if \(\sigma(X,Y)\) and \(\sigma(X,Z)\) are completely disjoint because they do share some elements, but it is not the case that either of the two is contained in the other.