7 Some nice applications

7.1 Why?

Because we have developed a lot of concepts so far with some examples here and there, but it’s also nice to see some of the many applications martingales have — also because that’s an important part of creating more feel and appreciation for the theory I think! As mentioned previously, martingales pop up in so many places that it is very difficult to give even a sense of that in the one chapter we have for it.

I’ve chosen to focus on some applications of Doob’s optional stopping theorem to look at some more properties of first hitting times of random walks. Such stopping times are often very important, for instance in math finance and in engineering. As a bonus we encounter several more interesting martingales in the process!

However, as already mentioned, there is so much more. If you also take coures like Stochastic Calculus, Stochastic Modelling in Finance and/or Stochastic Control with Applications to Finance you’ll see some of that. If you’re curious (already) though I can really recommend having a look through Chapter 15 in Williams (1991) for a nice selection that illustrates a diversity of possible applications!

7.2 Gambler’s ruin

Gambler’s ruin is a term that goes back to at least the 17th century and has been used to describe a number of different situations but it always has the following core meaning: if a player persistently plays a betting game that is not unfair in his advantage (so the expected profit per game per £1 stake is £1 or less) then no matter how large the player’s capital at the start is and no matter how hard they try (with betting strategies etc.), they will ultimately lose all their money.

We have seen some formulations of this in action already:

In Proposition 5.1 we saw that, under a mild technical condition on the betting strategy, the accumulated profits process for any betting strategy is a (super)martingale i.e. on average the player’s wealth will either stay constant (if the game is fair) or strictly decrease (if the game is strictly unfair). This does not obviously imply the gambler’s ruin statement as formulated above but it is clear that it is doesn’t make things look very good for the player.
In Exercise 6.7 we have seen that in the case that the accumulated profits process forms a simple random walk (i.e. when $Y$, the profit per game per £1 stake, is either -£1 or £1) then when the game is not unfair in the player’s advantage (i.e. $\E[Y] \leq 0$) and the player uses the simple strategy of staking £1 in each game, then no matter how large the player’s starting capital (denoted by $x$ in that exercise) is, almost surely the player will run out of money (in finite time)!

But you may very well still have some doubts — for instance you may very well be familiar with/have thought about the following betting strategy. Using the notation from Section 5.4, suppose that a player plays a sequence of fair games where the profit per game, per £1 stake is either -£1 or £1 with equal probability i.e. $Y$ takes the values $-1$ and $1$ with prob $1/2$ each. The strategy is to start with a stake of £1 for the first game. Each time you lose, you double the stake for the next game, and you stop playing after your first win. If we denote by $T=1,2,\ldots$ the first time you win a game and stop playing, then at that point in time your profit consists of the $2^T$ you have won in the last game minus the money lost in the previous games: \[2^T- \left( 1+2+4+\ldots+2^{T-1} \right) = 2^T-(2^T-1)=1 \tag{7.1}\] (as you’ll have guessed, using the geometric series). Note that you’ll win a game sooner or later almost surely i.e. $T<\infty$ a.s. (since the prob of losing $n$ games in a row is $(1/2)^n$, which tends to $0$ as $n \to \infty$), so seemingly we have here a strategy that guarantees that you walk away with a profit of £1 if only you are patient enough!

This seems to quite radically contradict what we’ve been saying so far about fair games. What gives? Well, the problem is that you not only may need a lot of patience, you also need a lot of capital — in fact to successfully execute this strategy you need an infinite amount of capital: there is no lowerbound on how large your loss can possibly become before you win your first game. In real life, playing with a finite amount of capital, sticking with this strategy can result in one of two possible outcomes: either you walk away with that £1 profit, or you exhaust all your available capital before you get to win your first game and you rather crawl away a broken man!¹

¹ Fun fact: this strategy is part of a set of strategies called “martingales” (different use of the word) and were popular among Parisian betting elite in 18th century France. There is an interesting connection with the St. Petersburg paradox

It is worthwhile to briefly discuss what this problem exactly looks like in our mathematical formulation, to cement our understanding. Using the notation and framework from Section 5.4, this betting strategy can be formulated as the simple previsible process $(C_n)_{n=1,2,\ldots}$ given by \[C_n=\begin{cases} 2^{n-1} & \text{if $n=1$ or $Y_1=\ldots=Y_{n-1}=-1$} \\ 0 & \text{otherwise} \end{cases} \] (i.e. your stake for the $n$-th game is $2^{n-1}$ if either it’s the first game or you haven’t won any games yet, otherwise you have stopped betting because you have won a game), the resulting accumulative profits process is $(X_n)_{n=0,1,\ldots}$ given by \[X_0=0 \quad \text{and} \quad X_n = \sum_{i=1}^n C_i Y_i \quad \text{for all $n=1,2,\ldots$.}\] There are several ways to express the (random!) time $T$ at which we stop playing, the one that follows most directly from the definition is \[T=\inf \{ n=0,1,\ldots \, | \, Y_n=1 \}\] and you can easily check that $T$ is (indeed) a stopping time (cf. Definition 6.1) and as argued above already we have that $T<\infty$ a.s. Note that the computation we already did in Equation 7.1 immediately translates to \[X_T=\sum_{i=1}^T C_i Y_i=-1-2-4- \ldots -2^{T-1}+2^T=1.\] Since $(X_n)_{n=0,1,\ldots}$ starts at level $0$ and then decreases into negative values until time $T$ when it becomes positive for the first time to hit the level $1$, we could alternatively express $T$ as \[T=\inf \{ n=0,1,\ldots \, | \, X_n=1 \}\] which shows that $T$ is a first hitting time for $(X_n)_{n=0,1,\ldots}$ as defined in Example 6.1.

Now, note that $X_0=0$ and $X_T=1$, so clearly $\E[X_0]=0 \not= 1 = \E[X_T]$ i.e. the conclusion of Doob’s optional stopping theorem (cf. Theorem 6.1) does not hold — a possibility we already warned for in Example 6.2. Apparently both conditions in Theorem 6.1 fail to hold in this case. That $T$ fails to be bounded is not hard to see: for any $N=0,1,\ldots$ the probability of losing the first $N$ games so that $T>N$ is $(1/2)^N>0$. For the second condition, apparently there does not exist an integrable random variable $Y$ that dominates $\lvert X_{n \wedge T} \rvert$ for all $n=0,1,\ldots$ i.e. that dominates the absolute values of all our accumulated profits up to (incl.) the time we stop playing. The (intuitive) reason for this is what we already alluded to above as the downside of this strategy: the quickly expanding losses we rake up before we win the first game. Indeed we can glimpse this by looking at the random time $T-1$, our accumulated profit is then \[X_{T-1} = - \left( 1+2+4+\ldots+2^{T-1} \right)=1-2^T\] and using that $T$ has a Geometric distribution it is not very hard to check that the expectation of this guy is $-\infty$: \[\begin{align*} \E[2^T] &= \sum_{k=1}^\infty 2^k \P(T=k) \\ &= \sum_{k=1}^\infty 2^k \left( \frac{1}{2} \right)^{k-1} \frac{1}{2} \\ &= \sum_{k=1}^\infty 1 \\ &=\infty. \end{align*} \]

Finally, a natural idea would be: can we not somehow avoid the downside of this strategy, we could for instance stop playing as soon as we reach some loss level $-a$ for some integer $a$ to avoid things becoming too catastrophic? Sure — then you change the time at which you stop playing to \[T=\inf \{ n=0,1,\ldots \, | \, X_n=1 \text{ or } X_n=-a \}.\] Now (as we’ll also see below) Doob’s optional stopping Theorem 6.1 does hold, and hence we get that $\E[X_T]=\E[X_0]=0$ — so we’re avoiding catastropic losses, but at a price: we now no longer have a strategy with a guaranteed profit of £1 but one in which we’ll either end up with a profit of £1 or a profit of -£$a$, with an expected value of exactly $0$…

7.3 Hitting times of simple random walks

Let’s now step away from the financial/betting angle a bit (you can of course apply the results from this section in that context if you’d like) but stick with a simple random walk $(X_n)_{n=0,1,\ldots}$ i.e. (to repeat the setup one more time, of course we’ve already extensively discussed this guy in Chapter 5) \[X_0=0 \quad \text{and} \quad X_n = \sum_{i=1}^n Y_i \quad \text{for all $n=1,2,\ldots$} \tag{7.2}\] where $Y_1, Y_2, \ldots$ are independent copies of a random variable $Y$ taking values $1$ and $-1$ with probs $p \in (0,1)$ and $1-p$ resp. (let’s exclude the trivial cases $p=0$ and $p=1$), all taking place on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_n)_{n=0,1,\ldots},\P)$ with $(\mathcal{F}_n)_{n=0,1,\ldots}$ the natural filtration of $(X_n)_{n=0,1,\ldots}$ i.e. \[\mathcal{F}_n=\sigma(X_0,\ldots,X_n) \quad \text{for all $n=0,1,\ldots$}\] or equivalently \[\mathcal{F}_0=\{\emptyset,\Omega\} \quad \text{and} \quad \mathcal{F}_n=\sigma(Y_1,\ldots,Y_n) \quad \text{for all $n=1,2,\ldots$} \tag{7.3}\]

In this section we’ll see how creatively constructing some martingales related to this random walk and making use of Doob’s optional stopping theorem allows us to learn interesting properties of first hitting times!

7.3.1 The symmetric case

First we focus on the symmetric case i.e. $p=1/2$. Recall that we have already seen that with this choice, the random walk $(X_n)_{n=0,1,\ldots}$ is a martingale (cf. Example 5.3). Consider the first hitting time (which we know already to be a stopping time, cf. Example 6.1) \[\tau=\inf \{ n=0,1,\ldots \, | \, X_n=-a \text{ or } X_n=b \} \tag{7.4}\] for some integers $a,b \in \{1,2,\ldots\}$. Recall that any path of the random walk starts at level $0$ at time $n=0$ and then evolves by moving 1 up or down at each time step, resulting in a path that visits only integers (cf. also Figure 5.2). So $-a<0$ and $b>0$ form the lower and upper bound of a strip on the time axis, and $\tau$ is the first time that a path reaches one of these two bounds (visualise analogue to Figure 6.1 (b)), and every possible path of the random walk goes into one of the following three (mutually exclusive) categories:

The path $X_0(\omega), X_1(\omega), \ldots$ reaches $-a$ before it reaches $b$. In this case $\tau(\omega)<\infty$ and $X_{\tau(\omega)}(\omega)=-a$;
The path $X_0(\omega), X_1(\omega), \ldots$ reaches $b$ before it reaches $-a$. In this case $\tau(\omega)<\infty$ and $X_{\tau(\omega)}(\omega)=b$;
The path $X_0(\omega), X_1(\omega), \ldots$ never reaches neither $-a$ nor $b$ i.e. it stays forever (strictly) inside the strip. In this case $\tau(\omega)=\infty$ and $X_{\tau(\omega)}(\omega)$ is undefined.

Now, for our analysis category 3 is pretty unpleasant. The good news is that even though this is not impossible to happen (for instance, there is a path starting at $0$, then $1$, then back to $0$, then again to $1$ etc., never hitting any strictly negative level and neither any level larger than 1) it has only probability $0$. If you think about it, then this already follows from Exercise 6.7 but it is also useful to set up an argument along the following lines.

Note that whenever a path experiences a sequence of $a+b$ consecutive moves upwards (or all downwards) then it must have reached $b$ (or $-a$ resp.) by the end of that sequence the latest. So any path in category 3 does not contain such a sequence. But there is this nice principle: in an infinite repeat of the same experiment, any (finite) combination of outcomes that is possible to happen (i.e. has prob $>0$) will eventually happen. You may well know this more poetically as the “monkey typing Shakespeare” principle: if a monkey sits down behind a keyboard and randomly bashes on the keys, as long as he does that forever then with probability $1$ he will at some point produce the full works of Shakespeare.

Here is a proof. Imagine cutting the time axis up in (infinitely many) parts of length $a+b$. In any such part, indepedently of each other, with probability $1-(1/2)^{a+b}<1$ we do not get to see $a+b$ consecutive moves upwards. Denote by $A_m$ the event that after the first $m$ parts we have not seen any part with $a+b$ consecutive moves upwards, and by $A$ the event that we never get to see any such part. Then for any $m=1,2,\ldots$ we have that $A \subseteq A_m$ and hence (cf. Theorem 1.1 i) \[\P(A) \leq \P(A_m) = \left( 1-(1/2)^{a+b} \right)^m\] and since the expression between brackets is less than $1$, by taking the limit for $m \to \infty$ it indeed follows that $\P(A)=0$.

So, almost surely we have that $\tau<\infty$ and that $X_\tau$ takes the value $-a$ or $b$. Note that, as we’ve done before but maybe good to reiterate once more, we do not need to care what happens on the event $\{\tau=\infty\}$ as long as we do any next steps almost surely and/or apply concepts that don’t care about events with probability $0$, such as the expectation operator.

Now we are almost ready to draw our first conlusion from all of this, for which we only need to apply Doob’s optional stopping theorem. Looking back at the two conditions in Theorem 6.1, we can’t use the first one as $\tau$ is not bounded, but the second one does hold! Indeed if we look at $X_{n \wedge \tau}$, then note what happens: the process starts at level $0$ at time $n=0$, while $n<\tau$ it wiggles around within the strip with bounds $-a$ and $b$, and for any $n \geq \tau$ we have that $X_{n \wedge \tau}=X_\tau \in \{-a,b\}$. Or, equivalently: the stopped process $(X^\tau_n)_{n=0,1,\ldots}$ always stays between $-a$ and $b$ (visually, the analogue of Figure 6.2). That is to say, if we define the trivial (because constant) integrable random variable $V:=\max\{a,b\}$ then we have that \[\lvert X_{n \wedge \tau} \rvert \leq V \quad \text{for all $n=0,1,\ldots$} \tag{7.5}\] and so it follows from Doob’s optional stopping theorem that \[\E[X_\tau]=\E[X_0]=0. \tag{7.6}\]

Here is the beauty of this. Recall that $X_\tau$ takes only the value $-a$ or $b$ (ignoring what happens if $\tau=\infty$) and hence is a very simple discrete random variable! Denoting $\bar{p}:=\P(X_\tau=-a)$ i.e. the probability that a path hits $-a$ before hitting $b$, the standard formula for the expectation of a discrete random variable yields \[\E[X_\tau]=-a \bar{p} + b (1-\bar{p}),\] and since we know from Equation 7.6 that this expectation is $0$, a tiny bit of algebra allows us to deduce that \[\bar{p}=\frac{b}{a+b}. \tag{7.7}\] So, in full the conclusion of this bit of work is that for a symmetric simple random walk starting from $0$, the probability of hitting $-a$ before $b$ is $b/(a+b)$ and vice versa the probability of hitting $b$ before $-a$ is $1-b/(a+b)=a/(a+b)$. Aren’t these some nice and elegant expressions??

But hang on, don’t leave just yet, there’s a lot more — besides $(X_n)_{n=0,1,\ldots}$ itself there are other martingales drifting around as well that can teach us even more about that same stopping time $\tau$! Indeed consider for instance the process $(X^2_n)_{n=0,1,\ldots}$. It follows from Exercise 6.8 that this is a submartingale, and according to Doob’s decomposition theorem (cf. Theorem 5.1) a previsible, non-decreasing process $(A_n)_{n=0,1,\ldots}$ should exist so that $(X^2_n-A_n)_{n=0,1,\ldots}$ is a martingale. It turns out that it is a simple constant process: $A_n=n$ for all $n=0,1,\ldots$, as we’ll confirm next.

Indeed consider the process $(Z_n)_{n=0,1,\ldots}$ defined by $Z_n=X^2_n-n$ for all $n=0,1,\ldots$ on the same filtered probability space we started this section with. Since $Z_n$ is a function of $X_n$, $(Z_n)_{n=0,1,\ldots}$ is adapted to the natural filtration of $(X_n)_{n=0,1,\ldots}$. You know well by now how to check that it is an integrable process. For the martingale property, it is handiest (I think) to first do a little bit of algebra to find a convenient connection between $Z_{n+1}$ and $Z_n$. Recall from Equation 7.2 that $X_n$ is a sum of $Y_i$’s so maybe we do not want to work out the square of that guy directly but rather use the observation that from Equation 7.2 it follows that $X_{n+1}=X_n+Y_{n+1}$, so that we can work out \[\begin{align*} Z_{n+1} &= X^2_{n+1}-(n+1) \\ &= \left( X_n+Y_{n+1} \right)^2 -(n+1) \\ &= X^2_n +2 X_n Y_{n+1}+Y^2_{n+1}-(n+1) \\ &= X^2_n -n+2 X_n Y_{n+1}+Y^2_{n+1}-1 \\ &= Z_n +2 X_n Y_{n+1}+Y^2_{n+1}-1, \end{align*} \] and plugging this into a conditional expectation we get that \[\begin{align*} \E[ Z_{n+1} \, | \, \mathcal{F}_n ] &= \E[ Z_n \, | \, \mathcal{F}_n ] +2 \E[ X_n Y_{n+1} \, | \, \mathcal{F}_n ] + \E[ Y^2_{n+1} \, | \, \mathcal{F}_n ]-1 \\ &= Z_n + 2 X_n \E[ Y_{n+1} \, | \, \mathcal{F}_n ] + \E[ Y^2_{n+1} \, | \, \mathcal{F}_n ]-1 \\ &= Z_n + 2 X_n \E[ Y_{n+1} ] + \E[ Y^2_{n+1} ]-1 \\ &= Z_n, \end{align*} \] where the first equation uses linearity (cf. Theorem 4.1 i), the second Theorem 4.2 ii (since $Z_n$ is $\mathcal{F}_n$-meas.) & v (since $X_n$ is $\mathcal{F}_n$-meas.), the third Theorem 4.1 vi (since the $Y_i$’s are independent copies of $Y$, $Y_{n+1}$ is independent of $\mathcal{F}_n$, cf. Equation 7.3), and the final one just works out the expectation and second moment of $Y_{n+1}$ using that it takes values $-1$ and $1$ only with prob $1/2$ each. So indeed $(Z_n)_{n=0,1,\ldots}$ is a martingale.

Here is what this martingale allows us to deduce. Assume for a moment that we could apply Doob’s optional stopping theorem to $(Z_n)_{n=0,1,\ldots}$. Note that $Z_0=X_0^2-0=0$ while $Z_\tau=X^2_\tau-\tau$ where $X_\tau$ is a still the same simple discrete random variable from above, so $X_\tau^2$ takes values $a^2$ and $b^2$ with probabilities $\bar{p}$ and $1-\bar{p}$ resp. So Doob’s optional stopping theorem would yield \[0=\E[Z_0]=\E[Z_\tau]=\E[X^2_\tau]-\E[\tau], \tag{7.8}\] and since \[\E[X^2_\tau]=a^2 \bar{p} + b^2 (1-\bar{p})=ab\] (for the final equation, just plug in the expression for $\bar{p}$ from Equation 7.7) this brings us to the following appealing expression for the mean of $\tau$: \[\E[\tau]=ab. \tag{7.9}\]

However, there is some bad news: we can’t show that the conditions of Doob’s optional stopping theorem actually hold for $(Z_n)_{n=0,1,\ldots}$, and so we can’t use this theorem to deduce that Equation 7.8 holds. Here’s the problem: our only hope is to show that the second condition in Theorem 6.1 holds. However, even though $X_{n \wedge \tau}$ and hence also $X_{n \wedge \tau}^2$ is bounded (as also seen above), the problem is the “$-(n \wedge \tau)$” term in $Z_{n \wedge \tau}=X_{n \wedge \tau}^2-(n \wedge \tau)$: since we have no integrable bound $\tau$, neither do we have an integrable bound on $-(n \wedge \tau)$…

We can work around that little setback though, with the following trick. What is definitely true is that the stopped process $(Z^\tau_n)_{n=0,1,\ldots}$ is a martingale (cf. Proposition 6.1). By the constant expectation property of martingales (cf. Equation 5.11) it follows that \[\E[Z^\tau_0]=\E[Z^\tau_n] \quad \text{for all $n=0,1,\ldots$.} \tag{7.10}\] Working out both sides a little bit, we see that \[\E[Z^\tau_0]=\E[Z_{0 \wedge \tau}]=\E[Z_0]=0\] and on the other hand for any $n=0,1,\ldots$ \[\E[Z^\tau_n]=\E[Z_{n \wedge \tau}]=\E[X^2_{n \wedge \tau}]-\E[n \wedge \tau]\] and plugging these back into Equation 7.10 yields \[0=\E[X^2_{n \wedge \tau}]-\E[n \wedge \tau] \quad \text{for all $n=0,1,\ldots$.} \tag{7.11}\] So this gives us something quite similar to Equation 7.8, except that we still have these $n$’s appearing in there. Maybe you can guess what’s coming next: we’re going to take the limit for $n \to \infty$ in Equation 7.11 to find Equation 7.8 back.

First consider the term $\E[X^2_{n \wedge \tau}]$. Be aware of what exactly appears in the expectation here: for any $\omega \in \Omega$, we’re looking at the value the path $X^2_0(\omega), X^2_1(\omega), \ldots$ has at the time point $n \wedge \tau(\omega)$ (if this confuses you, have a look back at how stopped processes work in Section 6.3). So for any $\omega \in \Omega$ fixed, we see that \[n \wedge \tau(\omega) \to \infty \wedge \tau(\omega)=\tau(\omega) \quad \text{as $n \to \infty$} \tag{7.12}\] and hence, for any $\omega \in \Omega$ with $\tau(\omega)<\infty$ (otherwise $X^2_{\tau(\omega)}(\omega)$ is not well defined as discussed above) \[X^2_{n \wedge \tau(\omega)}(\omega) \to X^2_{\tau(\omega)}(\omega) \quad \text{as $n \to \infty$}.\] That is, we have that $X^2_{n \wedge \tau}$ converges to $X^2_{\tau}$ as $n \to \infty$ on the event $\{ \tau<\infty\}$. Since this event has probability $1$, it follows that \[X^2_{n \wedge \tau} \stackrel{\text{a.s.}}{\longrightarrow} X^2_{\tau} \quad \text{as $n \to \infty$}. \tag{7.13}\]

To translate almost sure convergence of random variables into convergence of their expectations, we have two loyal friends: the MCT and the DCT (cf. Section 3.5). In this situation, there is no reason why there should be any monotonicity in our convergence and hence we go for applying the DCT (cf. Theorem 3.3). To be able to apply the DCT we need to show that its conditions are satisfied of course, but that’s not too difficult: we already have Equation 7.13, and analogue to Equation 7.5 we trivially have an integrable (because constant) upper bound $V$: \[\lvert X^2_{n \wedge \tau} \rvert \leq \max\{a^2,b^2\} =: V \quad \text{for all $n=0,1,\ldots$}\] and so it follows from the DCT that \[\E \left[ X^2_{n \wedge \tau} \right] \to \E[X^2_{\tau}] \quad \text{as $n \to \infty$}.\]

Almost there! The other term in Equation 7.11 is $\E[n \wedge \tau]$. Also for this one we would like to use either the DCT or MCT to deduce what happens if we let $n \to \infty$. Now, we already saw in Equation 7.14 that \[n \wedge \tau \stackrel{\text{a.s.}}{\longrightarrow} \tau \quad \text{as $n \to \infty$}, \tag{7.14}\] but the DCT is not going to help us here: indeed there is no integrable upperbound (as already mentioned above and indeed the reason we had to go down this route in the first place!). However we do have nice monotonicity in this case! Indeed for any $\omega \in \Omega$ fixed we have that \[0 \leq 1 \wedge \tau(\omega) \leq 2 \wedge \tau(\omega) \leq \ldots\] and hence we get from the MCT (cf. Theorem 3.2) that \[\E [ n \wedge \tau ] \to \E[\tau] \quad \text{as $n \to \infty$}. \tag{7.15}\]

Finally we can wrap up the whole argument now! Starting from Equation 7.11 we get that \[\begin{align*} 0 &= \lim_{n \to \infty} \big( \E \left[ X^2_{n \wedge \tau} \right] - \E [ n \wedge \tau ] \big) \\ &= \E[X^2_{\tau}] - \E[\tau], \end{align*} \] where we used Equation 7.14 and Equation 7.15. So we have now established that Equation 7.8 does indeed hold (albeit via an alternative route) and we can now safely execute the few computations under Equation 7.8 to conclude that it indeed holds that \[\E[\tau]=ab. \tag{7.16}\]

The key messages of these computations, besides obtaining the very nice Equation 7.7 and Equation 7.16, are as follows:

in any given stuation, like the symmetric random walk case here, there may very well be (other) martingales lurking around (like $(Z_n)_{n=0,1,\ldots}$ here) and if you can spot them that opens up a whole new angle to attack the problem at hand — indeed often enough (complicated) problems involving stochastic processes in some form can be (more or less unexpectedly) solved by detecting a crucial martingale!
if Doob’s optional stopping theorem doesn’t apply because we can’t (seem to) prove that its conditions are satisfied, regularly the above alternative route of writing down the constant expectation property of the martingale that is the stopped process, and then analysing parts of that expression using the DCT/MCT, still allows you to tinker your way to a useful conclusion.

Exercises

You can now do Exercise 7.1–Exercise 7.3.

7.3.2 The general/non-symmetric case

Having some seen of that (hopefully!) interesting first hitting time violence for symmetric random walks above, a natural question is whether we can still do similar stuff if we drop the assumption of symmetry i.e. if we consider the simple random walk $(X_n)_{n=0,1,\ldots}$ as defined in Equation 7.2 for any $p \in (0,1)$? The answer is a resounding “yes”!

One particularly interesting martingale for the strictly non-symmetric case i.e. $p \not=1/2$ is De Moivre’s martingale. Setting $q:=1-p$, it is the process $(Z_n)_{n=0,1,\ldots}$ given by \[Z_n := \left( \frac{q}{p} \right)^{X_n} \quad \text{for all $n=0,1,\ldots$,}\] living on the same filtered probability space where $(X_n)_{n=0,1,\ldots}$ lives (as introduced at the start of Section 7.3). Note that $Z_0=(q/p)^0=1$, and that while the random walk $(X_n)_{n=0,1,\ldots}$ visits (positive and negative) integers i.e. $\Z$, $(Z_n)_{n=0,1,\ldots}$ visits points of the form $(q/p)^k$ for $k \in \Z$. Note that these are all strictly positive, so $(Z_n)_{n=0,1,\ldots}$ is a strictly positive process.

Further it is worth observing that we can make a distinction between two different cases:

if $p \in (0,1/2)$ i.e. $q/p>1$, then $X_n$ is $>0$, $=0$ or $<0$ if and only if $Z_n$ is $>1$, $=1$ or $\in (0,1)$ resp.
if $p \in (1/2,1)$ i.e. $q/p \in (0,1)$, then it is the other way around: $X_n$ is $>0$, $=0$ or $<0$ if and only if $Z_n$ is $\in (0,1)$, $=1$ or $>1$ resp.

We look at this process, and what it can tell us about first hitting times for $(X_n)_{n=0,1,\ldots}$, in a bit more detail in Exercise 7.4.

Exercises

You can now do Exercise 7.4.

7.4 Wald’s martingale

In this section we look at first hitting times for general random walks (i.e. not necessarily only simple ones) using the fascinating Wald’s martingale. The stage is a filtered probability space and a random walk $(X_n)_{n=0,1,\ldots}$ as introduced at the start of Section 7.3, except that we now drop the assumption that it is “simple”: we allow the distribution of $Y$, of which $Y_1, Y_2, \ldots$ are independent copies, to be anything as long as it is not (a.s.) constant (to avoid trivialities).

Further fix some $a,b \in (0,\infty)$ and consider \[\tau = \inf \{ n=0,1,\ldots \, | \, X_n \leq -a \text{ or } X_n \geq b\}. \tag{7.17}\] This is a stopping time as it is the first hitting time of the Borel set $(-\infty,a] \cup [b,\infty)$ (cf. Example 6.1).

It is important to appreciate the difference with the simple random walk here. For any random walk, from time $n$ to time $n+1$ the process moves up or down by the amount $X_{n+1}-X_n=Y_{n+1}$ (cf. Equation 7.2). In the case of a simple random walk this is either $+1$ or $-1$, ensuring that the process only visits integers and in particular that it can’t cross an integer level without actually visiting it (and hence we can work with a stopping time of the form Equation 7.4). However for the general random walk we consider here, the movement $Y_{n+1}$ can take any (real) value in principle — well, we have assumed it to be bounded but that’s it. This process could hence very well move to a level below $-a$ (resp. above $b$) without actually visiting $-a$ (resp. $b$) first. So if we want to look at the first time that the process escapes the strip bounded below by $-a$ and bounded above by $b$ then we need the formulation Equation 7.17 rather than Equation 7.4.

Ok. Our goal in this section is to try and say something about the properties/distribution of this $\tau$ in Equation 7.17, similar to what we did in the previous section. We’ll do this via Wald’s martingale which we’ll introduce in a moment and again Doob’s optional stopping theorem if we can.

First though, because we’re now in a new situation with a new process, we basically need to start from scratch. Following the same strategy as in Section 7.3.1, let us first wonder what a path of the random walk can look like until we reach time $\tau$. Similarly to the three mutually exclusive categories we had at the start of Section 7.3.1, they now look as follows (keeping in mind the difference between our current random walk and the simple random walk):

The path $X_0(\omega), X_1(\omega), \ldots$ enters $(-\infty,-a]$ before it enters $[b,-\infty)$. In this case $\tau(\omega)<\infty$ and $X_{\tau(\omega)}(\omega) \in (-\infty,-a]$;
The path $X_0(\omega), X_1(\omega), \ldots$ enters $[b,\infty)$ before it enters $(-\infty,-a]$. In this case $\tau(\omega)<\infty$ and $X_{\tau(\omega)}(\omega) \in [b,\infty)$;
The path $X_0(\omega), X_1(\omega), \ldots$ never enters neither $(-\infty,-a]$ nor $[b,\infty)$ i.e. it stays forever inside the strip $(-a,b)$. In this case $\tau(\omega)=\infty$ and $X_{\tau(\omega)}(\omega)$ is undefined.

As before we need to start by showing that all the paths in category 3 together make up an event of probability $0$, so that we have $\tau<\infty$ a.s. and that the random variable $X_\tau$ is well defined a.s. The argument is very similar to the one in Section 7.3.1, just a bit more annoying to make precise because we have much less info about our $Y_i$’s. First recall that we have assumed that $Y$ cannot be a.s. constant. Hence by Exercise 7.6, there exists a $h>0$ so that with positive probability $Y$ takes a value larger than $h$ or smaller than $-h$. So over a length of time $(b-a)/h+1$, with positive probability has only upwards movements of at least size $h$, or only negative movements of size less than $-h$. If that happens, the walk either moves upwards by at least $b-a$ or moves downwards by at least $-(b-a)$ and therefore must enter either $(-\infty,-a]$ or $[b,\infty)$. All that matters is that there is a positive probability of this happening, because then on the infinitely long time axis we have an infinite repeat of periods of length $(b-a)/h+1$ so that, again by the “monkey typing Shakespeare” principle, with probability $1$ it will happen eventually.

Good, bit fiddly and annoying, but we know now that category 3 has probability $0$ i.e. that $\tau<\infty$ a.s. We look at two special cases.

To introduce Wald’s martingale, recall from your basic probability courses (and/or Wiki) the concept of moment generating functions (mgf’s). For any random variable $U$ on some probability space, it is defined as the function \[M_U(t) := \E[e^{tU}] \quad \text{for all } t \in \R. \tag{7.18}\] Note that since $e^{tU}>0$, $M_U(t) \in (0,\infty]$ is always well defined but may be $\infty$, except for $t=0$ where we trivially always have $M_U(0)=1$. Note: “$t$” is a standard choice for the argument of this function but, as always with function arguments, is essentially an arbitrary choice — it doesn’t have anything to do with the concept of time that we use as part of our martingale concepts for instance.

The mgf is one of several related transforms (others are e.g. probability generating functions for discrete random variables only, the Laplace transform, the characteristic function etc.) that all do something similar: map random variables (or, more precisely: distributions) to functions, (mostly) in an injective way i.e. two random variables with different distributions get mapped to different functions. In particular:

Lemma 7.1 For a random variable $U$ on some probability space, its moment generating function (mgf) is defined as the function $M_U: \R \to [0,\infty]$ defined by Equation 7.18. It has the following properties:

Mgf’s uniquely determine distributions: for two random variables $U_1, U_2$, if some $a<b \in \R$ exist so that $M_{U_1}(t)=M_{U_2}(t)<\infty$ for all $t \in (a,b)$ then $U_1$ and $U_2$ have the same distribution.
Mgf’s generate moments (indeed): if $h>0$ exists so that $M_U(t)<\infty$ for all $t \in (-h,h)$ then for any $k=1,2,\ldots$ \[\E[U^k]=M_U^{(k)}(0),\] where $M_U^{(k)}$ denotes the $k$-th derivative of $M_U$.

Now, for any $t \in \R$ so that $M_Y(t)<\infty$, consider the process $(Z_n)_{n=0,1,\ldots}$ defined by \[Z_0=1 \quad \text{and} \quad Z_n=\frac{\exp(t X_n)}{M_Y(t)^n} \quad \text{for all } n=1,2,\ldots. \tag{7.19}\] Note that for $t=0$ this is a trivial, constant process so we tend to ignore that particular case. With all the practise you’ve had by now it is not very hard to check that this guy is a martingale, cf. Exercise 7.5. It is commonly known as Wald’s martingale and it can be seen as the discrete time version of a continuous time stochastic process called the stochastic exponential or Doléans-Dade exponential of $X$, an important process in stochastic calculus and financial maths. Also note that this is in fact not a single, but a whole family of martingales: for any $t \in \R$ with $M_Y(t)<\infty$ a different one!

Assume for the sequel (for simplicity, can be relaxed a bit) that $Y$ has a bounded range, say that $\lvert Y \rvert \leq c$ (pointwise) for some $c>0$. As you can readily check this implies that $M_Y(t)<\infty$ for all $t \in \R$. Fix some $t \not=0$ with $M_Y(t) \geq 1$. Observe that for any $n=0,1,\ldots$, as long as $n<\tau$ $X_{n \wedge \tau}$ is inside the strip $(-a,b)$ while at time $n=\tau$ it “jumps” out of that strip, either into $(-\infty,-a]$ or $[b,\infty)$. But due to the bounded range of $Y$, we can “jump” at most a distance $c$ below $-a$ or above $b$ at time $n=\tau$, so that \[\lvert X_{n \wedge \tau} \rvert \leq \max\{a+c,b+c\} \quad \text{for all } n=0,1,\ldots.\] Using that $(Z_n)_{n=0,1,\ldots}$ is a positive process and that $M_Y(t) \geq 1$, this immediately yields for any $n=0,1,\ldots$ \[\begin{align*} \lvert Z_{n \wedge \tau} \rvert &= Z_{n \wedge \tau} \\ &\leq e^{t X_{n \wedge \tau}} \\ &\leq e^{t \lvert X_{n \wedge \tau} \rvert} \\ &\leq e^{t \max\{a+c,b+c\}}, \end{align*} \] so if we define the constant and hence integrable random variable $V:=\exp(t \max\{a+c,b+c\})$ then this (together with $\tau<\infty$ a.s. as already deduced above) shows that we may apply Doob’s optional stopping theorem (cf. Theorem 6.1): \[\E[Z_\tau]=\E[Z_0]=1\] i.e. \[\E \left[ \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right]=1. \tag{7.20}\] Recall that this is derived under the assumptions that $Y$ is not a.s. constant, that $\lvert Y \rvert \leq c$ (pointwise) for some $c>0$, and that $t \not=0$ satisfies $1 \leq M_Y(t) <\infty$.

7.4.1 Case 1: unity value for the mgf of $Y$

Staying within the general random walk framework set out above (incl. satisfying the assumptions listed under Equation 7.20), suppose now that $t_0 \not=0$ exists so that $M_Y(t_0)=1$ (this is satisfied in many cases of interest). Then Equation 7.20 reads \[\E[e^{t_0 X_\tau}]=1.\] Be aware that, compared with the simple random walk case, $X_\tau$ is now less simple: all we can really say is that its range is contained in $(-\infty,-a] \cup [b,\infty)$. This does mean that the events $\{ X_\tau \leq -a\}$ and $\{ X_\tau \geq b\}$ form a partition of $\Omega$ (ignoring the event $\{\tau=\infty\}$) and hence the law of total expectation (cf. Exercise 4.4) yields \[1=\E[e^{t_0 X_\tau}] = \E[e^{t_0 X_\tau} \, | \, X_\tau \leq -a] \P(X_\tau \leq -a) + \E[e^{t_0 X_\tau} \, | \, X_\tau \geq b] \P(X_\tau \geq b)\] i.e. if we define \[\eta_a:=\E[e^{t_0 X_\tau} \, | \, X_\tau \leq -a] \quad \text{and} \quad \eta_b:=\E[e^{t_0 X_\tau} \, | \, X_\tau \geq b]\] then we can express the probability that the random walk enters $(-\infty,-a]$ before it enters $[b,\infty)$ resp. its complement as \[\P(X_\tau \leq -a)=\frac{\eta_b-1}{\eta_b-\eta_a} \quad \text{resp.} \quad \P(X_\tau \geq b)=\frac{1-\eta_a}{\eta_b-\eta_a}. \tag{7.21}\] Clearly these expressions are far from as simple and explicit as the analogue we obtained for the simple random walk in Equation 7.7 but that’s to be expected as sa general random walk is a much more complicated object.

7.4.2 Case 2: simple random walk

Of course it is also very interesting to see what information we can obtain from Wald’s martingale for more specific cases — let’s take the simple (symmetric or asymmetric) random walk as example as discussed in (but it’s not the only possibility!). It turns out that Wald’s martingale can do some powerful stuff, see Exercise 7.7.

Exercises

You can now do Exercise 7.5–Exercise 7.7.

7.5 Some exercises

About the exercises

Each exercise has a (rough) indication of its difficulty, as follows:

*	easier: can be solved by (almost) only using relevant definitions/results,
**	medium: in addition to relevant definitions/results, needs a limited amount of work/creativity,
*******	harder: in addition to relevant definitions/results, needs a larger amount of work/serious creativity,
💀	warning: might make your brain hurt! These are mainly intended to provide some extra challenge for those of you keen on that and are generally quite hard. You don't need to worry about these too much for exam purposes.

The exam consists of mostly ** and *** level questions, some *, and possibly at most a few marks worth of 💀.

A bit of preaching: it is an incredibly important part of the study process to try and work on the exercises as much as possible. To become a better mathematician/learn new maths (and also to get a good exam mark ;)), above all you need to do it. And yes, of course that includes falling over things, and making mistakes, and getting stuck, and getting frustrated — all part of the game and what you’re supposed to be doing! Your lecturers have done that as well and still do it. What matters is that you don’t let that discourage you and that you make good use of the help and resources available to help you develop your skills. As part of that, many exercises have a hint in a block like this:

Hint (click to view)

Hint!

These are trying to help you on your way if you don’t know where to start or to provide some ideas if you get stuck. In spirit of the above, always have a look at these first and try again before you look at the full solution. (These hints are an extra service that won’t be available in the exam I’m afraid ;).)

Of course, we have our classes and there’s office hours, email etc. as well — I’m at any time very happy to help you with any questions you may have, and you should please never feel that any question is “too dumb” to ask!

Full/detailed solutions for the exercises will become available, just immediately below the exercises, after our Friday tutorial.

Exercise 7.1 [*] In Section 7.3.1 we came to the following conclusion: for a symmetric simple random walk starting from $0$, the probability of hitting $-a$ before $b$ is $b/(a+b)$ and vice versa the probability of hitting $b$ before $-a$ is $a/(a+b)$.

Briefly think about what happens to these two probabilities if you let $b \to \infty$, and briefly explain how that makes sense.

We also saw, with $\tau$ denoting the first hitting time of either $-a<0$ or $b>0$, that $\E[\tau]=ab$. What happens to this expression as $b$ increases, and how does that make sense?

Full solution (click to view)

Clearly if we let $b \to \infty$ then \[\frac{b}{a+b}=\frac{1}{a/b+1} \to \frac{1}{0+1}=1\] and \[\frac{a}{a+b} \to \frac{a}{\infty}=0\] (can also use that they are each other’s complements of course). If we let $b$ go to $\infty$ then it takes ever longer to reach the level $b$ and so there is ever more time for the process to reach $-a$ first. Apparently this translates into the probs going to $0$ and $1$ resp. Little warning: this is not necessarily true for any old stochastic process. As a trivial example (there are other, non-trivial ones as well): if we consider $X_n=n$, then obviously it will always hit any $b$, no matter how large, before hitting any negative level!

For the expectation, obviously $\E[\tau]=ab$ is increasing in $b$. The explanation is that the larger $b$ is, the longer it will take a path to reach either $-a$ or $b$. More precisely: if we compare any $0<b_1<b_2$, then paths that reach $-a$ before $b_1$ will also still reach $-a$ before $b_2$ and the stopping time won’t change. However for paths that reach $b_1$ before $-a$, these will now carry on for longer until they reach either $b_2$ or $-a$. So on one event $\tau$ doesn’t change, while on its complement $\tau$ gets larger, altogether resulting in a larger value for its expectation.

Note: these types of (intuitive) arguments, wondering whether an expression you computed “behaves as you would expect” can be quite useful especially after a lengthy computation: if it does something that seems intuitively strange/wrong, then chances are that you made a mistake somewhere along the line!

Exercise 7.2 [**] Consider again the symmetric simple random walk from Section 7.3.1. We have seen the following:

if we let the random walk start from level $0$ at time $n=0$ and define $\tau$ to be the first hitting time of $-a$ or $b$ for integers $a,b>0$, then $\E[\tau]=ab$ (cf. Equation 7.16);
if we let the random walk start from level $x$ (positive integer) at time $n=0$ and define $T$ to be the first hitting time of $0$, then $\E[T]=\infty$ (cf. Exercise 6.7).

Show that ii follows from i.

Hint (click to view)

Not much maths is needed for this one. First convince yourself that if we let a random walk start from $x>0$ and wait until it first hits $0$, we may just as well let the random walk start from $0$ and wait until it first hits $-x$. If you’re not sure how to see this, compare Equation 7.22 with Equation 7.2. Now figure out how you can say something about the first hitting time of $-x$ using the stopping times from statement i.

Full solution (click to view)

The key point is to realise that if we let the random walk start from some positive integer $x$ i.e. \[X_0=x \quad \text{and} \quad X_n=x+\sum_{i=1}^n Y_i \quad \text{for all $n=1,2,\ldots$} \tag{7.22}\] then compared to starting from $0$ the effect is simply that every path is shifted up by an amount $x$. This is intuitively probably clear but can also be seen by just comparing Equation 7.22 with Equation 7.2.

Consequently, we can simply equivalently reformulate statement ii as follows:

ii’. if we let the random walk start from level $0$ at time $n=0$ and define $T$ to be the first hitting time of $-x<0$, then $\E[T]=\infty$.

Now, after this translation the problem is not so hard anymore. We are looking at the same random walks in statements i and ii’. For any integer $b>0$, let $\kappa$ be the first hitting time of $-x$ or $b$. Then statement i shows that $\E[\kappa]=bx$. Further, it is clear that $\kappa \leq T$: if the random walk hits $-x$ before it hits $b$ then $\kappa$ and $T$ are equal, while if it hits $b$ before it hits $-x$ then $\kappa$ is smaller than $T$. By non-negativity of expectations (cf. Proposition 3.6 ii) this also means that \[\E[T] \geq \E[\kappa]=bx. \tag{7.23}\] But since this argument and Equation 7.23 in particular holds for any integer $b>0$, indeed it follows that $\E[T]=\infty$.

Exercise 7.3 [***] Let $(X_n)_{n=0,1,\ldots}$ be the symmetric simple random walk from Section 7.3.1 as given by Equation 7.2 on some filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_n)_{n=0,1,\ldots},\P)$ with $(\mathcal{F}_n)_{n=0,1,\ldots}$ the natural filtration of $(X_n)_{n=0,1,\ldots}$. Fix some integers $a,b>0$ and consider the stopping time \[\tau=\inf \{ n=0,1,\ldots \, | \, X_n=-a \text{ or } X_n=b\}.\] Recall that in Section 7.3.1 we have already derived a number of properties of this stopping time, including that $\tau<\infty$ a.s.

Now, further define the process $(Z_n)_{n=0,1,\ldots}$ given by $Z_n=X_n^3-3nX_n$ for all $n=0,1,\ldots$.

Show that $(Z_n)_{n=0,1,\ldots}$ is a martingale.
Using part i, Equation 7.7 & Equation 7.9, show that \[\E[\tau X_\tau]=\frac{ab(b-a)}{3}.\]
Finally, using part ii and Equation 7.9, compute both \[\E[\mathbf{1}_{\{ X_\tau=-a\}} \tau] \quad \text{and} \quad \E[\mathbf{1}_{\{ X_\tau=b\}} \tau].\]

Hint (click to view)

For i, as usual you’ll need to check all three conditions from Definition 5.2. To get the martingale property sorted a bit of algebra is needed, but you can use the same strategy as we used for the process $(Z_n)_{n=0,1,\ldots}$ we looked at in Section 7.3.1.

For ii, apply Doob’s optional stopping theorem (cf. Theorem 6.1) to $(Z_n)_{n=0,1,\ldots}$, and observe that $X_\tau^3$ is a simple random variable with only two possible values: $-a^3$ and $b^3$, with known probabilities (cf. Equation 7.7). This allows to work out $\E[X_\tau^3]$ and derive the result. If you’re struggling to find an integrable upperbound for $\lvert Z_{n \wedge \tau} \rvert$ for all $n=0,1,\ldots$ as needed in Doob’s optional stopping theorem, keep in mind the particularly simple nature of $X_{n \wedge \tau}$ and $X_{n \wedge \tau}^3$, as well as that Equation 7.9 tells us that the expectation of $\tau$ is finite.

For iii finally, bit tricky to see how to attack this I imagine! First convince yourself that, again by that especially simple nature of $X_\tau$, we have that \[\mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}}=1 \quad \text{(a.s.)}.\] Then take Equation 7.9, write it as $\E[1 \cdot \tau]=ab$, replace that factor $1$ by the sum of these two indicator functions and work it out a little bit. Use that same trick in the equation we derived in part ii. This gives you two equations in two unknowns (=the two expressions we are asked to compute the value of) which can be easily solved!

Full solution (click to view)

For part i, as usual we need to check all three conditions from Definition 5.2:

Adapted: for any $n=0,1,\ldots$, since $Z_n$ is a (nice) function of $X_n$ and $X_n$ is $\mathcal{F}_n$-measurable, $Z_n$ is also $\mathcal{F}_n$-measurable.
Integrable: fix some $n=0,1,\ldots$. Note that since each $Y_i$ takes only the values $-1$ or $1$, $X_n$ has as smallest possible value $-n$ and as largest possible value $n$. That is, $\lvert X_n \rvert \leq n$ and $\lvert X_n^3 \rvert \leq n^3$. Hence using the triangle inequality as well as $\lvert xy \rvert = \lvert x \rvert \lvert y \rvert$ (plus linearity of expectations) we can deduce that \[\begin{align*} \E[\lvert Z_n \rvert] &= \E[ \lvert X_n^3-3n X_n \rvert] \\ &\leq \E[ \lvert X_n^3 \rvert + 3n \lvert X_n \rvert] \\ &= \E[ \lvert X_n^3 \rvert] + 3n \E[\lvert X_n \rvert] \\ &\leq n^3 + 3n \cdot n \\ &<\infty. \end{align*} \]
Mg property: fix some $n=0,1,\ldots$. We follow the same idea as in Section 7.3.1: start with working out $Z_{n+1}$ a bit to find a term $Z_n$ in there and then pull it all through a conditional expectation (but obviously whatever you yourself find the easiest approach — of course you can also just start with $\E[Z_{n+1} \, | \, \mathcal{F}_n]$, and do the below algebra all inside the conditional expectation e.g.). Indeed we can write, using that by construction $X_{n+1}=X_n+Y_{n+1}$ \[\begin{align*} Z_{n+1} &= X_{n+1}^3 -3(n+1) X_{n+1} \\ &= (X_n+Y_{n+1})^3 -3(n+1)(X_n+Y_{n+1}) \\ &= X_n^3 +3X_n^2 Y_{n+1} + 3X_n Y_{n+1}^2 + Y_{n+1}^3 -3(n+1)(X_n+Y_{n+1}) \\ &= X_n^3 -3nX_n +3X_n^2 Y_{n+1} + 3X_n Y_{n+1}^2 + Y_{n+1}^3 -3 X_n -3(n+1)Y_{n+1} \\ &= Z_n +3X_n^2 Y_{n+1} + 3X_n Y_{n+1}^2 + Y_{n+1}^3 -3 X_n -3(n+1)Y_{n+1}. \end{align*} \] So to work out $\E[Z_{n+1} \, | \, \mathcal{F}_n]$ we plug the whole right hand side in this conditional expectation, and using linearity we can look at the individual terms. We get that \[\E[3X_n^2 Y_{n+1} \, | \, \mathcal{F}_n]=3X_n^2 \E[Y_{n+1} \, | \, \mathcal{F}_n]=3X_n^2 \E[Y_{n+1} ]=0,\] the first equation using that $X_n$ is $\mathcal{F}_n$-meas with Theorem 4.2 v, and the second that $Y_{n+1}$ is independent of $\mathcal{F}_n$ with Theorem 4.2 vi plus that $Y_{n+1}$ taking values $\pm 1$ with probs $1/2$ each meaning it has expectation $0$. By the same line of argument we get that \[\E[3X_n Y_{n+1}^2 \, | \, \mathcal{F}_n]=3X_n \E[Y_{n+1}^2 \, | \, \mathcal{F}_n]=3X_n \E[Y_{n+1}^2 ]=3X_n,\] since $Y_{n+1}^2$ takes value $1$ with prob $1$. Further, again using that $Y_{n+1}$ is independent of $\mathcal{F}_n$ with Theorem 4.2 vi plus the distribution of $Y_{n+1}$: \[\E[Y_{n+1}^3 \, | \, \mathcal{F}_n]=\E[Y_{n+1}^3]=0\] and \[\E[3(n+1)Y_{n+1} \, | \, \mathcal{F}_n]=3(n+1)\E[Y_{n+1}]=0.\] Finally, using that $Z_n$ and $X_n$ are both $\mathcal{F}_n$-meas with Theorem 4.2 ii, plugging this all back we ultimately arrive at \[\E[Z_{n+1} \, | \, \mathcal{F}_n]=Z_n+3X_n-3X_n=Z_n,\] as we were hoping to show.

For part ii, as the question points out to use part i and we want to prove something involving $\tau$, our first impulse should be to look at applying Doob’s optional stopping theorem (cf. Theorem 6.1) to $(Z_n)_{n=0,1,\ldots}$. In the second part of Section 7.3.1 we were in a not dissimilar situation with the martingale there, but we have now one decisive advantage: we know already, from Equation 7.9, that $\E[\tau]=ab<\infty$. This allows us to find the integrable upperbound that we need for Doob’s optional stopping theorem. Indeed recall that as discussed in Section 7.3.1, by def of $\tau$, for all $n=0,1,\ldots$ we know that $X_{n \wedge \tau}$ is bounded below by $-a$ and above by $b$. Further (trivially) $n \wedge \tau \leq \tau$. So using the triangle inequality and that $\lvert xy \rvert = \lvert x \rvert \lvert y \rvert$, we can write \[\begin{align*} \lvert X_{n \wedge \tau}^3 -3(n \wedge \tau) X_{n \wedge \tau} \rvert &\leq \lvert X_{n \wedge \tau} \rvert^3 +3(n \wedge \tau) \lvert X_{n \wedge \tau} \rvert \\ &\leq \left( \max\{-a,b\} \right)^3 + 3 \tau \max\{-a,b\}. \end{align*} \] So if we define the random variable \[V:=\left( \max\{-a,b\} \right)^3 + 3 \tau \max\{-a,b\}\] then it is an upperbound as needed in Doob’s optional stopping theorem i.e. it holds that $\lvert Z_{n \wedge \tau} \rvert \leq V$ for all $n=0,1,\ldots$ (aside: note that this upperbound is not allowed to depend on $n$). Further it is integrable since \[\begin{align*} \E[V] &= \E \left[ \left( \max\{-a,b\} \right)^3 + 3 \tau \max\{-a,b\} \right] \\ &= \left( \max\{-a,b\} \right)^3 + 3 \max\{-a,b\} \E[\tau] \\ &= \left( \max\{-a,b\} \right)^3 + 3 \max\{-a,b\} ab \\ &<\infty, \end{align*} \] using Equation 7.9 and linearity of expectations. The only other thing we need is that $\tau<\infty$ a.s., but we had already shown that in Section 7.3.1. So, it now follows from Doob’s optional stopping theorem that \[\E[Z_\tau]=\E[Z_0]=0\] i.e. that \[\E[X_{\tau}^3]-3\E[\tau X_{\tau}]=0. \tag{7.24}\] Finally, recall from Section 7.3.1 and in particular also Equation 7.7 that $X_\tau$ is a simple discrete random variable, taking values $-a$ and $b$ with probs $b/(a+b)$ and $a/(a+b)$ resp., so $X_{\tau}^3$ takes values $-a^3$ and $b^3$ with these same probabilities, and we can work out Equation 7.24: \[\begin{align*} \E[\tau X_{\tau}] &= \frac{1}{3} \E[X_{\tau}^3] \\ &= \frac{1}{3} \left( -a^3 \cdot \frac{b}{a+b} + b^3 \cdot \frac{a}{a+b} \right) \\ &= \frac{1}{3} \frac{ab(-a^2+b^2)}{a+b} \\ &= \frac{1}{3} \frac{ab(b+a)(b-a)}{a+b} \\ &= \frac{ab(b-a)}{3}. \end{align*} \]

Note: as an aside, if you can’t find an integrable upperbound to apply Doob’s optional stopping theorem you could still apply the same ideas we used in the second part of Section 7.3.1 and try to make that work for you. That would have been a viable route in this case as well — though it is more work of course and inevitably also for that route you would need the fact that $\E[\tau]<\infty$ which was the crucial ingredient in the above as well.

For part iii, here is the crucial observation. As already discussed in Section 7.3.1, ignoring $\omega \in \Omega$ with $\tau(\omega)=\infty$ which we can safely do here because we’re only interested in expectations, the random variable $X_\tau$ equals either $-a$ or $b$. That means that the three events $\{ X_\tau=-a\}$, $\{ X_\tau=b\}$ and $\{\tau=\infty\}$ form a partition of $\Omega$. This also means that we have a.s. \[\mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}}=1\] (because for any $\omega$ with $\tau(\omega)<\infty$, exactly one of these two indicator functions equals $1$ while the other equals $0$). This allows us to write the following, also using Equation 7.9 and linearity of expectations: \[\begin{align*} ab &= \E[\tau] \\ &= \E[1 \cdot \tau] \\ &= \E \left[ \left( \mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}} \right) \tau \right] \\ &= \E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau \right] + \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau \right]. \end{align*} \tag{7.25}\] We can pull a similar trick with the result from part ii: \[\begin{align*} \frac{ab(b-a)}{3} &= \E[\tau X_\tau] \\ &= \E \left[ \left( \mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}} \right) \tau X_\tau \right] \\ &= \E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau X_\tau \right] + \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau X_\tau \right] \\ &= \E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau \cdot -a \right] + \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau \cdot b \right] \\ &= -a \E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau \right] + b \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau \right]. \end{align*} \tag{7.26}\] So if we now write, for simplicity \[A:=\E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau \right] \quad \text{and} \quad B:=\E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau \right],\] then Equation 7.25 & Equation 7.26 together translate into a system of two (linear) equations for these two unknowns: \[\begin{align*} A+B &=ab \\ -aA+bB &=ab(b-a)/3. \end{align*} \] Using your favourite method (you could e.g. first multiply both sides of the first equation by $-b$ and then use that the sum of both left hand sides must equal the sum of both right hand sides to find an expression for $A$) you can solve this system for $A$ and $B$ to ultimately arrive at \[\E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \tau \right]=\frac{2ab^2+a^2b}{3(a+b)} \quad \text{and} \quad \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \tau \right]=\frac{2a^2b+ab^2}{3(a+b)}.\]

Exercise 7.4 [**/***] Consider the simple random walk $(X_n)_{n=0,1,\ldots}$ on the filtered probability space as introduced at the start of Section 7.3, with $p \not= 1/2$ i.e. it is not symmetric. Further let $(Z_n)_{n=0,1,\ldots}$ be De Moivre’s martingale as introduced in Section 7.3.2, living on that same filtered probability space.

Show that $(Z_n)_{n=0,1,\ldots}$ is, in fact, indeed a martingale.
Fix some integers $a,b>0$. Using part i, show that the probability that the random walk hits $-a$ before it hits $b$ equals \[\frac{1-(q/p)^{b}}{(q/p)^{-a}-(q/p)^{b}}.\]
Suppose that you visit a casino to repeatedly play the same game. Each time you play you either win £1 or lose £1, with probabilities $0.4$ and $0.6$ resp. You enter the casino with £$x$ in your pocket, for some $x \in \{1,2,\ldots,99\}$. You stop playing as soon as you have £100, or if you get kicked out of the casino because you have no money left (you can’t borrow any money or so). You want to have a chance of at least 50% of reaching the £100 (rather than getting kicked out). How much money should you bring with you i.e. how large should $x$ be?

Hint (click to view)

For i, as usual you’ll need to check all three conditions from Definition 5.2. For the integrability, keep the distinction between $p \in (0,1/2)$ and $p \in (1/2,1)$ in mind as discussed in Section 7.3.2. For the mg property you could again use that $X_{n+1}=X_n+Y_{n+1}$ and some of our favourite properties of conditional expectations.

To get the martingale property sorted a bit of algebra is needed, but you can use the same strategy as we used for the process $(Z_n)_{n=0,1,\ldots}$ we looked at in Section 7.3.1.

For ii, apply Doob’s optional stopping theorem (cf. Theorem 6.1) and working out $\E[Z_\tau]$ should do the trick.

For iii, think about/visualise how the problem can be (quite easily!) reformulated so that $x$ should satisfy an inequality involving the probability from part ii.

Full solution (click to view)

For part i it is the usual checking of all three conditions from Definition 5.2:

Adapted: for any $n=0,1,\ldots$, since $Z_n$ is a (nice) function of $X_n$ and $X_n$ is $\mathcal{F}_n$-measurable, $Z_n$ is also $\mathcal{F}_n$-measurable.
Integrable: fix some $n=0,1,\ldots$. Note that since each $Y_i$ takes only the values $-1$ or $1$, $X_n$ has as smallest possible value $-n$ and as largest possible value $n$. So the largest possible value that $Z_n$ can have is $(q/p)^n$ if $p<1/2$ i.e. $q/p>1$ and $(q/p)^{-n}$ if $p>1/2$ i.e. $q/p \in (0,1)$. Further $Z_n$ is strictly positive. Hence \[\E[\lvert Z_n \rvert]=\E[Z_n] \leq \max \left\{ \left( \frac{q}{p} \right)^n, \left( \frac{q}{p} \right)^{-n} \right\} <\infty.\]
Mg property: fix some $n=0,1,\ldots$. This process has a “multiplicative form” while the other processes derived from random walks so far had an “additive form”, nevertheless again (as previously done in this chapter) using that $X_{n+1}=X_n+Y_{n+1}$ and some of the well used properties of conditional expectations get us there pretty quickly: \[\begin{align*} \E[Z_{n+1} \, | \, \mathcal{F}_n] &= \E \left[ \left. \left( \frac{q}{p} \right)^{X_{n+1}} \, \right| \, \mathcal{F}_n \right] \\ &= \E \left[ \left. \left( \frac{q}{p} \right)^{X_n+Y_{n+1}} \, \right| \, \mathcal{F}_n \right] \\ &= \left( \frac{q}{p} \right)^{X_n} \E \left[ \left. \left( \frac{q}{p} \right)^{Y_{n+1}} \, \right| \, \mathcal{F}_n \right] \\ &= Z_n \E \left[ \left( \frac{q}{p} \right)^{Y_{n+1}} \right], \end{align*} \] where the third equation uses that $(q/p)^{X_n}$ is a (nice) function of $X_n$ and therefore $\mathcal{F}_n$-meas since $X_n$ is so that we can apply Theorem 4.2 v, and the fourth uses that $Y_{n+1}$ is independent of $\mathcal{F}_n$ so that we can apply Theorem 4.2 vi. Since $Y_{n+1}$ is a simple discrete random variable taking values $1$ and $-1$ with probs $p$ and $q=1-p$ resp., we can use the standard formula for the expectation of (a function of) a discrete random variable (cf. Equation 2.6 via Proposition 3.5) to compute \[\begin{align*} \E \left[ \left( \frac{q}{p} \right)^{Y_{n+1}} \right] &=\left( \frac{q}{p} \right)^{-1} \P(Y_{n+1}=-1) + \left( \frac{q}{p} \right) \P(Y_{n+1}=1) \\ &= \left( \frac{q}{p} \right)^{-1} q + \left( \frac{q}{p} \right) p \\ &= 1 \end{align*} \] (recall that $q=1-p$) so that it indeed follows that $\E[Z_{n+1} \, | \, \mathcal{F}_n]=Z_n$.

For part ii, define \[\tau=\inf \{ n=0,1,\ldots \, | \, X_n=-a \text{ or } X_n=b\}.\] We know that this is a stopping time since it is a first hitting time (cf. Example 6.1). Further by exactly the same arguments as in Section 7.3.1 for the symmetric random walk, it follows that $\tau<\infty$ a.s. Observe that (of course) we may equally well write \[\tau=\inf \left\{ n=0,1,\ldots \, \left| \, Z_n=\left( \frac{q}{p} \right)^{-a} \text{ or } Z_n=\left( \frac{q}{p} \right)^{b} \right. \right\}.\]

Now, since we are told to use the De Moivre’s martingale and the goal is to derive a property involving the stopping time $\tau$, it makes sense to try and use Doob’s optional stopping theorem (cf. Theorem 6.1) again. In order to satisfy the second condition of that theorem (we have no hope of satisfying the first one), we’re looking for an integrable upperbound on $\lvert Z_{n \wedge \tau} \rvert=Z_{n \wedge \tau}$ for all $n=0,1,\ldots$ (since $(Z_n)_{n=0,1,\ldots}$ is a positive process we can drop the absolute values). However this is not very difficult: by def of $\tau$, the stopped process $(Z^\tau_n)_{n=0,1,\ldots}$ starts at level $Z_0=1$ and then as time goes on drifts in the strip with bounds $(q/p)^{-a}$ and $(q/p)^{b}$ (it depends on whether $p<1/2$ or $p>1/2$ which of these two is the larger one) until it hits either of these bounds at time $\tau$ and then stays stuck there. That is to say, we have that \[Z_{n \wedge \tau} \leq \max \left\{ \left( \frac{q}{p} \right)^{-a}, \left( \frac{q}{p} \right)^{b} \right\} \quad \text{for all } n=0,1,\ldots\] and hence if we define the random variable $V$ as the above right hand side then it is integrable (trivially, as it is constant) and we have shown that we may indeed apply Doob’s optional stopping theorem.

So we have that \[\E[Z_\tau]=\E[Z_0]=1. \tag{7.27}\] Similar to previously, since $Z_\tau$ is a relatively simple random variable we can work out its expectation pretty easily. On the event $\{\tau<\infty\}$ i.e. ignoring $\omega \in \Omega$ with $\tau(\omega)=\infty$, which we can do because we know that $\{\tau<\infty\}$ is an almost sure event, we see that $Z_\tau$ is a simple discrete random variable with only two possible values: $(q/p)^{-a}$ and $(q/p)^{b}$. Denoting by $\bar{p}$ the probability that $Z_\tau$ takes the value $(q/p)^{-a}$, using the standard formula for the expectation of a discrete random variable we can compute \[\E[Z_\tau]=\left( \frac{q}{p} \right)^{-a} \bar{p} + \left( \frac{q}{p} \right)^{b} (1-\bar{p})\] and plugging this back into Equation 7.27 yields \[\bar{p}=\frac{1-(q/p)^{b}}{(q/p)^{-a}-(q/p)^{b}}. \tag{7.28}\]

Aside (not part of the question): intuitively you would expect that if you let $p$ approach $1/2$ that we tend towards the case of a symmetric random walk i.e. that Equation 7.28 converges to the $b/(a+b)$ from Equation 7.7. This is not so obvious though: if we define $f(x)=(1-x^b)/(x^{-a}-x^b)$, then $p$ approaching $1/2$ in Equation 7.28 is the same as $x$ approaching $1$ in $f(x)$. Both numerator and denominator in $f$ tend to $0$ as $x \to 1$. One possibility is to use l’Hopital’s rule to derive that indeed $\lim_{x \to 1} f(x)=b/(a+b)$ — recall that you could e.g. write \[x^c=e^{\log(x^c)}=e^{c \log(x)}\] if you’re not sure how to differentiate such expressions.

For part iii, to translate this into maths, we can use the (non-symmetric) random walk $(X_n)_{n=0,1,\ldots}$ we’re looking at in this question to model the accumulated profits over time, where we then should set $p=0.4$ and $q=0.6$. We can just let it start from level $0$ at time $n=0$, and then we run out of money as soon as the walk hits $-x$ while we have achieved the £100 if the walk hits $100-x$. So we’re looking to find $x$ so that the probability of hitting $100-x$ before hitting $-x$ is at least $0.5$. Or, equivalently, looking at the prob we expressed in part ii, to find $x$ so that the probability of hitting $-x$ before hitting $100-x$ is at most $0.5$.

Plugging this into the expression we found in part ii, we’re looking to solve $x$ from \[\frac{1-(0.6/0.4)^{100-x}}{(0.6/0.4)^{-x}-(0.6/0.4)^{100-x}} \leq 0.5.\] A little bit of algebra allows to rewrite this e.g. as \[\left( \frac{3}{2} \right)^x \geq 0.5 + 0.5 \left( \frac{3}{2} \right)^{100}\] so that \[x \geq \frac{\log(0.5+0.5 (3/2)^{100})}{\log(3/2)} \approx 98.3.\] So, we conclude that you need to bring £99 to have an at least 50% chance of making it to £100 before running out of money… That’s just how bad a position the 0.4 vs 0.6 prob of winning vs losing per game puts you in!

Exercise 7.5 [**] Verify that the process defined in Equation 7.19 is indeed a martingale.

Hint (click to view)

Obviously it’s a matter of checking all three conditions from Definition 5.2 again. For the integrability, note that $Z_n$ is a positive rv and you could plug in $X_n=Y_1+\ldots+Y_n$ while also making use of Proposition 3.6 viii. For the mg property, again using that $X_{n+1}=X_n+Y_{n+1}$ as well as properties of conditional expectations should get you there without too much hassle!

Full solution (click to view)

Here we go again: checking all three conditions from Definition 5.2!

Adapted: for any $n=0,1,\ldots$, since $Z_n$ is a (nice) function of $X_n$ and $X_n$ is $\mathcal{F}_n$-measurable, $Z_n$ is also $\mathcal{F}_n$-measurable.
Integrable: fix some $n=1,2,\ldots$. Recall from Equation 7.19 that $M_Y(t)<\infty$. Note that since $Z_n$ is positive (as is clear from its def), we can write \[\E[\lvert Z_n \rvert]=\E[Z_n]=\E \left[ \frac{\exp(tX_n)}{M_Y(t)^n} \right]=\frac{1}{M_Y(t)^n} \E[e^{tX_n}]\] (the final step of course by linearity of expectations). Plugging in the def of $X_n$ (cf. Equation 7.2) and recalling that the $Y_i$’s are independent and all copies of $Y$ hence all share the same mgf $M_Y$ we can work out \[\begin{align*} \E[e^{tX_n}] &= \E[e^{t(Y_1+\ldots+Y_n)}] \\ &= \E[e^{tY_1} \cdot \ldots \cdot e^{tY_n}] \\ &= \E[e^{tY_1}] \cdot \ldots \cdot \E[e^{tY_n}] \\ &= M_Y(t)^n < \infty, \end{align*} \] where the third equation uses Proposition 3.6 viii.
Mg property: fix some $n=0,1,\ldots$. We can compute, again using that $X_{n+1}=X_n+Y_{n+1}$: \[\begin{align*} \E[Z_{n+1} \, | \, \mathcal{F}_n] &= \E \left[ \left. \frac{\exp(tX_{n+1})}{M_Y(t)^{n+1}} \, \right| \, \mathcal{F}_n \right] \\ &= \E \left[ \left. \frac{\exp(tX_{n}) \exp(tY_{n+1})}{M_Y(t)^{n} M_Y(t)} \, \right| \, \mathcal{F}_n \right] \\ &= \E \left[ \left. Z_n \frac{\exp(tY_{n+1})}{M_Y(t)} \, \right| \, \mathcal{F}_n \right] \\ &= Z_n \frac{1}{M_Y(t)} \E[e^{tY_{n+1}}] \\ &= Z_n, \end{align*} \] where the fourth equation uses linearity to bring $1/M_Y(t)$ out of the conditional expectation, that $Z_n$ is $\mathcal{F}_n$-measurable with Theorem 4.2 v, and that $Y_{n+1}$ is independent of $\mathcal{F}_n$ with Theorem 4.2 vi; and the fifth equation that by def $M_Y(t)=\E[e^{tY_{n+1}}]$.

Exercise 7.6 [**] Let $X$ be a random variable on some probability space. Show that the following two statements are equivalent:

$\P(X=0)=1$,
for all $h>0$ it holds that $\P(\lvert X \rvert>h)=0$.

Hint (click to view)

You’ll agree that “i implies ii” is pretty obvious right? For the other direction there are a few ways. One possibility is to argue via expectations. Takes only a few lines and the only properties you need are Proposition 3.6 ii & iii!

Full solution (click to view)

It is clear that i implies ii. Assume now that ii holds. To derive i, you could for instance approximate the event $\{X=0\}$ by events $\{ \lvert X \rvert>h \}$, we have used such an argument before a few times in previous chapters. However it is maybe nice to also show another possible way, by making good use of the properties of the expectation operator as discussed in Proposition 3.6. Note that since (trivially) $\lvert X \rvert \geq 0$, also $\E[\lvert X \rvert]\geq 0$ (Proposition 3.6 ii). If $h>0$ is such that $\P(\lvert X \rvert>h)=0$ i.e. $\P(\lvert X \rvert \leq h)=1$ then also $\E[\lvert X \rvert]\leq h$ (again by Proposition 3.6 ii). So it follows from ii that $\E[\lvert X \rvert]\leq h$ for all $h>0$ while also $\E[\lvert X \rvert]\geq 0$. Hence it must be the case that $\E[\lvert X \rvert]=0$. It now follows from Proposition 3.6 iii that $X=0$ a.s.

Exercise 7.7 [**/***] Let’s pick up the situation from Section 7.4.2: $(X_n)_{n=0,1,\ldots}$ is an asymmetric simple random walk with $Y$ taking values $1$ and $-1$ with probability $p \in (0,1)$ but $p \not= 1/2$ and $q:=1-p$ resp. For integers $a,b>0$ consider the stopping time $\tau$ for which we can now use either Equation 7.4 or Equation 7.17 (convince yourself that these are indeed equivalent now!). The goal of this exercise is to investigate what Wald’s martingale can teach us about $\tau$ in this setting (in addition to what we know already from Section 7.3 and Exercise 7.4).

Work out the mgf $M_Y$ of $Y$ and determine $t \not=0$ so that $M_Y(t)=1$ (hint: with a simple substitution, the equation $M_Y(t)=1$ becomes a basic quadratic equation). What fails here if we were working with a symmetric random walk?
Using the value found in part i as your $t_0$, work out the first probability in Equation 7.21, and verify that it is equal to the one we already found in Exercise 7.4 ii using De Moivre’s martingale (as it should be!).

Now let’s reveal the true power of Wald’s martingale! It’s nice to observe that part of what makes the below possible is that Wald’s martingale is actually a whole family of them, because we can vary the value of $t$. So you would kind of expect a whole lot of derivative power from it!

Deduce from Equation 7.20 that for any $t \in \R$ with $M_Y(t) \geq 1$ we have that \[e^{-at} \E[\mathbf{1}_{\{ X_\tau = -a \}} M_Y(t)^{-\tau}]+e^{bt} \E[\mathbf{1}_{\{ X_\tau =b \}} M_Y(t)^{-\tau}]=1.\]

Recall that for a random variable, say $K$, with range (contained in) the non-negative integers, we define its probability generating function (pgf) as the function \[G_K(s)=\E[s^K]=\sum_{i=0}^\infty s^i \P(K=i) \quad \text{for all } s \in (0,1)\] Note that the pgf (for such random variables) always exist, has only finite values, and that (similar to the mgf) it uniquely identifies distributions in the sense that if $G_{K_1}(s)=G_{K_2}(s)$ for all $s \in (0,1)$ (or in some open interval in $(0,1)$) then $K_1$ and $K_2$ have the same distribution. Practically, the distribution of $K$ can be extracted from its pgf via \[\P(K=i)=\frac{G_K^{(i)}(0)}{i!} \quad \text{for all } i=0,1,\ldots,\] where $G_K^{(i)}$ denotes the $i$-th derivative of $G_K$.

Since $\tau$, besides a stopping time, is also a random variable taking values in the non-negative integers, it has a pgf $G_\tau$ fully determining its distribution. Using iii, find an expression for $G_\tau$.

Hint (click to view)

For part i, working from the def of the mgf (cf. Lemma 7.1), using the standard formula for the expectation of a (function of a) discrete rv you should be able to compute that $M_Y(t)=pe^t+qe^{-t}$. In order to solve $M_Y(t)=1$, consider using the substitution $s:=e^t$.

For part ii, the key point you need is to observe/argue that conditional on the event $\{X_\tau \leq -a\}$ we simply have that $X_\tau=-a$.

For part iii, show and then make use of the following identity: \[\mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}}=1 \quad \text{(a.s.)}\]

For part iv, that one is a bit tricky if you’re not clear how to attack it. Start by fixing some $s \in (0,1)$ (so that $s^{-1}>1$). Then solve the equation $M_Y(t)=s^{-1}$ (for some inspiration, look back at part i). You should find two solutions. Plugging each of these values into the result from part iii, you should find a system of two equations involving two expressions that we can consider as the unknowns: \[\E[\mathbf{1}_{\{ X_\tau = -a \}} s^\tau] \quad \text{and} \quad \E[\mathbf{1}_{\{ X_\tau = b \}} s^\tau].\] Solve that system (which is linear in these unknowns!) to obtain expressions for these unknowns, and then it is only a small step to obtain an expression for $G_\tau(s)$ from these two guys.

Full solution (click to view)

For part i, from the def Lemma 7.1, simply using the standard formula for the expectation of a (function of a) discrete random variable (cf. Equation 2.6 via Proposition 3.5) yields for any $t \in \R$ \[M_Y(t)=\E[e^{tY}]=e^{t \cdot 1} \P(Y=1)+e^{t \cdot -1} \P(Y=-1)=pe^t+qe^{-t}.\] Note that this shows that $M_Y(t)<\infty$ for all $t \in \R$. To solve the equation \[pe^t+qe^{-t}=1,\] subbing in $s:=e^t$ shows that we may equivalently solve for $s>0$ \[ps^2-s+q=0. \tag{7.29}\] You can now either use the standard formula for quadratic equations, or use the observation that since $M_Y(0)=1$ always holds (cf. Lemma 7.1), $s=e^0=1$ should be a solution (indeed you can easily verify that directly in Equation 7.29 as well) and hence we should be able to factorise \[ps^2-s+q=p(s-1)(s-A),\] where $A$ is the other root of this quadratic equation, and by working out the brackets we find that $A=q/p$ (echos of De Moivre’s martingale here??). So $M_Y(t)=1$ is solved by $t=0$ and $t=\log(q/p)$, and $t=\log(q/p)$ is the solution the question asks for.

Note that if we had $p=1/2$, then also $q=1/2$ and hence $\log(q/p)=0$, so in this case there would not exist a solution to $M_Y(t)=1$ other than $t=0$.

For part ii, we first need to work out $\eta_a$ and $\eta_b$ as defined in Section 7.4.1. Note that since we are now back in the simple random walk situation where at every time step we move $1$ up or down, we also have back that $X_\tau$ is a simple random variable with only possible values $-a$ and $b$ (this random walk can enter $(-\infty,-a]$ only by actually visiting $-a$ etc., as argued before). So conditional on the event $\{X_\tau \leq -a\}$ we simply have that $X_\tau=-a$, and hence \[\begin{align*} \eta_a &= \E[\exp(t_0 X_\tau) \, | \, X_\tau=-a] \\ &=\E[e^{t_0 \cdot -a}] \\ &=e^{-at_0} \\ &=e^{-a \log(q/p)} \\ &=\exp(\log(q/p)^{-a}) \\ &= \left( \frac{q}{p} \right)^{-a} \end{align*} \] and similarly you can compute $\eta_b=(q/p)^b$. Plugging these values into the expression for $\P(X_\tau \leq -a)$ from Equation 7.21 indeed gives the same expression as we found in part ii of Exercise 7.4.

For part iii, note that all conditions listed under Equation 7.20 are satisfied so we do have that \[\E \left[ \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right]=1. \tag{7.30}\] Note that the events $\{\tau=\infty\}$, $\{X_\tau=-a\}$ and $\{X_\tau=b\}$ form a partition of $\Omega$ where the former has probability $0$. Also using Exercise 4.2 we hence have that \[\mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}}=1 \quad \text{(a.s.)}\] and hence we can rewrite the above equation to \[\begin{align*} 1 &= \E \left[ \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right] \\ &= \E \left[ 1 \cdot \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right] \\ &= \E \left[ \left( \mathbf{1}_{\{ X_\tau=-a \}}+\mathbf{1}_{\{ X_\tau=b \}} \right) \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right] \\ &= \E \left[ \mathbf{1}_{\{ X_\tau=-a \}} \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right] + \E \left[ \mathbf{1}_{\{ X_\tau=b \}} \frac{\exp(t X_\tau)}{M_Y(t)^\tau} \right] \\ &= e^{-at} \E[\mathbf{1}_{\{ X_\tau = -a \}} M_Y(t)^{-\tau}]+e^{bt} \E[\mathbf{1}_{\{ X_\tau =b \}} M_Y(t)^{-\tau}], \end{align*} \] where the final equation uses that if $\mathbf{1}_{\{ X_\tau=-a \}}(\omega)=1$ i.e. $X_\tau(\omega)=-a$ we can simplify $\exp(t X_\tau)$ to $e^{-at}$ while otherwise if $\mathbf{1}_{\{ X_\tau=-a \}}(\omega)=0$ the whole term inside the expectation is $0$.

For part iv, I can imagine it’s maybe a tricky one to find your way through on your own/if you don’t realise what a good starting point is! Fix some $s \in (0,1)$, so that $s^{-1}>1$. Let’s first solve (recall $M_Y$ from part i) \[M_Y(t)=s^{-1} \iff pe^t+qe^{-t}=s^{-1}. \tag{7.31}\] Using the same substitution trick as in part i to turn this into a quadratic equation and solving that one using the standard formula, we find that Equation 7.31 has two solutions, say $t_{1,2}=t_{1,2}(s)$ of the form $t_{1,2}(s)=\log(\lambda_{1,2}(s))$ where \[\lambda_1(s)=\frac{1+\sqrt{1-4pqs^2}}{2ps} \quad \text{and} \quad \lambda_2(s)=\frac{1-\sqrt{1-4pqs^2}}{2ps}.\] Observe that since $M_Y(t_1)=M_Y(t_2)=s^{-1}>1$, we may invoke the result from part iii with $t=t_1$ and $t=t_2$ to obtain two equations that both hold: \[e^{-at_{1,2}} \E[\mathbf{1}_{\{ X_\tau = -a \}} M_Y(t_{1,2})^{-\tau}]+e^{bt_{1,2}} \E[\mathbf{1}_{\{ X_\tau =b \}} M_Y(t_{1,2})^{-\tau}]=1.\] Plugging in that $M_Y(t_1)=M_Y(t_2)=s^{-1}$ and $t_{1,2}(s)=\log(\lambda_{1,2}(s))$ this translates to \[\begin{align*} \lambda_1(s)^{-a} \E[\mathbf{1}_{\{ X_\tau = -a \}} s^\tau] + \lambda_1(s)^{b} \E[\mathbf{1}_{\{ X_\tau = b \}} s^\tau] &= 1 \\ \lambda_2(s)^{-a} \E[\mathbf{1}_{\{ X_\tau = -a \}} s^\tau] + \lambda_2(s)^{b} \E[\mathbf{1}_{\{ X_\tau = b \}} s^\tau] &= 1. \end{align*} \] Defining \[P_1(s)=\E[\mathbf{1}_{\{ X_\tau = -a \}} s^\tau] \quad \text{and} \quad P_2(s)=\E[\mathbf{1}_{\{ X_\tau = b \}} s^\tau]\] we can see the above two equations as a system of two simple linear equations in the unknowns $P_1(s)$ and $P_2(s)$, which you can solve using your favourite method to yield \[P_1(s)=\frac{\lambda_1(s)^{-a} \lambda_2(s)^{-a} (\lambda_1(s)^{b}-\lambda_2(s)^{b})}{\lambda_1(s)^{b-a}-\lambda_2(s)^{b-a}} \quad \text{and} \quad P_2(s)=\frac{\lambda_1(s)^{-a}-\lambda_2(s)^{-a}}{\lambda_1(s)^{b-a}-\lambda_2(s)^{b-a}}.\] Finally, using again Equation 7.30 we can now conclude that \[\begin{align*} G_\tau(s) &= \E[s^\tau] \\ &=P_1(s)+P_2(s) \\ &= \frac{\lambda_1(s)^{-a} \lambda_2(s)^{-a} (\lambda_1(s)^{b}-\lambda_2(s)^{b})+\lambda_1(s)^{-a}-\lambda_2(s)^{-a}}{\lambda_1(s)^{b-a}-\lambda_2(s)^{b-a}}. \end{align*} \] There you have it: an expression that fully determines the distribution of $\tau$ — no mean feat! :).

7.1 Why?

7.2 Gambler’s ruin

7.3 Hitting times of simple random walks

7.3.1 The symmetric case

7.3.2 The general/non-symmetric case

7.4 Wald’s martingale

7.4.1 Case 1: unity value for the mgf of \(Y\)

7.4.2 Case 2: simple random walk

7.5 Some exercises