Question 1.  [8 MarkS] Consider a Markov chain with 5 states {0, 1, 2, 3, 4} and the one- step transition matrix P given by

(a)  Draw the transition diagram for this Markov chain and identify the communicating classes. Which states are transient and which states are recurrent?

(b)  For each transient state in the state space, evaluate the expected time until the Markov chain first hits a recurrent state.

(c)  Find the general form of the stationary distribution for this chain.

Question 2. [8 MarkS] A gambler has $20 and needs to increase it to $100 in a hurry. He can play a game with the following rules: a fair coin is tossed; if a player bets correctly on heads or tails, he wins a sum equal to his stake, and his stake is returned; otherwise, he loses his stake. The gambler decides to use the following bold strategy: if he has $50 or less, he stakes all his money on the next coin toss; otherwise, he stakes just enough to increase his capital, if he wins, to $100.

Let X0  = 20 and let Xn be the gamble’s capital after n tosses of the coin. (a)  What is the appropriate sample space, given the gambler’s strategy?  (b)  Prove that the gambler will achieve his aim with probability 1/5.

(c)  Find the expected total number of tosses until the gambler either achieves his aim or loses his capital.

Question 3. [10 MarkS] A die with sides labelled by the numbers 1 to 6 is thrown repeat- edly, with the outcome of each throw being independent of the other throws. Let Xn denote the sum of the first n throws.

(a)  Assuming the die is fair, so that the six numbers are equally likely in a given throw, find

lim P[Xn  is a multiple of 11].

Hint: define a suitable Markov chain with 11 states.

(b)  Define Yn  = Xn + c where c is a constant integer and Xn is defined as before. Find

lim P[Yn  is a multiple of 11].

(c)  Suppose now that the die is not necessarily fair and that γj  = P[X1  = j], where γj  > 0 for j  =  1, . . . , 6, and     j(6)=1 γj   =  1.  Does your answer to part (a) change?  Give justification for your answer.

Note:  when answering (a), (b) and (c) above you should provide rigorous justification by appealing to results in Markov chain theory; just stating a correct answer will not be sufficient to gain full marks for that question part as many of the marks will be awarded for appropriate working and/or reasoning. Any results from Markov chain theory that you use should be stated (or referenced in the lecture notes, if you prefer) but should NOT be proved.

Question 4. [14 MarkS.] Consider a branching Markov chain (Xn )n-0 where X0  = 1 and the offspring distribution is given by

p(0) = , p(1) = , p(2) =  .

The number of offspring produced by each individual is independent of the number of offspring produced by other individuals.

(a)  Calculate the probability of ultimate extinction.

(b)  Calculate the expected size (i.e. the mean size) of the n-th generation.

(c)  Calculate the variance of the n-th generation.

For the remainder of the question assume that the offspring distribution is unchanged but that X0 has a Poisson distribution with PMF

P[X0  = m] = e>λ,     m = 0, 1, 2, . . . ,

where λ > 0.

(d)  Calculate the probability of ultimate extinction.

Note: if X0  = 0 then the probability of ultimate distinction is 1.      (e)  Calculate the expected size (i.e. the mean size) of the n-th generation. (f)  Calculate the variance of the n-th generation.