MATH2012 STOCHASTIC PROCESSES SEMESTER 2 EXAMINATION 2021/22

Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

SEMESTER 2 EXAMINATION 2021/22

MATH2012 STOCHASTIC PROCESSES

1. A Markov chain {Xn : n ≥ 0} on the state space S = {0, 1, 2} has the following one-step transition probability matrix

Let p(n) be the probability distribution of Xn, and P (n) be the n-step transition probability matrix.

(a) Find P (3). [4 marks]

(b) If the Markov chain starts in state 1 at step 0, find

P{X0 = 1, X1 = 0, X2 = 1, X3 = 2}. [3 marks]

(c) If the Markov chain starts in state 1 at step 0, find p (3). [3 marks]

(d) Explain why state 1 is aperiodic, and list all other aperiodic states. [3 marks]

(e) Explain (you may quote results from this module) why state 1 is positive recurrent, and list all other positive recurrent states. [3 marks]

(f) Find the mean recurrence times of the states. [4 marks]

(g) After sufficiently long time, what are the probabilities that the Markov chain is in each of the three states. [3 marks] [Total 23 marks]

2. A Markov chain {Xn : n ≥ 0} on the state space S = {1, 2, 3, 4, 5, 6, 7, 8} has the following transition probability matrix

(a) Draw the transition diagram. [4 marks]

(b) Decompose the state space of this Markov chain into intercommunicating classes, and, without performing any calculations, identify each class as transient or recurrent. [5 marks]

(c) State whether this is an ergodic Markov chain, and give your reason. [3 marks]

(d) If the Markov chain starts from state 8, describe the long term behaviour of the Markov chain. [4 marks] [Total 16 marks]

3. A laundrette has four washing machines and two repair persons. Each machine works an exponentially distributed amount of time, with an average of f hours, before failure. A failed machine is served by one repair person (if available) in order of failure, taking an exponentially distributed amount of time with an average of r hours. The machines work independently and the repair times are also independent. Let X(t) denote the number of machines working at time t ≥ 0, and let pi,j (δt) = P{X(t + δt) = j|X(t) = i} for small δt > 0, 0 ≤ i ≤ 4 and 0 ≤ j ≤ 4.

(a) Show that

        [5 marks]

(b) The generator matrix of the process {X(t), t ≥ 0} is given by

Replace each ∗ in the generator matrix with the correct entry. [8 marks]

(c) Assume the initial number of working machines X(0) is 4, and let pn(t) = P{X(t) = n}. Explain carefully how a set of differential and difference equations for pn(t), 0 ≤ n ≤ 4 can be derived from Kolmogorov’s forward equations dt/d P(t) = P(t)A. [5 marks]

(d) By using (c) or otherwise, write down a set of equations that the equilibrium distribution of {X(t) : t ≥ 0}, (π0, π1, . . . , π4), satisfies. [2 marks] [Total 20 marks]

4. The generator matrix of a continuous time Markov jump process {Xt , t ≥ 0} with states {1, 2, 3} is given by

(a) Explain what is meant by the embedded Markov chain associated with this process. Write down the transition probability matrix of this embedded Markov chain. [4 marks]

(b) Explain what is meant by the holding time of a state. Write down the probability density function of the holding time of state 2. Find the expected time spent in state 2 from the point of entry. [5 marks]

(c) Let the equilibrium distribution of {Xt , t ≥ 0} be given by πn = limt→+∞ pn(t), n = 1, 2, 3. Write down a set of equations that the πn’s satisfy, and solve for the πn’s. [7 marks]

(d) Briefly describe how you would simulate a sample path of this process, starting in state 2. [4 marks] [Total 20 marks]

5. The marital state of an individual in a population is described as B (bachelor), M (married), W (widowed), D (divorced) or ∆ (dead). Suppose the individuals in the population behave independently of each other. Assume the generator matrix is given by

where t represents the age of the individual.

(a) Assume an individual is in state M at age 35. Find the probability that this individual remains married at least until age 50, in terms of the transition rates given in the generator matrix. [5 marks]

(b) Assume that a married individual of age 35 remains married until a transition is observed at age 50. Find the probability that this individual is divorced immediately after age 50. [5 marks] [Total 10 marks]

6. Let {Bt : t ≥ 0} be a standard Brownian motion.

(a) Let Xt = Bt − tB1 for t ≥ 0. Find E(Xt) and Cov(Xt , Xs), and state the distribution of Xt , where 1 > s ≥ t > 0 are given constants. [5 marks]

(b) Let Ut = x0 exp ((α − β 2/2)t + βBt) for t ≥ 0, where x0 ≠ 0, α and β ≠ 0 are given constants. Use Ito’s lemma to show that {Ut , t ≥ 0} satisfies the stochastic differential equation

[6 marks]

[Total 11 marks]