Applied Probability 2015
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
Applied Probability
1. Consider a random walk on the state space S = {0, 1, 2, 3, . . .}. The transition probabilities for the Markov chain corresponding to the random walk are as follows, with p,q > 0 and p + q = 1. For i ≥ 1,
8 p if j = i +1
>
pi,j = < q if j = i − 1
>
: 0 otherwise.
For state 0,
p0,0 = 1.
(a) Identify the communicating classes of this Markov chain and state whether they are recurrent or transient.
(b) Let ui({)0} be the hitting probability on the state {0}, given that the process starts in state i. Write down the system of equations for ui({)0},i ≥ 1, and the associated boundary conditions.
(c) Assuming that q > p, evaluate nui({)0},i 2 So.
[11 marks]
2. Consider a Discrete-Time Markov Chain (Xn ,n 2 Z+ ) on the state space S with one-step transition probability matrix P = (pi,j ) and let A be a subset of S . Define W to be the event that the process eventually reaches the set A and ui(A) = P(W|X0 = i).
(a) Show that
for i 2 A, for i 2 S \ A.
what criterion
[6 marks]
3. Consider the state i in a DTMC (Xn ,n 2 Z+ ).
(a) Let f
= P(Xk = i,X` 
i,` = 1, . . . ,k − 1 |X0 = i) be the probability that start- ing from state i, the first return to state i occurs at the kth transition.
(i) Explain why f
= 0.
1
(ii) Give an interpretation of the probability fi,i 
k=1
(iii) State the definition of recurrence and transience for state i in terms of the prob- ability fi,i .
(iv) If a state i is ephemeral, what does this mean for fi,i ?
(b) Let p
= P(Xn = i |X0 = i).
(i) Give an interpretation of the probability p
1
(ii) Give a characterisation of transience and recurrence in terms of 
n=1
(c) Show that all states in a communicating class, C, are recurrent if the state i 2 C is recurrent. Hint: use your answer to (b)(ii) above.
[11 marks]
4. Consider the DTMC (Xn ,n 2 Z+ ) on states {1, 2, 3, 4} with one-step transition probability
|
matrix given by 0B005 P = B(B) 0
|
0.5 0 0.5 0 0 |
0 0.5 0 0.5 0 |
0.5 0 0.5 0 0.1 |
0(0) 1C 0 C(C)
|
(a) Identify all communication classes and state whether they are recurrent or transient. (b) Determine the period of all states.
(c) Do any classes have limiting distributions?
(d) Do any classes have stationary distributions? If so, determine those stationary distri- butions.
[10 marks]
5. Consider the DTMC (Xn ,n 2 Z+ ) on states {0, 1, 2, . . .} with one-step transition proba- bilities given, for all i ≥ 1, by
8 p, if j = i +1,
pi,j = 1 − p, if j = i − 1,
: 0, otherwise
and p0,1 = 1.
(a) What is the requirement on p for this to be an irreducible DTMC?
(b) Write down the global balance equations for this DTMC. DO NOT SOLVE.
(c) Let B0 = {0}, what is the partial balance equation with respect to B0 ?
(d) For any j ≥ 1 and Bj = {0, 1, . . . ,j}, write down the partial balance equation for Bj .
(e) Use all these partial balance equations to derive the global balance equations. (f) Solve either the partial balance equations or the global balance equations.
(g) For what values of p is the DTMC positive recurrent. Justify your answer.
[16 marks]
6. (a) Let U represent the probability of eventual extinction in a discrete-time branching process, given that it starts with one individual. Let µ represent the mean number of o↵spring produced by each individual.
(i) What do you know about µ if U < 1.
(ii) What do you know about U if µ < 1.
(iii) If U < 1, explain in words what must happen to the branching process with probability 1 − U > 0.
(b) In a discrete-time branching process, an individual has 0, 1, 2 or 3 o↵spring with prob- abilities 2/5, 0, 1/5, 2/5, respectively. Find U, the probability of eventual extinction of this process.
[8 marks]
7. Consider the DTMC (Xn ,n 2 Z+ ) on states {0, 1, 2, . . . ,N} with one-step transition proba- bility matrix P = (pi,j ). It is known that Xn is a Martingale with respect to itself.
(a) State the definition of such a Martingale.
(b) Therefore, show that
N
X jpi,j = i for 0 
i 
N.
j=0
(c) Deduce that the states 0 and N must be absorbing states.
[8 marks]
2022-06-17