Statistics – STATS 4024 Stochastic Processes 2018
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
Statistics – STATS 4024
Stochastic Processes
2018
1. Consider a homogeneous Markov chain in discrete time defined over a finite state space. The chain is defined by the stochastic matrix A with elements Aik e 皿, which define the probability of a transition from state i to state k in one time step. Assume that at time 0, the system is in state i.
(a) Which conditions do the matrix elements Aik have to satisfy for A to be a valid
stochastic matrix?
[1 MARK]
(b) Let gm) denote the probability that the Markov chain returns to state i in m
steps. Express g in terms of Aik . [1 MARK]
(c) Let fi(m) denote the probability of a first return to state i in m steps. Use the law of total probability to derive an expression for fi(2) in terms of g , fi(1) and Aik .
[2 MARKS]
(d) Let fi denote the probability of a return to state i. Explain why fi = m(|)=1 gm) is wrong and why you have to use fi = m(|)=1 fi(m) instead. [2 MARKS]
(e) Consider a homogeneous discrete-time Markov chain defined by the following
stochastic matrix: A = ╱ 1 . 2 |
1 2 1 2 0 |
、 1 . [1 MARK] |
ii. Use this state transition diagram and the result from part (d) of the question to decide which states are transient and which states are persistent. Justify your answer! [6 MARKS]
iii. Find the stationary distribution of this Markov chain. Justify your answer!
[2 MARKS]
iv. Does this Markov chain have a unique limiting distribution? Find the limiting distribution if it exists. Justify your answer! [4 MARKS]
(f) Let xt denote the state of the Markov chain at time t. Show that if (x1 , x2 , . . . , xm ) is a homogeneous Markov chain that has converged to its limiting distribution, then the reverse process running backwards in time, (xm , xm尸1 , . . . , x1 ), is also a homogeneous Markov chain. [4 MARKS]
(g) Assume you want to sample from a discrete probability distribution p(i) = , where φ(i) is easy to compute, but the normalisation constant Z is intractable. Use the result from the previous part of the question to discuss how that can be done. [2 MARKS]
2. (a) Let N (t) e 勿 be a discrete time-varying random variable representing the number of events that have occurred in interval [0, t], where t denotes time and N (0) = 0. Define pn (t) = P [N (t) = n]. Recall that the probability generating function for a
discrete stochastic process is defined as:
|
G(s, t) = pn (t)sn
n=0
Assume that, for λ > 0, the differential equation for the probability generating function is given by:
∂G(s, t)
i. Solve the differential equation under the boundary condition N(0) = 0.
[3 MARKS]
ii. Use this result to prove that the stochastic process defined by the differential equation above is the Poisson process with rate λ . [3 MARKS]
iii. Show that the inter-arrival times, i.e. the time intervals between successive events, have an exponential distribution. [3 MARKS]
iv. Derive an expression for the mean or expected inter-arrival time.
[3 MARKS]
v. Patients suffering from flu-like symptoms are admitted to the A&E depart- ment of a hospital at a rate of 10 per day. Assume that admissions form a Poisson process, and that doctors at the department work in 12-hour shifts. From the beginning of a shift, what is the expected time a doctor has to wait until the first call, and what is the probability that exactly 6 patients are admitted during a shift? [3 MARKS]
(b) Let T be a nonnegative random variable which is the time to failure for a given device. The distribution function of T is F (t) = P (T < t). The probability density function is denoted by f(t). The reliability function R(t) = P (T > t) is the probability that the device is still operational after time T. The failure rate function or hazard function is defined as
P (t < T < t + δt|T > t)
δt一0 δt
i. Express in terms of f(t). [1 MARK]
ii. Give an interpretation of the hazard function r(t). What does it measure?
[1 MARK]
iii. Show that
f(t)
r(t) =
[3 MARKS] CONTINUED OVERLEAF/
3
iv. Show that
R(t) = exp ┌ - 0 t r(u)du┐
[3 MARKS]
v. Whilst in use, a printer is observed to have a hazard function of r(t) = 2λt per hour where λ = 0.0001 hours尸2 . What is the probability that the printer is still operational after 100 hours? [2 MARKS]
3. In the lectures we discussed a queue with a single server. Let the random variable N(t) denote the number of individuals in the queue at time t (including the person being served). New customers arrive independently, and these arrivals form a Poisson process with intensity λ > 0. Service times have an exponential distribution with parameter µ > 0, that is the distribution with probability density function f(t) = µ exp(-µt), t > 0. Now consider a baulked queue where no more than m people (including the person being served) are allowed to form a queue. If there are m people in the queue, then any further arrivals are turned away. It can be shown that the probability distribution for this queue is defined by the following differential equations:
dpm (t)
dt
dpn (t)
dt
dp0 (t)
dt
= λpm尸1 (t) - µpm (t)
= λpn尸1 (t) + µpn+1(t) - [λ + µ]pn (t); 1 < n < (m - 1)
= µp1 (t) - λp0 (t)
(1)
(2)
(3)
(a) How do you get the limiting distribution limt一| pn (t) = pn , where pn is time invariant, from these differential equations? [1 MARK]
(b) Define ρ = . Show that pn = 1 and pn = ρn are solutions to equation (2).
[2 MARKS]
(c) Assuming that ρ 1, find the general solution for pn , 1 < n < m. Hint: Make use of the mathematical techniques that you have learned in the first lectures on finite difference equations and treat equations (1) and (3) as boundary conditions.
[6 MARKS].
(d) Assume that the service rate µ is twice as high as the arrival rate λ and that the maximum permissible length of the queue is m = 10. What is the probability that n = 5 people are in the queue? [1 MARK].
2022-04-29