0.1    Question 1 [50 marks]

Three processing units - Units 1, 2, and 3 - deal with requests until they are either classified as  ‘resolved’ or ‘dismissed’ . All requests initially go to Unit 1. You may assume that the process      {; ≥ 0}, describing the location of the request at time  (i.e. at unit 1, 2 or 3, or resolved (R), or dismissed (D)) is a Markov chain with state-space  = {1,2,3,  , } and transition matrix

0    2/3       0       1/3       0

0       0       1/2    1/2       0

  =     0     1/4     1/4     1/3     1/6    .

(0       0          0          0          1  )

a.     Write down the initial distribution for {;  ≥ 0}. [1] (1,0,0,0,0)

b.     Find the irreducible classes of intercommunicating states, classifying them as positive      recurrent, null recurrent or transient. State their periodicity and whether they are ergodic. [4]

{1} - Transient - aperiodic - not ergodic.

{2,3} - Transient - aperiodic - not ergodic.

{} - Positive recurrent - aperiodic - ergodic.

{} - Positive recurrent - aperiodic - ergodic.

c.     Without performing any calculations, state whether:

i.      an invariant distribution exists for {;  ≥ 0}, clearly stating your reasoning in one sentence; [2]

ii.     an equilibrium exists for {;  ≥ 0}, clearly stating your reasoning in one sentence. [2]

Invariant exists - finite state-space.

Equilibrium doesn’t exist since more than one closed class.

d.     Assuming that each attempt at resolution takes 1 unit of time, what is the probability that the request stays active in the system (i.e. is not resolved or dismissed) for more than 4    units of time? [5]

1/12

e.     Compute the expected time until the request is resolved or dismissed, again assuming that each attempt at resolution takes 1 unit of time. [3]

7

3

f.      Compute the expected number of times that unit 3 attempts to resolve a request. [4]

8/15

Now assume that at time 2, the request is being dealt with by unit 3 (i.e.  2  = 3).

g.     Compute the probability of the request ever returning to unit 3 (i.e.  = 3 for  > 2) given that  2  = 3. [4]

3/8

h.     State the distribution of the number of times the process visits state 3 in total, given  2  =

3. (You should count the fact that  2  = 3 as one visit.) [2]

Geometric(5/8)

i.      Compare the mean of the distribution you gave in part (h) with your solution to part (f) and explain in one sentence why they are either the same or different (whichever applies). [2]

Different because in (f) we were working under the assumption we may not reach state 3 ever, and in (h) we are working under the assumption we do reach state 3.

A system upgrade means that if a request reaches state 3, the system sends a warning message to the administrator. However, the warning mechanism is faulty: it sends  warning messages   each time the request reaches state 3 where  has a Poisson distribution with some parameter  > 0.

j.      Find the probability generating function of the total number of warning messages the       system sends before the request is resolved or dismissed, under the assumption that  2  =

3. [8]

5/8exp(( − 1))

1 − 3/8exp(( − 1))

k.     Compute the probability that the system sends no warning message for a particular          request, either because it never reaches state 3 or because when it does the system sends

no messages. Note here that the assumption of  2  = 3 has been removed. [6]

2     1       5/8exp(−)

3     3   1 − 3/8exp(−)

A further malfunction means the following changes to the system. Updated transition                 probabilities are given below where, for example,   denotes the one-step transition probability from state  to  .

•      Unit 3 can no longer resolve requests or send them to Unit 2, so that  3  = 0 and  32  = 0.

•      Once a request reaches Unit 3 it either stays there with probability 1/4 or classifies it as dismissed, i.e.  33  = 1/4 and  3  = 3/4.

•      Once labelled as dismissed the request immediately returns to Unit 3, so that 3  = 1.

Engineers trying to resolve the issue have built a system that tracks a request, emitting a signal   of ‘1’, ‘2’ or ‘3’ if the request is in unit 1, 2 or 3 respectively, and a signal of ‘4’ if the request is      either in state D or state R. Let  denote the signal emitted from the tracking system at time ,  again assuming that each attempt at resolution takes 1 unit of time. For example, if 2  = 3 then  we know that the request is with Unit 3 at time 2, or if 5  = 4 then we know that the request is in state  or  at time 5.

l.      Is the process {;  ≥ 0} Markov? If it is, justify your answer. If it is not, give an example of

where the Markov property fails. [7]

No it isn’t. Any counter-example will do.

0.2    Question 2 [50 marks]

Consider the following simple model of a population of  individuals. A virus is thought not to  be transmissable between individuals, but they can contract it by other means (e.g. from their    environment). Let   denote the number of individuals in the population with the virus at time  . Once infected, individuals eventually recover but can be re-infected at any time at the same rate as those who have not previously been infected. The distribution of the length of time an           individual is infected for is the same regardless of how many times they’ve contracted the virus  in the past.

You can assume that { ;  ≥ 0} is a continuous-time, time-homogeneous, Markov process with state-space  = {0, 1,2, . . . , }, defined by the following transition rates where these transitions   are possible,

 ,−1  ,+1

= 

= ( − )

and  ,  = 0 when  < ( − 1) or  > ( + 1).

a.     Write down the generator matrix,  , for { ;  ≥ 0}. [3]

− 

0

0

⋮

−

2

0

⋮

0

0

( − 1)

−

3

⋮

0

0            . . .        0

0            . . .        0

( − 2)    . . .        0

−         . . .        0

⋮             ⋱         ⋮

b.     Does an equilibrium distribution exist for { ;  ≥ 0}? Explain your answer in no more than one sentence. [2]

Irreducible Markov chain, finite state-space, guarantees that an equilibrium distribution exists

c.     Define the vector  = (0, . . . ,  ) to be the solution to  = 0. Find an expression for  

_                                                                                                     _              _

(for some  = 1, . . . , ) in terms of  0  and any other constants required. [4]

Binomial with  trials and probability of success 1/2

e.     In one sentence, explain what the (embedded) jump chain { ;  ≥ 0} of the process { ;  ≥ 0} would describe. [1]

The jump chain records whether the next event is an infection or recovery.

f.      Write down the transition matrix of {; ≥ 0}. [2]

0          1                0                       0             . . .    0    0

1/       0       ( − 1)/              0             . . .    0    0

  =        0        2/               0               ( − 2)/     . . .     0     0

(   0          0                 0                        0              . . .     1     0)

g.     What happens to {;  ≥ 0} in the long run? Does ( = ) converge as  → ∞, for  = 0, . . . , ? Justify your answer in no more than two sentences. [4]

h.     There is one infected person in the population at time  = 0. Compute the expected time until we have two infected individuals in the population for the first time. [6]

 +

Now consider a different population of  individuals who are dealing with a virus - known as    Covoid - which is only transmissable between individuals (cannot contract from the                     environment). An individual with the virus has probability ℎ + (ℎ) of passing it to another       individual in any time interval of length ℎ, where ℎ is very small. If an individual is infected, the   distribution of the length of time they remain infected is exponential with some parameter  >   0, and individuals eventually recover but can become re-infected at any time. The length of time they are infected is not related to whether they have had the virus before.

i.      What happens to the virus Covoid if, at some time  , there is nobody in the population with the virus? [1]

Died out - nobody can contract it.

j.      Write the the generator matrix of the process describing the number of individuals in the new population infected with Covoid. [3]

0            0                     0                      0               0      0                          . . .

     −( + )                                   0               0      0    . . .      0          0

  =     0            2            −2( + )             2               0      0     . . .       0           0

0              0                       3              −3( + )     3     0     . . .       0           0

⋮             ⋮                      ⋮                        ⋮                ⋮       ⋮      ⋱        ⋮           ⋮

k.     It is not known how many individuals have the virus Covoid at a given time  . Under what conditions does Covoid die out? Justify your answer, for example by considering the        probability of Covoid becoming extinct conditional on all  individuals being infected     simultaneously. [7]

Eventually we do reach state 0 with probability 1 (and regardless of where we start).      Now assume that the size of this new population is infinite, rather than having  individuals.

l.      Again, it is not known how many individuals in this infinite population have Covoid. Under what conditions is Covoid guaranteed to die out when we assume an infinite population? [2]

 ≥

m.    A vaccine has been manufactured that protects individuals against Covoid with probability  0.9. The Government cannot immunise everyone in the population. By considering different relative magnitudes of  and  , propose the minimum proportion of the population that    need to be vaccinated in order for Covoid to die out with probability 1. You should justify   your proposal by computing relevant quantities. [9]