Instructions

1		due in Moodle, Friday June 24, 4pm
2		Signed School Cover Sheet attached
3		TYPED PDF only - no microsoft word docs.
4		Follow the Homework Rules.
5		Computer output	: no commentary ÷ no marks.
6		Analytical results		: no working ÷ no marks.
7		◆ means you can use Matlab; else not.
8		excep(N). No(N)w(o)eb(Di)ssear(cus)c(si)h(o)in(n)g

Q1 (14) Theory

(a) If θˆis an estimator of a vector parameter θ show that MSE = E(l θˆ _ θ l2 ) =l bias l2  +trace(var(θˆ))

(a) For the regression problem y = Xn×pβ + ∈ where ∈ ~ N (0, σ2 I) with true value βo , consider the biased (’shrinkage’) estimator

βˆa      =    (1 _ )βˆ

βˆ   =    (XT X)-1 XT y = LSE

(i) Find the bias and variance of βˆa .

(ii)  Show βˆa has a smaller MSE than the LSE when

2τ     

a <

where vsnr =  and τ = T   1(p)  and λk are the singular values of X .

(b) Noise Model.

Consider the stationary process

Yt  = a + φYt -3 + ∈t , t = 1, 2, . . .

where ∈t  is a Gaussian white noise sequence of zero mean and variance σ 2 .

(i) Explain what are the stability/stationarity con- straints on φ?

(ii) Derive closed form expressions for the mean and acs of Yt .

Q2(14) (Impulse Response Estimation)

Consider a system with output sequence measured in noise

yt  = st + nt , st  = (h * u)t , t = 1, . . . , T

with impulse response hr  = Aγ sin(2πr ), r = 1, 2, . . . , m. The input sequence is a (0, σu(2)) white noise 1  independent   of the observation noise sequence which is a (0, σ2 ) white   noise sequence. The variance SNR is vsnr =σ(σ) .

(a) Ignoring start-up transients, and setting ωo   = show that for large m

σs(2)      =       1(m)hr(2)σu(2)  s σu(2)      1(~)hr(2)

=   A2 σu(2) [  + ]

(b)  ◆Simulation.

Write an m-file to simulate the system for T = 100 with the values:    [γ, To , σ, vsnr] =  [.8, 7, 1, 1] and m = mo  = 20.

(c)  ◆Show four displays:

plots of st , yt on the same graph;

histograms of yt , st on the same graph;

What do these plots reveal about the signals? an impulse response plot;

a Bode plot i.e. system frequency response.

(c)  ◆Parameter Estimation.

Write an m-file to compute the penalized least squares estimator, its standard errors2 , the singular values of the X matrix.

(d) To compute the standard errors you need to derive the following formula for the variance of the smoothness penalized least squares estimator (SP-LSE).

var(βˆλ ) = σ2 (X\ X+λDT D)-1 XT X(X\ X+λDT D)-1

(e)  ◆Using the simulated data from (b) compute the SP- LSE of β for m = mo  and a grid of λ values as fol- lows.

Plot the ’loss’ = l βˆλ  _ βo   l2  versus λ to find the minimizing value of λ . 3

Compute the corresponding ’minimizing’ SP-LSE, its standard errors and plot the estimated IR overlaid with the true IR and a 95% confidence interval. Also plot the singular values of the X matrix and comment on them.

Q3 (8). ◆ Statistical Graphics.

The graphics/plots you display in Q1, Q2, Q3 will earn up to 8 marks.