Tutorial 1 Problem Set
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
Tutorial 1 Problem Set
Questions marked by (H) are either challenging questions or questions designed for the self-learning part (modes of convergence of random variables and the method of moment). These questions will not be used in any part of the assessments.
Last two questions provide a good starting point of statistical programming.
1. Suppose we have k iid standard normal random variables, Z1, Z2 , . . . , Zk, where Zi ~ N(0, 1). The sum of the squares of these k random variables, Y = i(k)=1 Zi(2), can be modelled by the chi-square distribution (or χ2 (k) where k is the degree of freedom), which has the pdf
fY (y) = 
y 
21e2 
,
where Γ (.) is the gamma function. Furthermore, we know χ2 (k) has mean k and variance 2k . Also note that the sum of chi-squared distributed random variables is also chi-squared distributed.
Consider we have a sequence of iid random variables X1 , . . . , Xn drawn from the Gaussian distribution Ⅳ (µ, σ2 ) and the sample average Sn = 
Xi, show the following:
(a) Sn ~ Ⅳ (µ, σ2 /n).
(b) 
(Xi _ µ)2 ~ σ 2 χn(2) .
(c) n(Sn _ µ)2 ~ σ 2 χ1(2) .
2. Consider a random vector X e Rm follows a multivariate Gaussian distribution Ⅳ (µX , ΣX) and another random vector Y e Rn follows a multivariate Gaussian distribution Ⅳ (µY , ΣY). Consider X and Y are independent, find the distribution of the following random variables:
(a) Z = AX + Y , where A e Rn -m is a given matrix.
(b) Z = X _ BY , where B e Rm -n is a given matrix.
(c) Z = X + c, where c is a constant scalar.
Now, consider X and Y are correlated with covariance Cov(X, Y) e Rm -n , find the joint distribution of the random variable (X, Y).
3. Consider two random variables X and Y are independent, show that 匝[XY] = 匝[X]匝[Y].
4. (H) Consider we have two sequences of random variables X1, X2 , . . . , Xn and Y1, Y2 , . . . , Yn such that Xn 
X and Yn 
Y,
show that the joint random variable
(Xn, Yn) 
(X, Y).
5. (H) Consider a sequence of random variables X1, X2 , . . . , Xn such that Xn _→ X for any q > 1, use Markov’s inequality to show that Xn 
X . That is, convergence in mean implies convergence in distribution.
Lp
stronger than Xn _→ X for p > q > 1).
6. Let X1, X2 , . . . , Xn be iid random variables draw from the Gaussian distribution Ⅳ (µ, σ2 ).
(a) (H) Find the estimation of µ and σ 2 using the method of moments and justify how many moments
you need.
(b) Find the estimation using the maximum likelihood.
Write down the step-by-step derivations.
7. For λ > 0, the exponential random variable X ~ Exp(λ) has probability density function fX (x) = 
Let X1, X2 , . . . , Xn be iid random variables draw from the exponential distribution Exp(λ).
(a) (H) Find the estimation of λ using the method of moments and justify how many moments you need.
(b) Find the estimation using the maximum likelihood.
8. Consider the linear regression case where we have m distinct points, x1 , x2 , . . . , xm e R. We can treat observations as independent random variables y1 , y2 , . . . , ym depend on covariate x1 , x2 , . . . , xm as follows:
yi = θ0 + θ1xi + θ2xi(2) + θ3xi(3) + ei ,
where ei are iid Ⅳ (0, σ2 ) and m > 4. Derive the maximum likelihood estimator for θ = (θ0 , θ 1 , θ2 , θ3 ).
9. Consider the above linear regression case, but the observation noises ei , i = 1, . . . , m, are independent but not identically distributed. Each ei follows a Gaussian with zero mean but different variance, i.e., ei ~ Ⅳ (0, σi(2)). Derive the maximum likelihood estimator for θ = (θ0 , θ 1 , θ2 , θ3 ).
10. Draw n = 103 , 104 , 105 , 106 iid random variable X1, X2 , . . . , Xn from Ⅳ (0, 1) to estimate the mean (which is 0) by computing the sample average. Using 102 replicates of the sample average to estimate the variance of the estimated mean for each n. Plot the variance versus n. Justify if LLN and CLT are satisfied.
11. Repeat the above question, but change the distribution to the Cauchy distribution with pdf
1
π[1 + x2] .
A Cauchy random variable with the above pdf can be generated by x= tan(pi*(rand(m,m)-0.5)) in MATLAB.
Hint: the second moment of a Cauchy random variable is not defined.
2023-05-30