The quiz consists of just one of the following questions and you can choose which one to submit. You can submit either question 1, or question 2, but not both.

1. Question 14 on Quiz 1 sheet.  Please report your prediction value at 北 in your handwritten answer.

To answer this question, you must write your own code in either Mat- lab/R/Python/Julia and submit it online together with any hand-written answers.   The submitted code will be assessed and must reproduce the results in your answer. The submitted code must be your own implemen- tation, otherwise marks may be deducted.

For example, the code will not be deemed to be your own implementation if your submission is the same as somebody else’s, or if your submission uses high-level functions or packages/software such as glm in R. You must compute everything from first principles — using in-built linear equation solvers and pseudo-inverse functions like pinv .m in Matlab, and other basic matrix capabilities is acceptable.

solution: the answer is 122.49... The answer is obtained by computing:

尸山(北)              X尸+ y使       = 122.49..

make prediction Diabetes training

Note that X contains all of the features, not just k-th feature for computing βˆk . Coefficient βˆk  is computed using all of the features.

2. Let

n

X = UΣV⊤ =工σi ui v i(⊤)

i=1

be the full SVD of the n×p feature matrix X with n > p and p = rank(X) (so that exactly p of the n singular values are not zero).

(a) Show that the projection matrix defined as P := XX+  can be written

as

P =  工 ui ui(⊤) .

i:σi0

solution: This follows from

XX+  = (UΣV⊤ )(VΣ+ U⊤ ) = UΣΣ+ U⊤

Here ΣΣ+  ∈ Rn×n  is a square diagonal matrix with p number of ones on the main diagonal (corresponding to the p nonzero singular values). Hence, the result.

(b) Show that

n

In  =工ui ui(⊤) ,

i=1

where In  is the n × n identity.

solution: This follows directly from UU⊤ = In .

(c) Define the matrix P⊥ := I − P. Show that

P⊥ =  工 ui ui(⊤) .

i:σi=0

solution:

I − P =工ui ui(⊤) − 工 ui ui(⊤)  =  工 ui ui(⊤) .

i                        i:σi0                     i:σi=0

(d) Use the results above to prove that P⊥ P = 0 is the n × n zero matrix.

solution:

工 uj uj(⊤)

j:σj0

ui (ui(⊤)uj )uj(⊤)

ui (0)uj(⊤)  = 0.

(e) Suppose that Z ∼ N(0, In ) is a vector with n iid standard normal

variables and define the variables X 1  and X2 :

[X(X)2(1)] := ] Z .