Homeworks are important as they help solidify concepts discussed in class. However, feel free to consult me, other students, your TA, or the text for help.

Selected problem from the following problems will be graded and must be turned into GradeScope.

Most problems require a hand written solution. Please write out your solutions by hand to receive full credit. You are of course free to check your results using MATLAB or the equivalent.

Some problems—“ MATLAB Problems ”—will require a solution worked out computationally using a high level programming language.  Our default is MATLAB, however you may use another language if you choose.  To submit your work for these problems, either save output as a pdf or you can take a screenshot of your work. Either way, you can incorporate your work as a pdf in the HomeWork that you submit. A problem requiring a MATLAB solution will have an asterisk (* ) beside the problem statement.

1.  Reflecting a Vector Though a Plane: Let’s reflect the vector X = [2, 3, 4]T through the plane 2x . y ＋2z = 0. Let’s call the reflected vector  .

(a) Find a vector normal to the plane described by 2x . y ＋2z = 0. Call this vector V. For definiteness, let’s make the first component of V positive. (You may wish to consult an introductory calculus text to find out how to find a vector normal to a plane. It’s easy here.)

(b) Construct the projector PV  onto the vector V.

(c) Now find the reflector matrix F.

(d) Find the reflected vector  .

(e) Does lXl2   = ll2 ? Should it? Why or why not?

2.  A Householder Reflection: Consider the matrix A given as

┌2     2     3   5、

Let’s turn all of the entries below the diagonal entry in the second column of A into 0.

(a) What is the appropriate vector V that will be normal to the hyperplane such that the reflected vector below the diagonal element will be in the e1  = [1 0 0 0]T direction?

(b) What is the projector PV  onto the vector V?

(c) What is the Householder reflector F e R4*4 that you should use?

(d) Now construct the orthogonal matrix Q2  e R5*5 such that A\  = Q2A, where A\ has the form

┌2    r    r    r、

'0    r    r    r．

'                   ．

'                   ．

Find the matrix A\ .

(f) How many entries in A were changed after the Householder reflection?

3.  Givens Rotations: Consider the matrix A given as

┌6 '0 A = ' '0 ' ' '0

2 3 4 1

 

We want to use a Givens rotation to change the a32 entry in A from 4 to 0.

(a) Write out the Givens matrix G(2, 3, θ2,3 ) e R4*4 that changes the a32 entry in A from 4 to 0.

(b) Find the matrix A/ that one gets after doing this rotation, ie find A/  = G(2, 3, θ2,3 )A. After this rotation your A/ matrix

should have the form: ┌6 A/  = '

2 r 0 r

(c) After this Givens rotation, how many elements in A were changed?

4.  Symmetric Matrices have Real Eigenvalues and Orthogonal Eigenvectors for Distinct Eigenvalues: Let A e Rm*m  be a symmetric matrix, ie, suppose AT  = A. Let’s show that the eigenvalues of A must be real and that the eigenvectors associated with distinct eigenvalues must be orthogonal. We’ll show that the eigenvalues are real first.

(a) Begin with AX = λX. Write that down.

(b) Now multiply both sides by A to get A2X = λAX.

(c) Use the eigenvalue equation to eliminate A from the righthand side of the equation in (b).

(d) Left multiply the resulting equation from (c) by XT .

(e) Using the symmetric property of A show that your result from (d) can be written as lrl2(2)   = λn l●l2(2) . Your work should give you the r, ● and n. (Hint: A2  = AA = ATA.)

(f) From your results in (e), make an argument that the eigenvalues must be real.

Let’s now show that the eigenvectors for distinct eigenvalues must be orthogonal.

(a) Assume again that AT   = A and that you have two distinct eigenvalues of A—λ1  and λ2—such that λ1   ≠ λ2 .  Each

eigenvalue has an associated eigenvector—X1 and X2—respectively. We need to show that X1 × X2  = X 1(T)X2  = X2(T)X1  = 0.

(c) Now left multiply each equation by the transpose of the unlike eigenvector, ie, X2(T)AX1  = λ1X2(T)X1 .

(d) With the two equations from (c) and a little algebra, you can show that X 1(T)X2  = 0 if λ1  ≠ λ2 .

5.  Matrix Quantities from Singular Value Decompositions: One of the advantages of a singular value decomposition (SVD) is that it is a rich source of information about the matrix in question. For example, consider the matrix A and its singular value decomposition UzVT shown below.

┌'        、．           ┌' l3     0     0、．                 ┌'     0     、．

' l2       l6       l3．           ' 0       0     0．                 ' ．

Please address the following items directly from the SVD of A.

(a) By direct calculation, show that U and V are orthogonal matrices.

(b) By direct calculation, show that A = UzVT .

(c) What is the rank of the matrix A?

(d) What are the singular values of A?

(e) What is the absolute value of the determinant of A?

(f) What is lAl2

(g) What is lAlF ?

(h) Find z+ .

(i) Find an orthogonal basis for the column space of A. (j) Find an orthogonal basis for the null space of A.     (k) Find an orthogonal basis for the row space of A.

(l) Find an orthogonal basis for the left null space of A.

(m) Find a projector PA  e R3*3 that projects onto the column space of A.

6.  Relationship between Eigenvalues and Singular Values for Symmetric Matrices· : It turns out that there is a relationship between the eigenvalues and singular values of real symmetric matrices. Let’s explore this with a simple MATLAB exercise.

(a) Fire up MATLAB and generate a  10 * 10 random matrix.   You can use the command A  =  20*(rand(10,10)  - ones(10,10)/2). This will generate a 10 * 10 random matrix whose entries are on the interval [. 10, 10].

(b) Find the eigenvalues of A. You can use the command l  =  eig(A). MATLAB will return a vector of eigenvalues, l→ (c) Are these values all real?

(d) Find the singular values of A. You can use the command s  =  svd(A). MATLAB will return a vector of singular values, .

(e) Are they are real? (They better be!)

(f) Are they the same as the eigenvalues?

(g) Now, let’s do the same thing with a symmetric matrix. We can always symmetrize a matrix by letting As  =  .

In MATLAB, we can do this with the command As  =  (A+A’)/2. Visually inspect your As matrix to be sure that it appears symmetric.

(h) Now repeat (b) - (f) with As . Are the eigenvalues and singular values the same?

(i) Suppose that we take the absolute values of the eigenvalues and put them in descending order.  You can do this in MATLAB with the command l  =  sort(abs(eig(As)),’descend’).

(j) Now, eyeballing the output is good, but you can go further by taking the di住erence between the vector of eigenvalues and the vector of singular values, ie, find  =  . l→ Do this with the command r  =  s-l in MATLAB.

(k) Is the vector small? Maybe we could find the norm of , that is find l l2 . We can do this in MATLAB with the command norm(r). How big is your ?

(l) To be fair, we should really compare l l2   to l l2   or l l→l2 .  So, compute  . Is this close to the default machine epsilon ∈mach  = 10. 16 for MATLAB? Can you conclude that, at least in this (random) case, σi  = |λi | for symmetric matrices, if we put the absolute values of the eigenvalues in descending order?

7.  Low Rank Approximations via Singular Value Decomposition* : Consider the matrix A given as

┌1   2   3   4   5   6、

'2    3    4    5    6    7．

'                                       ．

'                                       ．

'9    8    7    6    5    4．

'                                       ．

'                                       ．

A = '2    5    3    6    4    7．

'                                       ．

'                                       ．

'1    2    1    2    1    2．

'                                       ．

'                                ．

'                                ．

'                                       ．

'1    3    2    4    3    5．

Fire up MATLAB and enter the matrix A.

(a) Perform a SVD of A using [U,S,V]  =  svd(A). What are the singular values of A?

(b) Perform low rank SVDs of A using [U,S,V]  =  svds(A,k) with k = 1, 2, 3. For each k, you’ll get the matrices Uk , zk and Vk .

(c) Construct Ak  = Ukzk VTk for each k.

(d) Define 6Ak  = A . Ak . Find 6Ak for each k.

(e) Find l6Ak l2  for each k.

(f) Find ∈k  =  for each k. Comment on how close your Ak approximates the real matrix A.

(g) Is it possible to find ∈k directly from the singular values of A? Explain.

8.  Pseudoinverse of a Rectangular Matrix* : Consider the matrix A given as

┌1   2   3   4、

'4   3   2    1．

'                         ．

'                    ．

'                    ．

'                         ．

'0    1    2    3．

'                         ．

'                         ．

'1    3    2    4．

Fire up MATLAB and enter the matrix A.

(a) Find the SVD for A. That is, find U, z, and V.

(b) What is the rank of A?

(c) Find z+ and UT from your results in (b). (Note: You can find z+ using pinv(S), where S is z.)

(d) From your results in (c), find the pseudoinverse of A, ie find A+  = Vz+ UT . (e) By direct calculation, show that A+ obeys the Moore-Penrose conditions: