Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Math 4570 Matrix Methods for Data Analysis and Machine Learning - Spring 2024

Midterm

1. (5 points) Let P2(R) be the inner product space with polynomials of degree less or equal than 2, where ⟨f, g⟩ is defined to be ⟨f, g⟩ = f(x)g(x)dx.

Let S be the subspace of the inner product space P2(R) generated by the polynomials 6 and 6x

Find a basis for the orthogonal complement of S.

2. (8 points) Find the least squares approximation to the function f(x) = xex by a quadratic function a + bx + cx2 in the interval [0, 1].

(Hint: Use the distance is induced by the inner product ⟨f, g⟩ = f(x)g(x)dx. Use WolframAlpha https: //www.wolframalpha.com/ to do the calculation of integrals if needed.)

3. (7 points) Let be a matrix, and

Let Here the norm || || is the standard l2-norm.

Calculate the gradient and Hessian matrix of the function and find the argminθJ of each function. You can use any results in the lecture notes.

4. (10 points) In this question, you may use Python (with only numpy library) to solve the matrix equation.

Consider the following data points. The mean of each column is zero.

a). Fit a linear model y = θ1x1 + θ2x2 to this dataset when the loss is You should report the best fit function and the RSS cost value.

b). Fit a linear function y = θ1x1 + θ2x2 to this dataset when the loss is the Ridge Loss with λ = 1 and with λ = 10. You should report the best fit function and the RSS cost value.

5. (10 points) Consider the data

Fit the data for i = 1, 2, ..., n = 10 using the model

f(x) = θ0 + θ1x + θ2ex

(1) Find a closed formula for parameters to minimize the RSS loss

Hint: define a new data matrix and use the least squares solution.

(2) Using formula in (1), find the function f(x) fitting the data using the result in (1).

(3) Calculate the RSS cost for your fitting in (2).

6. (10 points) Consider the categorical learning problem consisting of a data set with two labels.

Label 1: (contains 6 data points)

Label 0: (contains 7 data points)

Answer the following logistics regression questions.

(1) Use gradient descent to find the logistic regression model

and the boundary. (Plot the data and boundary, only use numpy and Matplotlib. ) Use initial value learning rate α = 0.02, and 1000 iterations,

(2) Find the probability for a test point for your logistic model in (1). What is the predicted label for ?

(3) (Bonus question. 2 extra points) Find quadratic Logistics Regression method for this question and obtain an quadratic boundary. (Hint: this means to use new features: )