关键词 > DataMining代写

Data Mining Homework 2: Ridge Regression

发布时间：2022-09-28

Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Data Mining

Homework 2: Ridge Regression

Part 1. Download the dataset Cars93.RData from Canvas. This is a processed version of the original data available in the MASS package. The data has been trimmed, re-scaled and splitted into training and testing data matrices (trainX, testX) and response vectors (trainY, testY ).

1. Check the column names of the data using head( ) function. (You can also use colnames( ).)

2. (LASSO) We want to build a LASSO model using one of the values {0.001, 0.01, 0.1, 1, 10} for the hyperparameter λ . Perform 5-fold cross validation using cv.glmnet( ) in the package glmnet on the training set. You can read about input and output of the function by using ?cv.glmnet( ). Which λ value yields the lowest cross validation error?

3. Build a LASSO model using glmnet( ) function with tuned λ . How many features are se- lected by the LASSO model? (How many nonzero coefficients does the model have?)

4. Predict Horsepower on the testing set. Compute the mean squared error. How much horse- power error LASSO makes on average?

5. (Ridge Regression) Use training data and build a ridge regression model to predict Horse- power with all of the features. Use λ = 0.1. In your model, what is the value of the intercept? What is the slope for the features EngineSize?

6. How many nonzero coefficients does the model have?

Part 2. In order to build the ridge regression model, we solve the problem

minL(β) = ∥Y − Xβ∥2(2) + λ∥β∥2(2) . (1)

In this part, you are going to implement your own ridge regression method.

7. Implement your own ridge regression method called ridgemodel( ). The function should take input of a data matrix X, a response vector Y , and a hyperparameter λ . As output, return the coefficients of the ridge regression model.

• Do not use glmnet( ) or any function from other packages.

• Hint:

– You can use the formula derived in class.

– Make sure to append a column of 1 to data matrix X to produce a model with an intercept.

8. Run your function with the training data and λ = 0.1. Verify the obtained model is compa- rable to the model obtained from Question 5.

Part 3. Gradient descent is an algorithm that finds a solution of a minimization problem by itera- tively moving along a descending direction. The goal of this part is to try an alternative approach for constructing a ridge regression model using the gradient descent algorithm.

To minimize the problem (1) and compute the ridge regression coefficients, follow the steps below:

(0) Before you begin,

• Verify that ∇L(β) = −2XT(Y − Xβ) + 2λβ .

• Append a column of 1 to data matrix X .

(1) Let k = 0. Choose an initial point β 0 as a vector of all 0. Choose step size α = 0.0005, and λ = 0.1.

(2) Iteratively update

βk+1 ← βk − α∇L(βk )

k ← k + 1

(3) Terminate the algorithm when k reaches 1000.

(4) (Optional) For each iteration, save the function value L(βk ). After the algorithm terminates, plot the iteration count (k) and the function value L(βk ). This plot helps to visualize con- vergence of the algorithm.

9. Verify that the ridge regression model obtained by the above procedure is comparable to the model from Question 5.