Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Machine Learning, 2023 Spring

Assignment 3

Notice

Plagiarizer will get 0 points.

LATEXis highly recommended. Otherwise you should write as legibly as possible.

Requirement Provide a numerical experiment for SGD algorithm, and submit the results in a report. Both code and report need to be submit. Contents of the report:

1. Background: Introduce the purpose, background of the experiment.

2. Data acquisition: How the data is generated.

3. Algorithm:  The optimization model and the applied alogrithm, especially the alogrithm parameters.

4. Experiment result for the following taskes:

(a)  SGD with fixed step length. (At least 3 different step length.)

(b)  SGD with decresing step length.

(c) Demonstrate early termination.  (If the result is not perfect, please demonstrate it in discussion.)

(d) Compare GD and SGD under large dataset.

Tips

(a) The programing language is not restricted.(Python / Matlab)

(b) The optimization model should be suitable for GD/SGD algorithm.  e.g.  Logistic regression / Least square with about 10 variable.

(c) Choose enough training data N .N should be no less than 103 .  You can also approximate the distribution, and out of sample error with more data generated. (First genreate ≈ 10 ∗ N data points, to approximate the distribution, and sample N training data in it. The out of sample error can also be evluated with the approximate distribution.)