Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Deep Learning CS583 Fall 2022

Homework Assignment 1

1  Question

Choose a software framework. A few candidates:

● Marvin “http://marvin.is/”

● Tensorflow “https://www.tensorflow.org/”

● Caffe “http://caffe.berkeleyvision.org/”

● Pylearn2 “http://deeplearning.net/software/pylearn2/”

Download the benchmark dataset MNIST from “http://yann.lecun.com/exdb/mnist/”. Implement multiclass logistic regression and try it on MNIST. Comments: MNIST is a stan- dard dataset for machine learning and also deep learning.  It is good to try it on one-layer neural networks (i.e., logistic regression) before multilayer neural networks.  Downloading the dataset from other places in preprocessed format is allowed, but practicing how to read the dataset prepares you for other new datasets you may be interested in. Also, it is recom- mended to try different initializations and learning rates to get a sense of how to tune the hyperparameters (remember to create and use validation dataset!).

Check the tutorials for some of the parameters (e.g., “https://machinelearningmastery.com/how-   to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/”).

● Build a three-layer feedforward network:  [6 points]

z → h1  → h2  → p(y|h2 )．                                             (1)

The hidden layers h1  and h2  have dimension 500.  Train the network for 250 epochs1 and test the classification error. Do not use regularizations. Plot the cross-entropy loss on the batches and also plot the classification error on the validation data.

● Repeat the above experiment, but train the network with the following regularizations and compare with the results in the previous experiment:

L2 regularization [4 points]

Comments: no need to implement them on your own; the software framework typically provides implementations for L2 regularization and dropout.