Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Machine Learning and Data Mining I (DS 4400)

Homework 4

1. Neural Network. Consider a neural net for a binary classification which has one hidden layer as shown in the figure below. We use a linear activation function a(2) = c2 at hidden units and a sigmoid activation function a(2)  =  1/(1 + e 3 ) at the output unit to learn the function for P (y = 1|zw) where z = (z1 . z2 ) and w = (w1 . w2 . 8 8 8 . w9 ).

 

(a) What is the output P (y = 1|z. w) from the above neural net? Express it in terms of zi . c and weights wi . What is the final classification boundary?

(b) Draw a neural net with no hidden layer which is equivalent to the given neural net, and write weights  of this new neural net in terms of c and wi .

(c) Is it true that any multi-layered neural net with linear activation functions at hidden layers can be represented as a neural net without any hidden layer? Explain your answer.

2.  Feedforward Neural Network.  Consider a three-layer neural network to learn a function f  :  x  → y, where x  =  [x1 . x2] consists of two features.  The weights w1 . 8 8 8 . w6  can be arbitrary. There are two possible choices for the function implemented by each unit in this network:

 

– S: sigmoid function, s(2) =  ,

 L: linear function, L(2) = c2,

where in both cases 2 =     i wixi . Assign proper activation functions (S or L) to each unit in the following graph so that we can generate functions of the form f (x1 . x2 ) =  at the output of the neural network y . Derive "1 and "2 as a function of w1 . 8 8 8 . w6 and c.

3. Movie Genre Classication using FeedFroward NNs. In this exercise, you will implement a Feed Forward Neural Network using PyTorch to classify movie genre from movie descriptions.

Feature Extraction.  We have extracted the feature vectors from the movie descriptions and pro-      vided them on piazza. The feature vector is generated as follows. After cleaning the text, Google’s      pretrained word2vec model (gensim) is used to get a xed-length vector representation for each      word. To calculate the feature vector for the whole description, a weighted average is computed.      The weights correspond to the TF-IDF value of the words which gives lesser weights to common      words, and vice-versa. You can take a look at word2vec model here (https://radimrehurek.com/gensim/) if you are interested, however, this is not necessary for the homework. You can use the given fea-      ture representations.

(a) Download the dataset of HW4 (Click Here!), which consists of three les:  train .txt, validation .txt and test .txt.  Each line of a le contains a movie name, movie genre and a short description of the movie.  The task is to classify movie genre using the given description. Read the dataset and report the data statistics (number of classes, number of instances in each class) for train, validation and test sets.

(b) Implement a Feed Forward Neural Network with n layers using PyTorch that takes as input the number of layers n and the number of nodes in each layer, sl  for l ∈ {1. 8 8 8 . n}. The loss function must be the cross-entropy loss. Use an activation function of your choice and also choose optimizer, learning rate, batch size, and epoch.

(c) Train the NN with one hidden layer having 64 neurons on the training dataset. Use the val- idation dataset during training to select the best performing model. Evaluate the best model on the test dataset. Report the mean classification accuracy and mean per class accuracy for train set, validation set and test set. Also, plot training and validation losses with respect to epoch in the same figure.

(d) Train the NN with two hidden layers having 64 neurons each on the training dataset.  Use the validation dataset during training to select the best performing model. Evaluate the best model on the test dataset. Report the mean classification accuracy and mean per class accu- racy for train set, validation set and test set.  Also, plot training and validation losses with respect to epoch in the same figure.

(e) Repeat (c) using 128 neurons for the hidden layer.

(f) Repeat (d) using 128 neurons for both the hidden layers.

(g) Compare the results and explain the effects of the number of hidden layers and number of neurons.

Homework Instructions:

* This homework must be done individually.

* Please submit a pdf copy of the solution on the Canvas page of the course by the deadline. You can scan your handwritten solution or make pdf from your typed solution. Your pdf needs to include the plots and analysis of the programming question. Please also submit your Python source code in addition to the pdf. Both .py and jupyter notebook files are accepted.

* The name of the pdf must be hw4-LN.pdf where you need you to use your last name in place of LN” . Similarly, use hw4-LN.py for the programming part.