COM4509/6509 Coursework Part 2

Hello, This is the second of the two parts. Each part accounts for 50% of the overall            coursework mark and this part has a total of 50 marks available. Attempt as much of this as you can, each of the questions are self-contained and contain some easier and harder bits   so even if you can't complete Q1 straight away then you may still be able to progress with  the other questions.

Overview

This part of the assignment will cover:

•     Logistic regression and PCA: 13 Marks, lecture 6 and 8.

•     Neural networks: 16 Marks, lecture 7 and 8.

•     Auto-encoders: 16 Marks, lecture 8 and 9.

What to submit

•     You need to submit two jupyter notebooks (not zipped together) and a pdf copy of part 2, named:

assignment_part1_ [username] .ipynb

assignment_part2_ [username] .ipynb

assignment_part2_ [username] .pdf

replacing [username] with your username, e.g. abc18de.

•     Please execute the cells before your submission. The pdf copy will be used as a backup in case the data gets corrupted and since we cannot run all the notebooks  during marking. The best way to get a pdf is using Jupyter Notebook locally but if  you are using Google Colab and are unable to download it to use Jupyter then you can use the Google Colabfile → print to get a pdf copy.

•     Please do not upload the data files used in this Notebook. We just want the two python notebooks and the pdf.

Assessment Criteria

•     The marks are indicated for each part: You'll get marks for correct code that does what is asked and gets the right answer. These contribute 45. You should make sure any figures are plotted properly with axis labels and figure legends.

•     There are also 5 marks for "Code quality" (includes both readability and efficiency).

Late submissions

We follow the department's guidelines about late submissions, Undergraduate handbook link. PGT handbook link.

Use of unfair means

This is an individual assignment, while you may discuss this with your classmates, please make sure you submit your own code. You are allowed to use code from the labs as a basis of your submission.

"Any form of unfair means is treated as a serious academic offence and action may be taken under the Discipline Regulations." (from the students Handbook).

Reproducibility and readibility

Whenever there is randomness in the computation, you MUST set a random seed for       reproducibility. Use your UCard number XXXXXXXXX (or the digits in your registration    number if you do not have one) as the random seed throughout this assignment. You can set the seeds using torch.manual_seed(XXXXX) and np.random.seed(XXXXX). Answers for each question should be clearly indicated in your notebook. While code segments are      indicated for answers, you may use more cells as necessary. All code should be clearly     documented and explained. Note: You will make several design choices (e.g.                   hyperparameters) in this assignment. There are no“standard answers”. You are             encouraged to explore several design choices to settle down with good/best ones, if time permits.

Question 1: Logistic regression and PCA [13 marks]

MedMNIST is a collection of healthcase based datasets that are pre-processed to match to  format of the original MNIST dataset. In this questions, you will perform logistic regression and dimension reduction using PCA on the PneumoniaMNIST dataset from the                MedMNIST. The task for this dataset is to detect whether a chest X-ray shows signs of        Pneumonia or not and is therefore a binary classification task.

1.1: Data download [1 mark]

The code cell belows provides the code to download the dataset as a compressed numpy file directly from the MedMNIST website. If you prefer, you can follow the instructions at https://github.com/MedMNIST/MedMNIST to download and load the data.

import numpy as np

import urllib .request

import os

#  0o川n?oep  斗V?  pe斗es?斗  斗o  斗V?  ?oOe?  }o?p?J

urllib .request .urlretrieve( 'https://zenodo .org/record/6496656/files/

pneumoniamnist .npz?download=1' , 'pneumoniamnist .npz')

#  7oep  斗V?  OomdJ?ss?p  nnmd(  eJJe(  }工??

dataset = np.load( './pneumoniamnist.npz')

#  TV?  ?oep?p  pe斗es?斗  Don斗e工ns  ?eDV  eJJe(  工n斗?Jne??(

for key in dataset .keys():

print(key, dataset[key] .shape, dataset[key] .dtype)

train_images (4708, 28, 28) uint8

val_images (524, 28, 28) uint8

test_images (624, 28, 28) uint8

train_labels (4708, 1) uint8

val_labels (524, 1) uint8

test_labels (624, 1) uint8

1.1a After downloading the data, merge the validation set into the training set and reshape the images so that each is a 1D array. Then scale the pixel values so they are in the range    [0,1].

#  川J工斗? (onJ  Dop?  V?J? .

1.2: Dimensional reduction and training [6 marks]

1.2a Using the Scikit-learn PCA class, transform the training and test data into at least        seven different sets of reduced dimensions, i.e create 7 alternate datsets with (k1 , k2 , .. . , k7 ) number of features. Briefly explain your choice reduced features. Keep a copy of the         unreduced data so that in total you have eight datasets.

You should fit the tranformation based on the training data and use that to transform the test data. You can find details of the PCA transformation class here.

#  川J工斗? (onJ  Dop?  V?J? .

1.2b Train eight logistic regression classifiers (LRC): one on the original features           (unreduced), and seven on PCA features with seven different dimensions in 1.2a, i.e., LRC on k1 PCA features; LRC on k 2 PCA features; ..., LRC on k7 PCA features and LRC on the      unreduced data. You will need to decide on any options for the logistic regression fitting and explain which choices you make. You can use the Scikit Learn Logistic Regression    classifier, further information is given here.

#  川J工斗? (onJ  Dop?  V?J? .

1.3: Model evaluation [6 marks]

1.3b For each of the trained classifiers in 1.2b, calculate the classification accuracy on the training data and the test data. Extract the total explained variance by summing the         PCA.explained_variance_ratio_ for each of your PCA transformations. Plot the       training accuracy and test accuracy against the total explained variance at each k n. You     should include the results for the case trained on the original features, which corresponds to a total explained variance of 1.

#  川J工斗? (onJ  Dop?  V?J? .

1.3b Describe at least two relevant observations from the evaluation results above.

#  川J工斗? (onJ  ensw?J  V?J? .

Question 2: Convolutional neural networks for image recognition [16 marks]

Fashion-MNIST is a dataset of Zalando's article images. It consists of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image,        associated with a label from 10 classes: 0=T-shirt/top; 1=Trouser; 2=Pullover; 3=Dress;     4=Coat; 5=Sandal; 6=Shirt; 7=Sneaker; 8=Bag; 9=Ankle boot.

It is available online at https://github.com/zalandoresearch/fashion-mnist but here we will use the version built into PyTorch as part of the TorchVision library see here for    documentation.

In this question, you should PyTorch to train various forms of neural network models to classify these images. You can refer to Lab 7 on how to define and train neural networks with PyTorch.

2.1: Data download and inspection [3 marks]

2.1a Use the PyTorch Torchvision API to load both the train and test parts of the Fashion- MNIST dataset. You can use the code used in Lab 7 to load the CIFAR10 as a basis for this.

#  川J工斗? (onJ  Dop?  V?J? .

2.1b Use the torch.utils.data .random_split function to split the 60,000 training set into 2 subsets: the first part will be used for training, the second part will be used for        validation. You must choose a sensible split of this into the training and validation sets.     Create a DataLoader for each of the train, validation, and test splits.

#  川J工斗? (onJ  Dop?  V?J? .

2.1c Display 2 example images from each of the classes (20 images in total).

#  川J工斗? (onJ  Dop?  V?J? .

2.2: Network training [8 marks]

In this section you will train a set of neural network models to classify the Fashion-MNIST  data set. Only the number of convolutional (Conv) layers and the number of fully connected (FC) layers will be specified below. You are free to design other aspects of the network. For example, you can use other types of operation (e.g. padding), layers (e.g. pooling, or           preprocessing (e.g. augmentation), and you choose the number of units/neurons in each    layer. Likewise, you may choose the number of epochs and many other settings according  to your accessible computational power. You should choose sensible values for the batch    size and learning rate. If you wish, you may use alternate optimisers, such as Adam.

When training each model you should keep track of the following values:

1.    Training accuracy

2.    Validation accuracy

3.    Test accuracy

Remember the accuracy is the number of correct classifications out of that portion of the dataset.

2.2a Train a neural network composed of 2 fully connected layers with an activation       function of your choice. Train the model on the training set, use the validation set to choose the best design among at least three different choices, and test the chosen model on the   test set.

Remember that your dataloader will give you a 2D image. The CNNs can process these but your fully connected (nn.Linear) layers are expecting each sample to be a vector.

#  川J工斗? (onJ  Dop?  V?J? .

2.2b Define and train using a neural network composed of 2 convolutional layers and 2 fully connected layers. Train the model on the training set, use the validation set to        choose the best design among at least three different choices, and test the chosen model on the test set.

#  川J工斗? (onJ  Dop?  V?J? .

2.2c Train a neural network composed of 3 convolutional layers and 3 fully connected layers. Train the model on the training set, use the validation set to choose the best design among at least three different choices, and test the chosen model on the test set.

#  川J工斗? (onJ  Dop?  V?J? .

2.3: Comparison of model performance [5 marks]

2.3a In separate plots, show the training accuracy, validation accuracy and test accuracy for each of these models.

#  川J工斗? (onJ  Dop?  V?J? .

2.3b Describe at least two observations of the data plotted in this section.

#  川J工斗? (onJ  ens川?J  V?J?

3. Denoising Autoencoder [16 marks]

The CIFAR-10 dataset

In this assignment, we will work on the CIFAR- 10 dataset collected by Alex Krizhevsky,      Vinod Nair, and Geoffrey Hinton from the University of Toronto. This dataset consists of     60,000 32x32 colour images in 10 classes, with 6,000 images per class. Each sample is a 3- channel colour images of 32x32 pixels in size. There are 50,000 training images and 10,000 test images.

3.1: Data loading and manipulation [3 marks]

3.1a Download both the training and test data of the CIFAR- 10 dataset, e.g., by following the pytorch CIFAR10 tutorial. You can also download via other ways if you prefer.

#  Write your  code  here .

3.1b Add random noise to all training and test data to generate noisy dataset, e.g., by       torch.randn(), with a scaling factor scale, e.g., original image + scale * torch.randn(), and  normalise/standardise the pixel values to the original range, e.g., using np.clip(). You may choose any scale value between 0.2 and 0.5.

A random transformation can be applied using a Lambda transform when composing the load data transform, which looks a little like this:

transforms .Lambda(lambda x: x + ..... )

Note: Before generating the random noise, you MUST set the random seed to your UCard   number XXXXXXXXX for reproducibility, e.g., using torch.manual_seed(). This seed needs to be used for all remaining code if there is randomness, for reproducibility.

You may want to create separate dataloaders for the noisy and clear images but make sure they are not shuffling the data so that correct pair of images are being given as input and   desired output.

#  Write your  code  here .

3.1c Show 20 pairs of original and noisy images.

#  Write your  code  here .

3.2 Applying a Denoising Autoencoder to the modified CIFAR10 [10 marks]

This question uses both the original and noisy CIFAR- 10 datasets (all 10 classes). Read about denoising autoencoders at Wikipedia and this short introduction or any other   sources you like.

3.2a Modify the autoencoder architecture in Lab 8 so that it takes colour images as input (i.e., 3 input channels).

#  Write your  code  here .

3.2b Training: feed the noisy training images as input to the autoencoder defined above; use a loss function that computes the reconstruction error between the output of the     autoencoder and the respective original images.

#  Write your  code  here .

3.2c Testing: evaluate the autoencoder trained in 3.2b on the test datasets (feed noisy      images in and compute reconstruction errors on original clean images. Find the worst      denoised 30 images (those with the largest reconstruction errors) in the test set and show them in pairs with the original images (60 images to show in total).