EESM5900O Assignment 2
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
EESM5900O Assignment 2
Deadline: 2023-12-1, 6 PM
full marks: 100 points
Instructions:
For this assignment, you need to complete the following three tasks. Please compile your solutions into a single PDF file, clearly labeling each solution with the corresponding task/- figure number.
Task 0: Preparation
This part aims at introducing how to run a python script. We provide two methods here, including running on a local computer and running on Google Colab. We will use the file MNIST training.py as a demo, which is posted on Canvas. If you are familiar with python,
you can skip this section.
Running on a local computer
There are various Python IDEs and code editors for beginners and professionals, including Sublime, Vscode, and PyCharm. Here, we introduce steps to run the python script in Terminal.
1. Open Terminal.
. For the Windows users, please follow this link https://www.wikihow.com/Open- Terminal-in-Windows.
. For the Mac users, you can use spotlight search (command + space) to find Terminal.
. For the Linux users, using shortcut (CTRL+ALT+T) can open Terminal.
2. Redirect the working directory to the directory that contains the desired python script.
cd [PATH]
3. run the script. Here we set the learning rate (--lr) to 0.02.
python3 MINST training . py − −
lr =0.02
Google Colab is a Cloud-based Jupyter notebook environment. With its simple and easy- to-use interface, Colab helps us get started with your coding journey with almost no setup. On Colab, we can either run a Jupyter file ( . ipynb) or a Python file ( . py). For the first use, please refer to this tutorial1. Now we introduce how to run Python script files on Google Colab.
. Step 1: Open Google Golab (https://colab.research.google.com), login with your Google account, and create a new notebook.
. Step 2: Upload the Python files, as shown in Fig. 0a.
Figure 0a: Upload files.
. Step 3 (Optional): Choose GPU as the Hardware accelerator, as shown in Fig. 0b.
. Step 4: Run aPython file with arguments (e.g., --argument=hyperparmater, --action). Note the exclamation mark cannot be omitted.
! python3 MINST training . py − −
lr =0.02
Task 1: MNIST training task [20pt]
Run the demo codes MINST_training. py posted on Canvas. Note that you need to install the PyTorch package if you want to run the code on the local computer.
1. Set the training epoch to 10 and run the codes. Show the screenshot of the logs in Fig. 1a. An example of the screenshot is shown in Fig. 1b. [15 pt]
2. Describe the change of training loss with respect to the training progress(epochs). What does it imply? [5 pt]
Figure 0b: Use GPU to accelerate the codes.
Figure 1b: An example of the screenshot.
In this problem, we would like to investigate the computation complexity of a neural network. Particularly, we will use the time package to measure the inference latency of each layer. The following is an example to illustrate how to use the time package.
import time
start time = time . time ()
main ()
end time = time . time ()
print ( ” Running time : ” , end times − start time )
def forward ( self , x ) :
time flag1 = time . time ()
x = self . conv1 (x)
time flag2 = time . time ()
print ( ” Latency of conv1 : ” , time flag2 − time flag1 )
1. Please report the inference time of each layer and fill in Table 2a.
2. Please calculate the number of parameters in each layer. For example,a fully-connected layer with 8 input neurons and 10 output neurons has 8 × 10 weights and 10 biases. Thus, the number of parameters is 90. Please fill in Table 2a.
Table 2a: Inference latency and number of parameters
Task 3 Federated Training [40pt]
The codes for federated learning(FL) training are provided at Canvas. You can simply run experiments with different hyperparameters, e.g.,
python main . py −−frac =0.2
Problem 3.1
Training loss is a metric used to assess how an ML model fits the training data. To observe the training convergence, we can plot a curve of the training loss vs. the number of commu- nication rounds. When the loss value (almost) stops varying, we can tell that the training process has converged.
In this problem, please run the experiments with the default setup, and plot the curve of training loss(i.e., training loss as a function of communication round) for 100 communication rounds.
1. When does the FL training almost converge?
2. Give one possible way to speed up the convergence.
3. Give one possible way to improve model performance.
Test accuracy and test loss are computed on the test dataset, which show the generaliza- tion performance of a model. Let clients perform the FL training after a given number of communication rounds with the following setups2. Please write down the test accuracy and test loss in the following Table. Comparing the IID and non-IID case, what can you find according to the results?
Table 3a: Train loss and test accuracy in different FL setups.
2023-11-28