G53MLE-E1 Question 1: Foundations of Machine Learning spring 2016
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
G53MLE-E1
Question 1: Foundations of Machine Learning
(a) It is said that generalisation is the ultimate goal of machine learning. Explain what this term means, and name two general techniques (i.e. not specific to any particular machine learning method) that will improve generalisation.
(6 marks)
(b) Explain carefully the following four machine learning problems: (i) binary classification, (ii) regression, (iii) multiple independent binary classification, and (iv) 1-of-K multiclass classification. Use set and numerical domain notation to denote the input and output spaces of each problem.
(8 marks)
(c) (i) Explain the difference between a linear binary classifier and a non-linear binary classifier. (ii) Draw two diagrams: one with a set of 2-dimensional data points of two classes for which a linear classifier would be a good solution, and one where a non-linear classifier would be the better choice. (iii) draw two diagrams, one in which data is separable, and one in which there is classoverlap.
(11 marks)
(d) (i) Draw a set of 2-dimensional linearly separable data points with binary labels, and clearly indicate the decision boundary found by a maximum-margin classifier, and at least one decision boundary that does not obtain a maximum margin but still represents a perfect classifier. (ii) Explain how it’s possible to obtain such a non-maximum margin solution, using the concept of a loss function.
(9 marks)
SECTION B
Question 2: Artificial Neural Networks and Deep Learning [overall 33 marks]
Below is given a training set of eruption duration and time to next eruption for two type of geyser eruptions of The Old Faithful. This will be used for questions 2a, 3a and 3d.
|
3.6 |
79 |
1 |
|
1.8 |
54 |
-1 |
|
3.3 |
74 |
1 |
|
2.3 |
62 |
-1 |
|
4.5 |
85 |
1 |
|
2.9 |
55 |
-1 |
|
4.7 |
88 |
1 |
|
3.6 |
85 |
1 |
|
2.0 |
51 |
-1 |
|
4.4 |
85 |
1 |
|
1.8 |
54 |
-1 |
(a) Draw a diagram of an ANN’s topology that can learn this pattern based on the given data, using a single hidden layer with three units. Name all relevant elements using indices that indicate source and target layer numbers, and ensure you account for biases. Initialise all weights to 1.
(6 marks)
(b) How can the topology of an Artificial Neural Network with one hidden layer be designed to effectively bring about dimensionality reduction?
(4 marks)
(c) Design a Deep-Learning architecture using Convolutional Neural Networks as one of a number of components for the task of classifying images into images that contain cats and images that don’t. Name the different layers/components of the network; explain what function they have, and provide the cost function used in the output layer to inform back-propagation.
(12 marks)
(d) Explain how the ReLU revolutionised Deep Learning, by explaining the concept of the vanishing gradient. Give a diagram of the ReLU and compare this with at least one other activation function to illustrate your explanation.
(11 marks)
Question 3: Linear Classification and Regression [overall 33 marks]
Below is given a training set of eruption duration and time to next eruption for two type of geyser eruptions of The Old Faithful. This will be used for questions 2a, 3a and 3d.
|
X1: Eruption duration (min) |
X2: Time to next eruption (min) |
Y: Eruption Type |
|
3.6 |
79 |
1 |
|
1.8 |
54 |
-1 |
|
3.3 |
74 |
1 |
|
2.3 |
62 |
-1 |
|
4.5 |
85 |
1 |
|
2.9 |
55 |
-1 |
|
4.7 |
88 |
1 |
|
3.6 |
85 |
1 |
|
2.0 |
51 |
-1 |
|
4.4 |
85 |
1 |
|
1.8 |
54 |
-1 |
(a) Give the general formula for logistic regression for classification, and give the formulation a suitable cost function if one wanted to predict the Eruption Type from the Eruption duration and Time to next eruption features. Initialise the model by setting all parameters to value one and give the prediction for all 11 examples using that model
(8 marks)
(b) Give pseudo-code for gradient descent, and explain why this is a frequently used technique for finding optimal solutions.
(8 marks)
(c) Is it in general possible to use exhaustive search to find the optimal parameters in a parametric algorithm? Justify your answer. Would it be feasible, in this particular case, to search only sensible solutions? If so, how many values need to be evaluated?
(7 marks)
(d) Find the optimal weights for the classification problem for the geyser Eruption Type data. You do not necessarily have to use gradient descent.
(10 marks)
2022-01-15