闪电代写 -代写CS作业_CS代写_Finance代写_Economic代写_Statistics代写_代码代做_IT代写_加急帮助

Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Introduction to Machine Learning

(CSCI-UA.473): Homework 2

Submission Instructions

You must typeset the answers using LATEX and compile them into a single PDF file. Name the pdf file as 〈 Your-NetID 〉 hw2.pdf and the notebook containing the coding portion as 〈 Your-NetID 〉 hw2.ipynb. The PDF file should contain solutions to both the theory portion and the coding portion. Submit the files through the following Google Form - https://forms.gle/Rf63VnEaMoLcWr7p7 The due date is October 4, 2022, 11:59 PM. You may discuss the ques- tions with each other but each student must provide their own answer to each question.

Questions

Question 1: Empirical vs. Expected Cost (10 points)

We approximate the true cost function with the empirical cost function defined by:

Ex [E(g(x),f(x))] = 工 E(g(xi ),yi ), (1)

i=1

where N is the number of training samples, f is the unknown function, g is the learnable function, E is the cost function, yi is the label associated with the input xi . In Eq. 1, the left-hand side of the equation represents the expected value of the cost between g(x) and f(x) for every x in the dataset, and the right-hand side approximates this expectation by computing a mean over the errors assigning equal weight to each sample. In the above equation is it okay to give an equal weight to the cost associated with each training example? Given that we established that not every data x is equally likely, is taking the sum of all per-example costs and dividing by N reasonable? Should we weigh each per-example cost differently, depending on how likely each x is? Justify your answer.

Question 2: Simple Linear Regression Model (10 points)

Consider the following model: Yi = 5 + 0.5Xi + ϵi , ϵi N(0, 1)

1. What is E[Y |X = 0], E[Y |X = −2] and Var[Y |X]?

2. What is the probability of Y > 5, given X = 2?

3. If X has a mean of zero and variance of 10, what are E[Y] and Var[Y]?

4. What is Cov(X,Y)?

Question3: Least Squares Regression (10 points)

Consider the linear regression model:

y = θ 1 x1 + θ2 x2 + ... + θk xk + ϵ,ϵ (2)

where y is a dependent variable, xi corresponds to independent variables and θi corresponds to the parameters to be estimated. While approximating a best-fit regression line, though the line is a pretty good fit for the dataset as a whole, there may be an error between the predicted value yˆ and true value y for every data point x = [x1 ,x2 , ...,xk ] in the dataset. This error is captured by ϵ ∼ N(0,σ2 ), where for each data point with features xi , the label yˆ is drawn from a Gaussian with mean θTx and variance σ 2 . Given a set of N observations, provide the closed form solution for an ordinary least squares estimate θˆ for the model parameters θ .

For the ordinary least squares method, the assumption is that Var(ϵi |Xi ) = σ 2 , where σ is a constant value. However, when Var(ϵi |Xi ) = f(Xi ) σ 2 , the error term for each observation Xi has a weight Wi corresponding to it. This is called Weighted Least Squares Regression. In this scenario, provide a closed form weighted least squares estimate θˆ for the model parameters θ .