Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

EECE5644 Introduction to Machine Learning

Midterm Exam

1.)  Maximum likelihood estimation.  In this problem, Y=B+1 with probability v, and Y=B with probability 1-v, where v is known. You must estimate B given Y.

a.   Show that the likelihood function is given by fY | B (y|B)=1-v, for B=y, and fY | B (y|B)=v for B=y-1.

b.   For v=1/4, find the maximum likelihood estimator for B using Y=y.

c.   Suppose that {Nj } j=1,..D is an independent set of identically distributed Bernoulli trials, with P[Nj=1]=v, and P[Nj=0]=1-v.  Both D and v are known, and suppose       that Yj=B+Nj, j=1,…D.  Note that the same realization of B is used for all the Yj,       and that each Yj equals B or B+1 independently for each index j.  Show that Ymin      =min{Y1, Y2, Y3, …YD } is sufficient to estimate B given the observation Y1, Y2, Y3,      …YD .  (A statistic t(Y1, Y2, Y3, …YD ) is sufficient for B if the likelihood function of B

for the measurements Y1, Y2, Y3, …YD depends on the measurements only through t.)

d.   The likelihood function for B given Ymin  is f(ymin |B)=1-vD, for ymin=B, and                  f(ymin |B)=vD, for ymin=B+1.  Find the maximum likelihood estimator for B given Y1,

Y2, …YD .

2. Bayesian estimation.  Suppose {Yj } are defined as in problem 1.  Additionally, it is known that parameter B has a probability density function fB (b)=e-b, b>=0.

a.   Using the prior density of B only, what is an estimator of B?  Provide an explanation of your choice of estimator.

b.   Using the given information in 1.d, as well as the prior distribution, what is the                maximum a posteriori estimator of B given {Y1, Y2,… YD }?  Explain your answer carefully, providing all calculations.

3. Bayesian classification.  Consider the binary Bayesian classification problem.  Let class 1 have prior probability 1/3 and class 2 have prior 2/3.  Suppose that we have uniform costs.  Let the   measurement x have density f1 (x)=K e-x, 0<=x<=1, and 0 otherwise under class 1.  Suppose that the measurement x has density f2 (x)=K e-(1-x), 0<=x<=1, and 0 otherwise under class 2.

a.   Find the Bayesian optimal classifier in this case.  Completely specify the decision region for class 1 using the x axis.  Simplify your decision rule as much as possible.

b.   Find an expression for the probability of error of the Bayesian optimal classifier. Simplify your result as much as possible.