l(x₁, x₂, · · · , x_n; p) = n log p +

n

i=1

xiΣ

− nΣ

log(1 − p).

(iii) Show the MLE for p is 1/x¯.

Exercise 3

Let  X Poi(λ),  a Poisson  distribution with parameter   λ  >  0. This is a discrete probability distribution on the non-negative integers, with p.m.f. given by

f_X(k) =

λk −λ

k!

, k = 0, 1, 2, · · ·

(i) Show that E X = λ.

(ii) Show that the log-likelihood function for the parameter λ is given by

l(x₁, x₂, · · · , x_n; λ) =

n

i=1

xiΣ

log λ − nλ − log(x₁!x₂! · · · x_n!).

(iii) Show the MLE for λ is x¯.

Exercise 4

Let X Binom(N, p), a binomial distribution with parameters N and p. This is a discrete probability distribution on integers between 0 and N, with p.m.f. given by

f   (k) = .NΣp^k(1 − p)^N−k,    k = 0, 1, 2, · · · , N

(i) Show that E X = Np.

(ii) Suppose the parameter N is already known. Show that the log-likelihood function for the parameter p is given by

l(x₁, x₂, · · · , x_n; p) =

n

i=1

xiΣ

log p +

.nN −

n

i=1

xiΣΣ

log(1 − p)

+ log ..NΣ.NΣ · · · .N ΣΣ .

(iii) Show the MLE for p is x¯/N .

Exercise 5

Let X, Y  be random variables satisfying the relation Y  = α+ βX + ε, where ε    (0, σ²) is independent from X. This models a linear relationship between Y and X, subject to some random error ε. Suppose (x₁, y₁), (x₂, y₂), , (x_n, y_n) are i.i.d. sample outcomes of (X, Y ).

In lecture, we computed the log-likelihood function to be

n

l(⃗x, ⃗y; α, β, σ²) = − n log(2πσ²) −   1   Σ(y  − α − βx )

= − n log(2πσ²) −   1   ∥⃗y − α⃗1 − β⃗x∥2,

where ⃗x  :=  (x₁, x₂, , x_n),  ⃗y  :=  (y₁, y₂, , y_n),  and ⃗1  :=  (1, 1, , 1) Rn.  We  further computed the MLE for α and β to be

ˆ ˆ (⃗x − x¯⃗1) · (⃗y − y¯⃗1)

n i=1

(x_i − x¯)(y_i − y¯)

αˆ = y¯ − βx¯,  and β  =

=

∥⃗x − x¯⃗1∥2

n

i=1

(x_i

− x¯)2 .

We concluded that the line y = αˆ + βˆx is the least squares regression line for the sample points (x₁, y₁), (x₂, y₂), · · · , (x_n, y_n).

(i) Using the likelihood function given above, show that the MLE for σ² is

σˆ² =  1 ∥⃗y − α⃗1 − β⃗x∥2  =  1 Σ(y

− α − βx ) .