Data Mining: 625.740
Homework for Module on Linear Regression
1. Show that, if x, y are jointly Gaussian, the regression of y on x is given by
E(y|x) = ασyx
σx
+ µy −
ασyµx
σx
, where Σ = ?
σ
2
x ασxσy
ασxσy σ
2
y
?
.
2. Given data (X1, Y1), . . . ,(Xn, Yn), consider the regression through the origin model
Yi = βXi + νi
, where E(νi
|Xi) = 0 and Var(νi
|Xi) = σ
2
.
(a) Find β, ˆ the least squares estimate for β.
(b) Find the standard error of the estimate, q
Var(βˆ).
(c) Find conditions that guarantee that the estimate is consistent:
∀ε 0, P(|βˆ − β| ε) → 0 as n → ∞.
3. The columns in the file polynomial data.txt are the X and Y values of a polynomial function
y =
Pn
k=0 akx
k with added Gaussian noise.
(a) For each n ∈ {3, 4, 5} , fit an n
th degree polynomial to the data. What would you say is
the most likely value of n?
(b) Estimate the level of the noise.
