Data Mining: 625.740
Homework for Module 3
1. Let the conditional densities for a two-category one-dimensional problem be given by the
Cauchy distribution
p(x|ωi) = 1
πb ·
1
1 +
x−ai
b
?2
, i = 1, 2.
(a) If P(ω1) = P(ω2), show that P(ω1|x) = P(ω2|x) if x = (1/2)(a1 + a2). Sketch P(ω1|x)
for the case a1 = 3, a2 = 2, b = 5. How does P(ω1|x) behave as x → −∞? as x → ∞?
(b) Using the conditional densities in part a, and assuming equal a priori probabilities, show
that the minimum probability of error is given by
P(error) = 1
2
−
1
π
tan−1
a2 − a1
2b
.
Sketch this as a function of |(a2 − a1)/b|.
2. The Poisson distribution for discrete k, k = 0, 1, 2, . . . and real parameter λ is
P(k|λ) = e
−λ λ
k
k!
.
(a) Find the mean of k.
(b) Find the variance of k.
(c) Find the mode of k.
(d) Assume two categories C1 and C2, equally probable a priori, distributed with Poisson
distributions and λ1 λ2. What is the Bayes classification decision?
(e) What is the Bayes error rate?
3. Let p(x|ωi) ∼ N(µi
, σ2
I) for a two-category k-dimensional problem with P(ω1) = P(ω2) = 1
2
.
(a) Find Pe, the minimum probability of error.
(b) Let µ1 = 0 and µ2 = (m1, . . . , mk)
T 6= 0. Show that Pe → 0 as the dimension k
approaches infinity. Assume that P∞
k=1 m2
k → ∞.
4. Under the assumption that λ21 λ11 and λ12 λ22, show that the general minimum risk discriminant function for a classifier with independent binary features is given by g(x) = wT x + w0.
What are w and w0
