Homework 4,
1. Implement the TISP variable selection method for classification (as described in
pages 12 of the Regularized Loss course notes), with the hard-thresholding penalty,
described in page 11 (with η = 0). Take special care to normalize each column of
the X matrix to have zero mean and variance 1 and to use the same mean and standard
deviation that you used for normalizing the train set also for normalizing the test set.
a) Using the Gisette data, train a TISP classifier on the training set, starting with
w(0) = 0, with 100 iterations. Find appropriate λ-s to select approximately
10, 30, 100, 300, 500 features. Plot the misclassification error vs the number of
selected features. Report in a table these misclassification errors on the training
and test set, the corresponding numbers of selected features and the values of λ.
(4 points)
b) Repeat point a) on the dexter dataset. (2 points)
c) Repeat point a) on the madelon dataset. (2 points)
1
