Assignment 5 Manifold Learning and Pattern Classification

Machine Learning and Adaptive Systems
(ECE656)
Computer Assignment 5 (Manifold Learning and Pattern Classification)
The purpose of this computer assignment is to use the Laplacian Eigenmaps (LE)
manifold learning method as a nonlinear feature extraction method for pattern
classification applications. The database to use for this computer assignment is ORL face
dataset https://cam-orl.co.uk/facedatabase.html. This dataset contains 400 facial images
of 40 individuals. For every individual, there are 10 images each of size 92 x 112 pixels.
The images were taken at different times, varying the lighting conditions, facial
expressions (open-closed eyes, smiling-not smiling) and facial details (glasses-no
glasses). All the images were taken against a dark homogeneous background with the
subjects in an upright, frontal position (with tolerance for some side movement). In this
computer assignment, we choose only 50% of this database for 20 different individuals.
1. One image from each subject should be randomly selected for testing, while the rest
of the images are used for training of the standard LE to generate the sub-manifold in
low dimensional space. Try two different sub-manifold dimensions e.g., M=50 and
100. Repeat this experiment 10 times so that every image of each subject can serve
as the testing sample once. The testing images should be learned (embedded)
incrementally by the algorithm covered in the lecture 26.
2. Use a k-nearest neighbor (KNN) classifier for k=7 to perform classification of the
testing dataset for 20 classes (different individuals) based upon their low dimensional
features in the sub-manifold domain. Provide the classification results using a
confusion matrix and comment on the performance of (a) the incremental sample
embedding method, and (b) the K-NN classifier that used the low dimensional
features for decision-making.
3. Bonus Points (20%): Provide a comparison of the incremental learning with the
batch LE in terms of the ability to embed points in the sub-manifold space and
capture the intrinsic structural information of the input manifold as well as the
classification results using the K-NN classifier.
4. Provide a detail discussion on your results and point out the advantages/disadvantages
of the manifold-based features in a brief report.