Overview
In this mini project, you will need to apply the feedforward neural network (FNN) on the MNIST dataset for hand written digit recognition.
Dataset
The MNIST database of handwritten digits has a training set of 60,000 digit images, and a test set of 10,000 images. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.
Method
For every image, we will use its pixel-wise intensities as features. To define a FNN model, you will need to specify the following hyper-parameters:
i) number of hidden layers and number of neural units for each layer;
ii) learning rate
iii) activation function (sigm, tanh_opt) and
iv) output function (‘sigm’,’linear’, ‘softmax’);
Question 1. Please further split the 60,000 training images (and labels) into two subsets: 50,000 images, and 10,000 images. Use these two subsets for training models and validation purposes. In particular, you will train your FNN model using the 50,000 images and labels, and apply the trained model over the rest 10,000 images for evaluation purposes.
Please specify at least three sets of hyper-parameters (see the above). For each set, call the third-party functions or tensorflow to train a FNN model on the training samples (50,000 images in this case), and apply the learned model over the validation set (10,000 images in this case). For each model and its results, please compute its confusion matrix, average accuracy, per-class Precision and Recall. Report the model that achieves the top accuracy.
Question 2. Apply the top ranked model over the testing samples (10,000 images). Call the above function to compute the confusion matrix, average accuracy, per-class precision/recall rate. In addition, select and visualize TEN testing images for which your mode made wrong predications. Try to analyze the reasons of these failure cases.
