Lab Assignment 1: Feedforward Neural Networks

Neural Networks and Deep Learning
Lab Assignment 1: Feedforward Neural Networks
Summary: In this assignment, you will demonstrate you understand how to train, evaluate,
and analyze feedforward neural networks. Your submission should include one PDF file that
merges two separate PDF documents: one showing your code and another showing your
methods, results, and analysis.
1. Neural Network Hyperparameters [15 points]:
(a) Design and conduct your experiment (Code)
• Load a real dataset not covered in class that is designed for the classification
problem; e.g., from sklearn.datasets, Kaggle, your own data, etc.
• Create an 80/20 train/test split of the dataset.
• Train at least eight neural network models using all possible combinations
of different numbers of hidden layers (at least 2 different values), numbers
of neurons per layer (at least 2 different values), and two types of activation
functions. Set all other hyperparameters constant when training; e.g., number
of iterations for training, batch size, and gradient descent approach.
• Evaluate each model on the test set using accuracy and a confusion matrix.
(b) Report your methods, results, and analysis (Write-up)
• Describe the methods you used for your experiment. This should include
a discussion of the dataset (e.g., source? number of examples?) and what
parameters were used to train all the models.
• Report your results for every tested model.
• Discuss your analysis of what general trends emerge from your results. For
example, did a certain number of hidden layers, number of neurons per layer,
or activation functions lead to consistently better results. If so, why do you
think this occurs? You also could analyze examine what, if any, insights are
gained by looking at both the different evaluation approaches (i.e., accuracy
and confusion matrix). Your discussion should consist of 2-4 paragraphs.
2. Impact of Training Duration and Training Data [10 points]:
(a) Design and conduct your experiment (Code)
• Load a real dataset not covered in class that is designed for the classification
problem; e.g., from sklearn.datasets, Kaggle, your own data, etc.
• Create an 80/20 train/test split of the dataset.
1
• Train neural networks using four approaches: train with 25%, 50%, 75%, and
100% of the training data respectively. For this experiment, select one set
of hyperparameters to use for the neural networks and keep those constant
when training; e.g., number of hidden layers, number of neurons per layer,
activation function, batch size, and gradient descent approach.
• Create one plot with four curves that shows the performance of each of the
four approaches with respect to the number of epochs used during training.
For performance evaluation, use the accuracy metric.
(b) Report your methods, results, and analysis (Write-up)
• Describe the methods you used for your experiment. This should include a
discussion of the dataset and the parameters used to train all the models.
• Show the plot that visualizes the performance for each of the four approaches.
• Discuss your analysis of what general trends emerge from your results. For
example, what is the influence of the amount of training data and the training
duration? Your discussion should consist of 2-4 paragraphs.
How to Submit Lab Assignment 1: Please submit a pdf named with your first and last
name; i.e., firstname lastname.pdf. A successful submission will consist of two contributions.
First, it should include the source code of your implementation as the first part of the
PDF file (i.e., portions indicated by “Code”).1
. Second, it should include a report with all
results and analysis (i.e., portions indicated by “Write-up”) as the second part of the
PDF file. All material that you submit must be your own.
1We require submitting the code as a PDF to avoid many issues that we have observed in the past with
being able to access submitted code. These issues have arisen, in part, because we make no programming
language requirements. Issues also have arisen from students not providing read permissions for links to
their files; e.g., on Google Colab
2