Problem Set 3
1. Regularization [2 points]: Define what is regularization in deep learning
and describe the motivation for using regularization.
2. CNN Features [3 points]: Describe what are low-level, mid-level, and
high-level features in CNNs.
3. Fine-Tuning [2 points]: Identify two advantages of fine-tuning a neural
network over training a network from scratch.
4. Computer Vision Problems [6 points]: Define the problems of image
classification, object detection, and semantic segmentation and describe how
the output layers of computer vision models for these three problems differ.
5. Recurrent Neural Networks; i.e., RNNs [12 points]:
(a) Describe two advantages of using recurrent layers instead of fully connected layers in a neural network.
(b) Describe how recurrent layers differ from convolutional layers and when
one should choose the former versus latter in a neural network.
(c) Describe how to design recurrent neural networks to solve each of the
following problems: one-to-many, many-to-one, and many-to-many sequence problems. Your response should indicate what should be the
input and output of each network.
(d) Describe the motivation for gated RNNs and identify two types of gated
RNNs.
(e) Assume you design a 2-layer RNN to predict a character sequence when
given an input character sequence. What will happen to the number of
model parameters when the number of input characters doubles?
1
