CECS 551
Assignment 5
Total: 20 Points
General Instruction
Submit uncompressed file(s) in the Dropbox folder via BeachBoard (Not email).
1. (20 points) Find best hyper-parameters using GridSearchCV.
Find Assignment 5.ipynb.
The toy dataset and the network design are identical with the assignment 3.
Adam optimizer is used for the optimization, and you need to determine three
parameters, beta 1, beta 2, learning rate.
Adam optimizer is given in Figure 1. beta 1 corresponds to ρ1, beta 2 corresponds
to ρ2, learning rate corresponds to ϵ.
You are asked to complete the last part of Assignment 5.ipynb to perform a grid
search using GridSearchCV.
You need to have at least three values for each hyper-parameters.
Print out the ‘negative mean squared error’ and the corresponding hyper-parameters.
-0.046273 {’beta_1’: 0.9, ’beta_2’: 0.999, ’learning_rate’: 0.0001}
-0.003971 {’beta_1’: 0.9, ’beta_2’: 0.999, ’learning_rate’: 0.001}
-0.041319 {’beta_1’: 0.9, ’beta_2’: 0.999, ’learning_rate’: 0.01}
.
.
-0.003831 {’beta_1’: 0.8, ’beta_2’: 0.99, ’learning_rate’: 0.01}
.
.
Best: -0.003831 {’beta_1’: 0.8, ’beta_2’: 0.99, ’learning_rate’: 0.01}
Figure 1: Adam optimizer
