CSCI 362: Machine Learning Assignment 5
SGD with Momentum
Implement mini-batch gradient descent with momentum:
• Download or cut-and-paste ols.py from Project 2 in the DL@DU Github repo to your computer.
Please make sure to also get the data for Project 2 from here (be sure to click Raw).
• The program ols.py uses mini-batch gradient descent to train a linear model to best fit (in the leastsquares sense) the housing data.
As students noted in class, the mini-batches are chosen with replacement, which might not be what
you want. Feel free to modify the mini-batching scheme in ols.py. You may wish to have a look here:
excursion into sampling theory.
• Next, implement momentum in your ols.py. You may refer to Simmons’ solution code for Project 1
(which is called proj1_ols.py and is posted on the homepage of our Canvas site for this course).
• Recall that, for linear models, the optimal learning rate and momentum can be computed. In the
screenshot below, the optimal learning parameters were used to train a linear model on the housing
data. The well-trained linear model predicts about 86 percent of the variation in the data (shown in
the graph as validation).
• Please make sure that your final code for this assignment successfully trains a linear model with
performance similar to that pictured below. Then upload a screenshot of the output of your code
training with those optimal parameters.
