Homework #2 Implement linear regression with gradient descent

Homework #2
CS 539,
100 points total [6% of your final grade]

Delivery: Submit via Canvas
For this assignment, you will:
 (70 pts) Implement linear regression with gradient descent
 (30 pts) Make predictions by using your implementation
Part 1: Implement linear regression with gradient descent
In this problem, you will implement the linear regression algorithm in python3. We provide the following files:
a) linear_regression.py - You will implement several functions. As we discussed in class, implement the
functions by using vectorization. You may refer to matrix calculus here:
https://en.wikipedia.org/wiki/Matrix_calculus
Do not change the input and the output of the functions.
b) test.py - This file includes unit tests. Run this file by typing ‘nosetests -v test.py’ in the terminal as you
did in homework 1 in order to check whether all of the functions are properly implemented. No
modification is required.
Part 2: Make predictions by using your implementation
Given training and test sets, you will make predictions of test examples by using your linear regression
implementation (linear_regression.py). We provide the following file:
a) application.py – write your code in this file. Do not change X and y.
Please play with the parameters alpha and number of epochs to make sure your testing loss is smaller than 1e-2
(i.e., 0.01). Report your parameters, training loss and testing loss. In addition, based on your observations,
report a relationshp between alpha and number of epochs. Note that a single epoch means the single time you
see all examples in the training set.
What to turn in:
 Submit to Canvas your linear_regression.py, application.py and a pdf document for part 2.
 This is an individual assignment, but you may discuss general strategies and approaches with
other members of the class (refer to the syllabus for details of the homework collaboration
policy).