UCLA CS 145 Homework #1
1. Linear Regression
In LinearRegression\linearRegression.py, fill in the missing lines in the python code to estimate 𝛽, using (1) closed-form solution, (2) batch gradient descent, and (3) stochastic gradient descent. a. Report the learned weights and MSE (Mean Square Error) in the test dataset for each version, are they the same and why? b. Apply z-score normalization for each feature x and report whether the normalization affect 𝛽 and MSE (Mean Square Error) in the test dataset, for all three versions of the algorithm, and why?
2. Logistic Regression and Model Selection
In LogisticRegression\LogisticRegression.py, fill in the missing lines in the python code to estimate 𝛽, using (1) batch gradient descent and (2) Newton Raphson method. a. Report the learned weights and accuracy (Mean Square Error) in the test dataset for each version, are they the same and why? Discuss the pros and cons of the two methods. b. Similar to linear regression, regularization can be added to logistic regression. Consider the new objective function as: 𝐽(𝛽)=−∑(𝑦𝑖𝒙𝑖 𝑇𝛽𝑖 −log(1+exp{𝒙𝑖 𝑇𝛽}))/n+𝜆∑𝛽𝑗 2 𝑝 𝑗=0 where n is the total number of data points in the training dataset and p is the dimensionality of attributes. Please compute its first derivative 𝜕𝐽(𝛽) 𝜕𝛽𝑗 and implement a regularized batch gradient descent algorithm accordingly. Discuss how the regularization is affecting the training loss (in terms of average log likelihood) and test accuracy based on the experimental results.
