CSM148 Homework 2

CSM148 Homework 2

Instructions: All work must be completed individually.
Start each problem on a new page, and be sure to clearly label where each problem and subproblem
begins. All problems must be submitted in order (all of P1 before P2, etc.).
1 Bias, Variance and Regularization
(a) (5 points) The figures below show the decision boundary of three different classifiers. In which of the
figures below does the classifier have a larger bias and in which figure does the classifier has a larger
variance? Draw out an approximate graph where you demonstrate how the training and test error for
each classifier will change over time during the course of training the model, and explain how do you
expect each classifier to perform (accuracy) on the test set.
Note: You don’t need to calculate the errors, but you should show how the training and test errors
compare for different classifiers.
(b) (5 points) One strategy to reduce variance and improve generalization is regularization. In figure
below, the blue lines are the logistic regression without regularization and the black lines are logistic
regression with L1 or L2 regularization. In which figure L1 regularization is used and why?
(a) (b)
1
2 Classification Metrics
You have trained a Logistic Regression classifier which gives you the following predictions on a test set:
Correct Label Predicted Probability
1 0.97
1 0.93
1 0.65
1 0.52
1 0.37
0 0.89
0 0.71
0 0.56
0 0.54
0 0.16
(a) (5 points) Draw the Receiver Operating Characteristic (ROC) curve. Show clearly the points where
the curve changes direction by e.g. including ticks on the x- and y-axis in the corresponding locations.
(b) (2 points) Compute the AUC score.
(c) (2 points) Draw the confusion matrix when the decision threshold is 0.5.
(d) (2 points) Using the previous confusion matrix, compute Accuracy, Precision, Recall, and F1 score.
(e) (5 points) Can we improve any of the previous scores (without a negative effect on any of the other
scores) by changing the threshold? If yes, which threshold value would you choose and why? If not,
explain why not.
2
3 Logistic Regression
Suppose we fit a multiple logistic regression: log  P (Y =1)
1−P (Y =1)

= β0 + β1X1 + . . . + βpXp.
(a) (2 points) Suppose we have p = 2, and β0 = 3, β1 = 5, β2 = −8. When X1 = X2 = 0, what are the
odds and probability of the event that Y = 1?
(b) (2 points) Suppose we increase the X1 value by 2, how does it change the log odds and odds of the
event that Y = 1? What if instead, we decrease the X2 value by 2?
(c) (2 points) Explain how increasing or decreasing β0, β1 or β2 affect our predictions.
(d) (2 points) What is the formulation of the decision boundary? Which points are on the decision
boundary?
(e) (2 points) Suppose we fit another two logistic regression models: one with only X1 and the other one
with only X2, and we observe that the coefficients of X1 and X2 in the two models are different than
those specified in part (a). Explain what is the potential reason and why it could be problematic that
the coefficients are different than those specified in part (a).

4 Logistic Regression with Interaction Term
You are analyzing how the birth weight of a baby (normal weight=0, low weight=1) depends on the age of
the mother (number of years over 23, e.g. a 25-year-old will have value 2) and the frequency of physician
visits during the first trimester of pregnancy (0=not frequent, 1=frequent). You have also decided to include
an interaction term for age and frequency. Your logistic regression coefficients are as follows:
Feature Coefficient
Intercept -0.48
Age 0.06
Frequency -0.45
Age × Frequency -0.19
(a) (4 points) Discuss the meaning of each coefficient, and explain what does the coefficient of the
interaction term show.
(b) (3 points) Specify the logistic regression models when the mother visited the physician frequently and
when they didn’t. Explain how the mother’s age affects the odds in each scenario.
(c) (4 points) Compare how physician visits affect odds of low weight at ages 18, 23, 25, 28, 30, by
calculating the odds ratio of low birth weight for mothers with frequent physician visits over those
with non-frequent physician visits, in the following table (fill the “Odds Ratio” column in the table
below). Note: for age, you should use number of years over 23.
Age Odds Ratio 95% Confidence Interval
18 (0.705, 4.949)
23 (0.325, 1.201)
25 (0.262, 1.036)
28 (0.206, 0.916)
30 (0.050, 0.607)
(d) (4 points) Interpret the numbers in the “Odds Ratio” column, considering the listed confidence
intervals. Hint: what does an odds ratio of 1 mean (holding other predictors fixed)?.
(e) (5 points) compare the “difference in probability” of low birth weight for mothers at ages 18, 23, 25,
28, 30, in the table below. Interpret your results and compare your interpretation to part (d).
Age Difference in probability 95% Confidence Interval
18 (-0.788,0.393)
23 (-0.197,0.088)
25 (-0.232,0.046)
28 (-0.315,-0.016)
30 (-0.540,-0.092)
4