Problem 1.
Derive the gradient in softmax regression ,
∂𝐽
∂𝑤𝑘,𝑖
∂𝐽
∂𝑏
𝑘
Problem 2.
Read the section “Logistic regression vs. softmax” in the lecture note. It
shows that a two-way softmax can be reduced to logistic regression.
Please show that logistic regression can also be reduced to 2-way
softmax, i.e., for any parameter of the logistic regression model, there
exists some parameter of the softmax regression model that does the
same thing.
Problem 3.
Consider a 𝑘-way classification. The predicted probability of a sample
is y∈ ℝ where is the predicted probability of the th category.
𝐾
, 𝑦
𝑘
𝑘
Suppose correctly predicting a sample of category 𝑘 leads to a utility
of 𝑢 . Incorrect predictions do not have utilities or losses. 𝑘
Give the decision rule, i.e., a mapping from y to 𝑡, that maximizes the ^
total expected utility.
END OF W7
First (soft) deadline: Nov 9
Second (hard) deadline: Nov 16 before exam
Reference solutions will be released on Nov 11 for students to better
prepare for the mid-term.
Students may refer to the provided solutions, but must submit their
own written/typed solutions before Nov 16 to get marks. Copy and
paste the provided solutions will be considered as plagiarism.
