Assignment 5: Neural Networks

Assignment 5:

1 Question 1 Neural Networks (40 pts)
: Answer the following questions. Use the neural network diagram (Figure 1) for reference wherever applicable. Work out the math for the questions, if you want partial credit.
• 1 a. Consider a neural network that predicts something (out), given a particular input (x).
Consider the figure 1 depicting the neural network. What is the dimension of the input and
the output? (2 pts)
• 1 b. What is the dimension of the hidden layer? (2 pts)
• 1 c. Let us say, for each dimension ‘j’ of input x and each dimension ‘k’ of the hidden layer
h, we get hk =
P
j
W1j,k ∗ xj
(no other additional terms like bias). What is the value of
each dimension of the hidden layer in terms of W1 and ‘x’? (6 pts)
• P
1 d. Let us say, the output is calculated from the hidden layer, in the same way out =
k
W2k ∗ hk. What is the output ‘out’ in terms of the weights (W1 and W2) and the input
‘x’? (Do not use hidden layer terms in this expression) (8 pts)
• 1 e. Let us say you have the ground-truth label for this input ‘x’ as ‘y’. You define a loss
function ‘y’ - ‘out’. In order to estimate how much you should increase or decrease the
weights W1 and W2, you calculate the gradient. i.e, ∂loss
∂W21
. What will be the gradient for
W21? The derivative should be in terms of W1i,j , W2i
, and x. (8 pts)
• 1 f. If you use sigmoid activation on the hidden layers, i.e. hk = Sigmoid(
P
j
W1j,k ∗ xj ).
What will be the ‘out’ value, in terms of W1i,j , W2i
, and x? (10 pts)
• 1 g. Why do we need activation functions in neural networks? (4 pts)
Figure 1:
2 Question 2 Clustering (30 pts)
: In this question, consider an array of numbers [11, 25, 34, 17, 18, 65, 23, 41, 73, 19, 7, 82]. We are
going to group these numbers into 2 clusters using k-means. Let us consider the initial centroids
of the 2 clusters as 15 and 55.
• 2 a. What will be the cluster assignments (cluster-1 or cluster-2) for each of the numbers
in the array for the above-mentioned centroids? Work out the math for this problem. (8
points)
• 2 b. What will be the new centroids of these clusters? (4 pts)
• 2 c. Based on the new centroids, work out the new cluster assignments for each of the
numbers in the array. (8 pts)
• 2 d. Mention 3 drawbacks of k-means clustering. (6 pts)
• How would you use k-means clustering in a semi-supervised algorithm? (4 pts)
3 Question 3 Minimax Tree (30 pts)
Consider the minimax game tree shown below. Decisions by MAX are represented as upwardpointing triangles; decisions by MIN are represented as downward-pointing triangles; small letters denote outcomes of the game
• 3 a. Consider Figure 2. What are the values of the two MAX nodes? (4 pts)
• 3 b. Consider Figure 2. Of the eight outcomes, which one(s) would be pruned by an alphabeta search? (7 pts)
Figure 2: The outcome to the MAX player is as follows [a:8, b:3, c:1, d:7, e:2, f:5, g:6, h:4]
2
• 3 c. Consider Figure 3. Write down the minimax value of every non-terminal node next to
that node. (4 pts)
• 3 d. Consider Figure 3. Cross out the branches that do not need to be examined by alphabeta search in order to find the minimax value of the top node, assuming that moves are
considered in the non-optimal order shown. (7 pts)
• 3 e. Return the final path for Figure 3 (4 pts)
• 3 f. Return the final path for Figure 2 (4 pts)
Figure 3:
3