Machine Learning II Homework #4

Machine Learning II
Homework #4

Homework 4:
• Show ALL Work, Neatly and in Order.
• No credit for Answers Without Work.
• Submit a single pdf file includes all of your solutions.
• DO NOT submit individual files or images.
• For coding questions, submit ONE .py file and include your comments.
Note 1: Use python to check your results that you draw.
E.1:
Sketch the response of the following network when the weights and biases are W1 =

−1 1T
,b
1 =

0.5 1T
,W2 =

1 1
,b
2 = [−1] for −2 < p < 2.
a
2= purelin(W
2
a
1+b
2
)
Linear Layer
a
1= poslin(W
1
p+b
1
)
Pos. Linear Layer
p
a
1
n 2
1
2
Inputs
w
1
1,1
a
1
n 1
1
1
w
2
1,1
b
1
2
b
1
1
b
2
1
a
2
n 1
2
1
1
1
1
w
1
2,1 w
2
1,1
E.2:
Consider the following neuron.
Sketch the neuron response (plot a versus p for −2 < p < 2) for the following cases.
i. w = 1b = 1 f = hardlims
ii. w = –1b = 1 f = hardlim
iii.w = 2b = 3 f = purelin
1
S
Input
n a
b
1
p w
f
General Neuron
a f b = ( + ) wp
1= 1= f hardlims
1–= 1= f hardlims
2= 3= f purelin
2= 3= f satlins
2–= 1–= f poslin
iv.w = 2b = 3 f = satlins
v. w = –2b = –1 f = poslin
E.3:
Consider the following neuron.
, , , , , ,
w
1
1,1
w
2
1,2
w
2
1,1
w
1
2,1
S
S
S
n 1
1
n 1
2
p
n 2
1
b 1
1
b 1
2
b 2
1
a 1
1
a 1
2
a 2
1
1
1
1
Input Sat. Linear Layer Linear Layer
a satlin W b 1 1 1 = ( + ) p a purelin W a b 2 2 1 2 = ( + )
w1 1
1
2= w2 1
1
1= b1
1
2= b2
1
1–= w1 1
2
1= w1 2
2
1–= b1
2
0=
Sketch the neuron response (plot a versus p for
  3 −3 < p < 3) for the following cases (Use Satlins
as Transfer function).
i. n
1
1
ii. a
1
1
iii.n
1
2
iv.a
1
2
v. n
2
1
vi. a
2
1
2
E.4:
We have four categories of vectors:
Category I : 
−2
0

,

−2
2
!
, Category II: 
−1
3

,

1
3
!
Category III : 
2
0

,

2
2
!
, Category IV: 
−1
−1

,

1
−1
!
i. Design a two-neuron perceptron network (single layer) to recognize the four categories of
vectors. Select the ”BEST” decision boundaries, and explain what ”BEST” means. Sketch the
decision boundaries, and find the weights and biass. SHOW ALL WORK. Use the following
targets:
Category I :
1
0

, Category II: 
1
1

, Category III: 
0
1

, Category IV: 
0
0

ii. Draw the network diagram. Use the exact abbreviated notation that is used in class and
textbook.
iii. Suppose the following vector is added to category IV. Perform one iteration of the perceptron
rule with this vector. (Start with the weights you determined in part i.) 
−3
1

E.5:
Consider the following perceptron network.
7LWOH
([S
S D

 Q

:
 E
5[ 
6 [5
6 [ 
6 [ 
6 [ 
,QSXWV




6\P+DUG/LPLW/D\HU
D KDUGOLPV :SE 
5 6
b
2–
0
=
W 1 1
1– 1
=
i. How many different classes can this network classify?
ii. Draw a diagram illustrating the regions corresponding to each class. Label each region with
the corresponding network output.
3
iii. Calculate the network output for the following input. p =

1
−1

iv. Plot the input from part iii in your diagram from part ii, and verify that it falls in the correctly
labeled region.
Note 1: Write a Python Script to solve E.6. This problem is to implement the Perceprton
learning rule. Please read the summary page in chapter 4 and implement the learning rule.
E.6: Python Exercise
The vectors in the ordered set defined below were obtained by measuring the weight and ear
lengths of toy rabbits and bears in the Fuzzy Wuzzy Animal Factory. The target values indicate
whether the respective input vector was taken from a rabbit (0) or a bear (1). The first element
of the input vector is the weight of the toy, and the second element is the ear length. Note: You
are not allowed to use any NN packages like Sklearn and etc, you need mostly numpy package.
p1 =

1
4

,t1 = 0; p2 =

1
5

,t2 = 0; p3 =

2
4

,t3 = 0; p4 =

2
5

,t4 = 0;
p5 =

3
1

,t5 = 1; p6 =

3
2

,t6 = 1; p7 =

4
1

,t7 = 1; p8 =

4
2

,t8 = 1;
i. i. Use Python to initialize and train a network to solve this ”practical” problem.
ii. Use Python test the resulting weight and bias values against the input vectors.
iii. Please plot the inputs and check your trained weight vector and validate your results by
plotting the trained weight and bias.
4