Homework 3 CS 436/580L

Homework 3
CS 436/580L: Introduction to Machine Learning

Instructions
1. You can use either C/C++, Java or Python to implement your algorithms.
2. Your implementations should compile on remote.cs.binghamton.edu.
3. Make sure remote.cs.binghamton.edu has the packages that you require before
starting to implement.
4. This homework requires you to implement Perceptrons. Using existing packages for the algorithm is not allowed.
5. Please make sure your code is readable and well-commented.
6. Your homework should contain the following components:
(a) README.txt file with detailed instructions on how to compile and run
the code.
(b) Code source files
(c) Type-written document containing the results on the datasets.
7. Adhere to file naming conventions on myCourses.
1 Perceptrons, and Neural Networks
In this question, you will implement the Perceptron algorithm and compare it with
your own implementations of Naive Bayes. If you are unsure of whether your implementation is correct, then you may compare them with WEKA/Scikit-Learn implementations of Naive Bayes. As in homework 2, the classification task is spam/ham
(use the same dataset made available as part of homework 2).
1
80 points Implement the perceptron algorithm (use the perceptron training rule and not
the gradient descent rule). Your task here is to experiment with different
values of number of iterations and the learning rate. Report the accuracy for
20 suitable combinations of number of iterations and the learning rate. Repeat
your experiment by filtering out the stop words. Compare the accuracy of
your perceptron implementation with that of Naive Bayes (implemented in
Homework 2).
20 points Consider the data set given below. Assume that the co-ordinates of the points
are (1,1), (1,-1), (-1,1) and (-1,-1).
Construct a neural network that will have zero training error on this dataset.
Write down and explain the solution (no programming is necessary for this
part).
(Hint: Think XOR. You will need exactly one hidden layer and two hidden
nodes.).
2
What to Turn in
• Your code
• README file for compiling and executing your code.
• A detailed write up that contains:
1. The accuracy on the test set different values of the number of iterations
and the learning rate.
2. Compare the accuracy across the different models and report your observations.
3. Written answer to XOR question.
3