Project #3 NLP task and a neural network

Natural Language Processing

Programming Project #3
The purpose of the project is to install TensorFlow or PyTorch on your own machine and use it, within a Python program, to implement any project involving an NLP task and a neural network. For most tasks, I would prefer if you use some variation of a recurrent neural network, but I am open to other sorts of architectures if you think something else makes sense for your project. The task can be something that we covered (or will cover) in class, but it does not have to be. You should test your implementation on a dataset and report the results. (You are allowed to create a dataset, but it will be much easier if you find one.) Note that we are not covering either TensorFlow or PyTorch in class, so you will have to go through an on-line tutorial to figure out how to install and use the library.

Default project: Use one of the libraries to re-implement Project #1, for any one of the three datasets. If you choose dataset #2 or #3 (the ones for which I did not provide my test set), you will need to split the dataset into a smaller training set, probably a tuning set, and a test set. Use the training set along with the tuning set to experiment with architectures and hyperparameters, and only test the final version on the test set. If you choose dataset #1, keep the test set as is, and you'll probably want to split the training set into a training set and a tuning set. My expectation is that a deep learning technique (including any recurrent neural network variation) will likely work best for the second dataset, because the documents contain only single sentences. I am not sure if the results will be good for any of these datasets, because they are very small by deep learning standards. I have not tried this myself yet, and I would not be surprised either way.

Regardless if you choose the default project or something entirely different, I expect more from a two- or a three-person group than from an individual. From an individual, just getting a project to work with a single reasonable architecture is enough. From a group, I'd like to see more experimentation (with architectures, hyperparameters, etc.)

When you are complete, you should submit a SHORT writeup (one to three pages) explaining:
• Which library (TensorFlow or PyTorch) did you use?
• What task did you work on?
• What dataset did you use?
• How was the datasets split into training, tuning, and testing portions?
• What architectures and other variables did you experiment with? (Include types of networks, numbers and sizes of layers of layers, hyperparameters, etc.)
• Summarize the result of the things that were varied.
• What were the choices for the final network?
• What were the final results?