1) Story Generation with Transformers (1%)

1. Use the ROC story data provided in the previous assignment (https://cs.rochester.edu/nlp/rocstories/) to train and test a story generation system. Starter code is available, prm.py, for a sub-word unit

transformer-based language model. You will need to create a SentencePiece model for tokenization, see sp.py for suggestions.

2. Do a hyper-parameter search.

3. How many parameters does your best model have? How does this compare to the best RNN model?

2) Translator (2%)

1. Adapt the model in (1) into a translation system. Using data from https://tatoeba.org/en/downloads   (Custom export - Sentence pairs), create a system that translates Spanish, and Chinese (Mandarin) into English. You will NOT need to modify the NN code in prm_NN.py.

2. How does this model compare with your RNN model?

3. Modify the code and training to allow any of the 3 languages to be translated into any of the 3   languages. Would adding a Spanish-Chinese dataset help? How many parameters does this model have?

3) BERT Sentiment Analysis (3%)

1. Choose a pre-trained language model from Hugging Face from the BERT family, e.g. BERT, RoBERTa, ALBERT, etc.

2. Fine tune the model using the IMDB movie review dataset, a text classification dataset.    a. https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews b. Divide the dataset into training, validation, and test sets.

3. Evaluate the fine-tuned model on the validation and test sets.

4. Experiment with different hyperparameters, such as learning rate and batch size, and observe the effect on the model performance.