Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

COMP42415 Text Mining and Language Analytics

2023/24

COMP42415: Text Mining and Language Analytics coursework

Python implementation (70% of Total Marks )

You are asked to use the provided dataset in order to develop and evaluate machine learning models for predicting the rating of a food review (classification). Please create a single Jupyter notebook to solve all the required tasks. The required tasks are the following:

1.    Prepare the dataset by applying any pre-processing or cleaning steps that you consider as necessary. Then, split the dataset into a training set containing 70% of the samples and a test set containing 30% of the samples. Follow an appropriate strategy for the split. You must use these training/test sets for all the models in this coursework. (10%)

2.    Implement a Naïve Bayes model for predicting the rating of a food review. Train your model on the training set and test it on the test set. Use an appropriate text representation. (5%)

3.    Implement a k-Nearest Neighbours model for predicting the rating of a food review. Train your model on the training set and test it on the test set. Use an appropriate text representation. You must select the best k by examining the performance of the model for k ∈ {1, 3, 5, 7}, using an appropriate cross-validation approach. Create a plot for k vs. classification performance to justify your choice. (10%)

4.    Implement a Convolutional Neural Network (CNN) model for predicting the rating of a food review. The model must have at least two convolutional layers. Train your model on the training set and test it on the test set. Use an appropriate text representation. (13%)

5.    Implement a Recurrent Neural Network (RNN) or a Long Short-Term Memory (LSTM) model for predicting the rating of a food review. The model must have at least two RNN/LSTM layers. Train your model on the training set and test it on the test set. Use an appropriate text representation.  (12%)

6.    Compute the confusion matrix, accuracy, F1-score, precision and recall for each model. (10%)

7.    Store the four trained models in files and implement a function “predict_food_review(text, model)” that given a text string (“text”) and model filename (“model”), it will load the pre-trained model, and predict the food review rating of the input text. The function should be able to work without requiring to rerun all or part of your code. (10%)

Note: You are strongly advised to use the Pandas library for loading and manipulating the dataset. Your code must run successfully on the Jupyter server used for the coursework’s submission. You are strongly advised to do your code development directly on the Jupyter server or at least test your code on it before submission.

Written Report (30% of Total Marks, 1500 words max)

You are asked to provide a written report about the developed NLP solution. The report should be divided into the following five  sections:  (1)  Dataset,  (2)  Data  preparation,  (3)  Machine  learning  models,  (4) Experimental results, (5) Discussion.  The following are required for each section, respectively:

1.    Critical discussion about the dataset (suitability, problems, class balance, etc.). (6%) 

2.    Description and justification of the data preparation step(s) used. (6%)

3.    Description and commentary on the machine learning architectures used, including a description and justification of the text representation method(s) used. (7%)

4.    Detailed performance evaluation of the trained machine learning models in terms of the computed performance metrics. (5%)

5.    Critical discussion on the achieved results, including potential limitations and usage instructions/suggestions. (6%)

Attention: The report should be included at the end of the Jupyter notebook using markdown cells. The report should be self-contained, i.e. write the report as if it was not bundled with your code.

Examiners expectations

What the examiners expect from your software implementation:

•             Your program must be runnable – a program that partially works or does not run at all will receive no mark.

•             You are asked to use Python 3.x.

•             Your source code should be documented with comments, making it to be as easily followed as possible.

What the examiners expect from the report:

•             Your report needs to be professional, and the language should be scientific.

•             Your report should provide justification for the design decisions you made in your solution.

Word Limit policy

Tables and figures are excluded from the word limit. Examiners will stop reading once the word limit has been reached, and work beyond this point will not be assessed. Checks of word counts may be carried out on submitted work. Checks may take place manually and/or with the aid of the word count provided via electronic submission.