Assignment: MTHM503J
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
Assignment: MTHM503J
October 2022
Introduction
This assignment consists of four sections. Section A requires you to work in Python. In all other sections, you are free to use either R or Python.
All answers, with the exception of the oral video presentation in section D, should be submitted in a rendered version of a Jupyter / R Markdown notebook. The rendered files can be either in HTML or PDF format.
Your Jupyter / R Markdown notebooks should be clearly organised and annotated numerically according to the corresponding questions.
In these exercises, I want you to focus on writing clear, understandable code (with comments where necessary), and the code you submit will be evaluated not only for its accuracy but also for its clarity.
This is an individual coursework and you must not collaborate with oth- ers when working on this assessment. Please read the College guidelines on plagiarism: http://intranet.exeter.ac.uk/emps/studentinfo/subjects/ mathematics/assessment/academicmisconduct/.
Important: in your submissions please mark question numbers next to your answers.
A. Traffic accidents in Python [60 marks]
This part of the coursework involves python code, written summaries, and fig-
ures. This section must be answered using Python.
This problem will examine traffic accident data from the UK Department
of Transport Road Safety database. To answer the last part of this question you
will probably want to consult: https://data.gov.uk/dataset/cb7ae6f0-4be6-4935-9277-47e5ce24a11f/ road-safety-data for more information.
1. [10 marks] Starter data questions.
(i.) Read the accidents2019 .csv file into a pandas data frame named accidents
and display the first 2 lines of the data frame.
(ii.) How many accidents happened in 2019?
(iii.) What are the column names in the data frame?
(iv.) Report the date and coordinates (longitude and latitude) of the accident in the 100th row of the data frame.
(v.) Which month had the highest number of accidents and how many were there in this month?
2. [5 marks] Casualties.
(i.) Calculate the total number of casualties.
(ii.) What is the difference in the number of casualties between Lower Layer Super Output area (LSOA) E01032739 (City of London 001F) and E01033708 (Hackney 027G)?
(iii.) Which LSOA saw the highest total number of casualties in 2019?
3. [5 marks] Police officer attendance.
(i.) Did_Police_Officer_Attend_Scene_of_Accident: What are the pos-
sible values that occur in this column, and how often does each value occur?
(ii.) Did_Police_Officer_Attend_Scene_of_Accident = 1 indicates that a
police officer attended the accident, and the other values indicate an officer was not in attendance. What fraction of accidents was attended by a police officer?
(iii.) Based on these data, if you have an accident on a weekday (i.e. Monday, Tuesday, Wednesday, Thursday, Friday) are you more or less likely to have the police attend than when an accident happens at the weekend?
4. [10 marks] Visualisation.
(i.) Visualise the locations of all accidents by a scatter plot of Latitude vs Longitude. Annotate the axes and add a plot title.
(ii.) Create a similar scatter plot as in the previous question 5(i) but zoom in on a 2 by 2 degree area that includes Exeter/Devon and choose a different color for accidents that happened in a rural area and in an urban area. Include an orange marker that indicates the coordinates of Exeter.
5. [30 marks] What affects the number of casualities in an acci- dent?
Use the data to examine the factors influencing the number of causalities that occur in an accident. Your answer should consist of a (maximum) of two graphs with a (maximum) of two short descriptions. You may use regression models in your answer, but this is not obligatory.
B. What factors influence an individual’s weight? [60 marks]
The data within weight_height .csv contains measurements of individuals’ heights, weights, their age and their sex.
This question is straightforward: build a regression model to explain the patterns in individuals’ weights in these data. You should aim to present a parsimonious model: one that is just complex enough to explain the data.
Your answer should demonstrate how you came to choose this model, which should likely involve graphs (three maximum) and text (three short paragraphs maximum).
C. Power demand clustering [80 marks]
Aim
The aim of this part of the assignment is to perform clustering on power data recordings at substations in order to see whether there are groups that have similar demand profiles and to see whether there are differences between years.
Data
There are two types of data: (i) variable - the measurements from the monitors; and (ii) fixed - characteristics of the substations, that include information that may be useful when trying to understand and name your clusters.
There are three datasets containing the variable data, each corresponding to a different year.
• January_2013 .RData
• January_2014 .RData
• January_2015 .RData
You can load each dataset using R’s load function.
These datasets contain data for the same 535 substations. Each row of a dataset represents a day’s recording at a given substation, on a particular date. The first two columns indicate the date and Substation ID. The remaining 144 columns indicate the raw power data recorded by monitors at 10 minute intervals throughout the day.
The fixed data is in the Characteristics .csv file. This contains the fol- lowing information:
• SUBSTATION_NUMBER - so you can link with the variable data
• TRANSFORMER_TYPE - ground or pole mounted (indicating urban or rural substations)
• TOTAL_CUSTOMERS - the number of customers receiving their electricity from this substation
• Transformer_RATING - indicates the total power being delivered by the substation
• Percentage_IC - the percentage of industrial and commerical (not domes- tic) customers
• LV_FEEDER_COUNT - the number of feeders coming from the substation
• GRID_REFERENCE - the Ordnance Survey grid reference for the location. (See https://getoutside.ordnancesurvey.co.uk/guides/beginners-guide-to-grid-references/ for information, if you wish to use this information but it is not obligatory
to do so.)
1. [10 marks] Initial data analysis.
(i.) Create a summary table for the Characteristics .csv dataset and vi-sualise the distributions of the percentage of industrial and commercial customers, transformer ratings and transformer types.
(ii.) Using this and other analyses you think appropriate, describe the relation-ships between the different substation characteristics (transformer type, number of customers, rating, percentage of I&C customers and number of feeders).
2. [20 marks] Hierarchical clustering.
In this question, we are going to perform hierarchical clustering on the January_2013 dataset.
(i.) Each row of the data represents a day of observation. Divide the power recorded in each 10 minute interval by its corresponding daily maximum (i.e. the maximum value in the row). Print the first two rows of this processed data.
(ii.) Each substation has a number of days when power data were collected. In this part, we are going to calculate the average daily power demand profiles, which represent the average power produced by the substation in each 10 minute interval, across the many days of collection. In other words, for each substation, obtain a single row of length 144 where each element of it represents the average power produced within that 10 minute interval. Note, for this question, you are to use the normalised data obtained from the previous part. Print the first two rows of this processed data.
(iii.) Using your preferred choice of a distance function (i.e. Euclidean distance, Manhattan distance etc.), create a distance matrix for these data (i.e. the averaged data created in the previous part) and produce a dendrogram by performing hierarchical clustering.
(iv.) Using your dendrogram visualisation as a guide, choose an appropriate number of clusters and label each substation according to its cluster mem- bership. How many substations are in each of your clusters?
(v.) For each of your clusters, visualise the daily average demand profiles. Note a demand profile is the power produced in each of the 10 minute intervals across the day.
(vi.) For each of your clusters, do the demand profiles vary by weekday (i.e. Monday-Friday) versus weekend?
(vii.) Compare the characteristics data for each of your clusters. Using this information, choose names for your clusters and give a potential explana- tion for the power usage patterns.
3. [20 marks] Allocating new substations to clusters. The Dataset new_substations .RData contains information for five new substations.
(i.) For each substation, on the same plot, plot the daily average demand for weekdays and weekends.
(ii.) Assign each new substation to the cluster from the January 2013 data whose centroid is nearest the new substation’s daily average demand pro- file.
(iii.) Based on your summaries and plots, is the cluster allocation of the new substations as you expected?
4. [30 marks] Exploring differences between years.
The power company want to know whether there are any differences between power demands between years. They are particularly interested in whether the groupings/clusters of substations change between years. Perform suitable analyses of the power demands by year and explore whether the membership of clusters changes between years. You should write a report to the power distribution company using a rendered Jupyter / R Markdown notebook to detail your analyses, results and present a conclusion. This should, at maximum, have four graphs and four short paragraphs.
D. Presentation [50 marks]
The presentation is based on PartC/Q4 only. You should submit a recording of a narrated presentation that should be 5 minutes long. You are free to choose whatever presentation style you wish for this, and creativity in presentations is always good to see :).
In this you should explain what the problem is, how you approached it, and what your findings are.
2023-01-08