STATS 769 Advanced Data Science Practice 2020
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
STATS 769
TERM TEST - SEMESTER 2, 2020
STATISTICS
Advanced Data Science Practice
1. [10 marks]
This question relates to a data frame covid that contains three variables, strin- gency_index, new_cases_per_million, and log_cases.
This data set is similar to, but NOT exactly the same as one of the data sets used in the course lab work.
> dim(covid)
[1] 21366 3
> head(covid)
stringency_index new_cases_per_million log_cases
ABW .1 |
0 .00 |
18 .733 |
2 .930287 |
ABW .3 |
33 .33 |
18 .733 |
2 .930287 |
ABW .7 |
44 .44 |
74 .930 |
4 .316554 |
ABW .8 |
44 .44 |
46 .831 |
3 .846545 |
ABW .9 |
44 .44 |
18 .733 |
2 .930287 |
ABW .10 |
44 .44 |
84 .296 |
4 .434334 |
The plot below shows the relationship between stringency_index and log_cases.
The following code is attempting to use a “validation-set” approach to calculate a test error estimate for a linear regression model (to describe the relationship between stringency_index and log_cases).
> N <- nrow(bovgc)
> tr-gn <- s-mpla(saq(N), N* .8)
> yrr-gn <- bovgc8lof b-sas[tr-gnJ
> xrr-gn <- bovgc8strgnfanby gncax[tr-gnJ
> yrast <- bovgc8lof b-sas[-tr-gnJ
> xrast <- bovgc8strgnfanby gncax[-tr-gnJ
> egt <- lm(lof b-sas - strgnfanby gncax, bovgc)
> LofLoss <- eunbtgon(p, y) 【
+ -ma-n(y*lof(p) + (1 - y)*lof(1 - p), n- .rm=rRuE)
+ )
> prac <- pracgbt(egt, c-t- .er-ma(strgnfanby gncax=xrr-gn)) > LofLoss(prac, yrr-gn)
(a) There are several errors in the above code. Identify the errors and write
new code to fix the errors.
[10 marks]
2. [15 marks]
This question relates to a CSV file "covid .csv".
This file is similar to, but NOT exactly the same as the files used in the course lab work.
There are several thousand lines in the CSV file, but the first few lines are shown below.
(a) Explain in detail what the following shell code is doing and what the result-
ing output would look like.
You will not be able to give exact details of the output, but you should be able to describe its general structure.
awk -F, -e ✬NR > 1 { print($2) } ✬ covid .csv > ages
Rscript -e ✬table(read .table("ages")[[1]]) ✬
[10 marks]
(b) Write a SINGLE bash expression (that may contain multiple commands
piped together) that prints out the total number of 52-year-old males from Lima in the covid .csv file.
HINT: you could search the covid .csv file for the rows that contain Lima and then search within those rows for rows that contain male and then search within those rows for rows that contain 52 and then count the number of those rows.
The result of your code would look like this:
65
[5 marks]
3. [10 marks] This question relates to the the JSON file, "covid .json", shown below.
This is similar to, but NOT exactly the same as the JSON files used in the course lab work.
(a) Write down the result of the following R code.
> lgar-ry(isonlgta)
> eromJsoN(ra-cLgnas(mbovgc .isonm))
[5 marks]
(b) A MongoDB database called covid contains a large number of documents with
the same structure as the file "covid .json".
Write R code to query the covid database and extract ONLY the continent and location for records with population greater than 100,000,000 and location not equal to "World".
The output of your code would look like this:
continent
1 Asia
2 South America
3 Asia
4 Africa
5 Africa
6 Asia
7 Asia
8 Asia
9 North America
10 Africa
11 Asia
12 Asia
13 Europe
14 North America
location
Bangladesh
Brazil
China
Egypt
Ethiopia
Indonesia
India
Japan
Mexico
Nigeria
Pakistan
Philippines
Russia
United States
[5 marks]
4. [5 marks] This question relates to the the XML file, "covid .xml", shown below.
This is similar to, but NOT exactly the same as the XML file used in the course lab work.
(a) Write R code that reads the XML file into R and uses an XPath
expression to select the cases when the day is greater than 17 and produces a numeric vector.
The output of your code would look like this:
[1] 0 3
[5 marks]
2022-09-05