All Sec!ons Assignments Assignment 3
Assignment 3: Exploring the Twi"erverse
Due: Monday, December 7 by 4:00pm (Toronto !me)
November 24, 2020: Added a clarifica"on to the "Sor"ng your output" sec"on regarding the ordering of strings in Python.
Goals of this Assignment
Prac!ce working with structured files.
Prac!ce building and using dic!onaries.
Write uni"ests for a func!on.
Use top-down design to break a problem down into subtasks and implement helper func!ons to complete those tasks.
Note the last goal in the list -- in this assignment, you will be doing significantly more design of func!ons than in the previous ones. We will give you a smaller number of func!ons to write, and you will write helper func!ons to solve the subtasks of each of those func!ons.
Introduc!on: The Twi"erverse
This handout page explains the problem you will be solving, the data you will be working with, and the tasks you need to complete. Please read it carefully -- you will need to read all of the informa!on here to complete the assignment successfully.
Twi"er is a social networking website where users can post very short messages known as "tweets".
Each Twi"er user can choose to "follow" other users, which means that they see those users' tweets.
A Twi"er user sees the tweets of users they are "following", and their tweets are seen by their "followers" (the users who follow them).
All the "follow" connec!ons define a network among Twi"er users, and it's quite interes!ng to look for pa"erns in the connec!ons. Tools like Twiangulate let you explore ques!ons like "what connec!ons do two users have in common?". In this assignment, you'll write func!ons that let you ask
ques!ons (or "queries") about a Twi"er dataset.
While it is possible to get data directly from Twi"er, we have simplified things for you and provided the data already stored in a file.
Your Tasks
For this assignment, you are required to:
1. Write the 6 required func!ons described later in this handout.
2. Write helper func!ons as needed. All helper func!ons should have complete docstrings, but there are no other requirements for your helper func!ons.
3. Write unittest s for one of the required func!ons, all_followers .
This is your first experience designing a program of this size. You will have to do far more planning than in the previous two assignments. It is likely you will need to break most of the required func!ons down into subtasks and write helper func!ons for those subtasks.
The Twi"er Data File
A Twi"er data file contains a series of one or more user profiles, one a#er the other. Each user profile has the following elements, in this order:
A line containing a non-blank, non-empty username. You may assume that usernames are unique (i.e. a single username will not occur more than once in the file), and that usernames do not contain any whitespace. Usernames are case sensi!ve.
A line for the user's actual name. If they did not provide a name, this line will be blank.
A line for the user's loca!on, or a blank line if they did not provide one.
A line for the URL of a website, or a blank line if they did not provide one.
Zero or more lines for the user's bio, then a line with nothing but the keyword ENDBIO on it. This marks the end of the bio, and is not considered part of it. (You may assume that no bio has the string ENDBIO within it.) If the user did not provide a bio, the ENDBIO line will come immediately a#er the website line, with no blank line in between.
Zero or more lines each containing a single username of someone that this user is following, then a line with the keyword END on it. (You may assume that no one has END as their username.) A user cannot follow themselves. You may assume that every user that appears in this sec!on of the file has a user profile in the Twi"er data file.
A blank line is one that contains only whitespace characters. For this assignment, whitespace characters include spaces, tabs, and newlines.
No!ce that the keywords ( ENDBIO and END ) act as separators in this file. All of their le"ers are capitalized, and the keywords contain no punctua!on. You may assume that no keyword separators will appear as any part of the content of a user profile.
Examples
Here is a sample user profile that might occur among many in a file:
tomCruise
Tom Cruise
Los Angeles, CA
http://www.tomcruise.com
Official TomCruise.com crew tweets. We love you guys!
Visit us at Facebook!
ENDBIO
katieH
NicoleKidman
END
The file data.txt in the provided starter files is a small example of a complete Twi"er data file (and was made by hand). The file rdata.txt is a much larger example (and is made from real data extracted from Twi"er). These should help you confirm your understanding of the file format and will be useful in tes!ng your program.
Cycles in the data
Although a user cannot follow themselves, there can be "cycles" such as this: user A can be following B who is following A. This is the shortest possible cycle. Of course, cycles can be longer.
The Query File
Note that the word "query" just means "ques!on". In computer science, we use it to specify a request for informa!on.
For this assignment, a query will be provided in a file. Below we will review the high-level parts of the
query, look at an example, and then describe the query file format.
Overview
A query has three components: a search specifica!on, a filter specifica!on, and a sor!ng specifica!on.
The search specifica"on describes how to generate a list of Twi"er usernames, star!ng with an ini!al username (a list of length one) and then finding their followers or people they are following, then people that are those people's followers or who they are following, and so on. When processing the search specifica!on, don't try to do anything to avoid cycles. For instance, if the search specifica!on says to find the people who user A is following, and from there the people they are following, you could find yourself back at user A. Don't try to avoid that.
A#er processing the search specifica!on, we have a list of Twi"er usernames. Its length could be zero. For example, if the ini!al username is 'dianelynnhorton' and the search specifica!on contains a single 'followers' keyword, then the number of results of the search will be zero if 'dianelynnhorton' has no followers.
The filter specifica"on describes how to filter the list of usernames produced by the search specifica!on. The filtering can be based on whether or not they are following a par!cular user, whether or not a par!cular user is their follower, whether or not their name contains a par!cular string (case-insensi!ve), or whether or not their loca!on contains a par!cular string (case-insensi!ve).
A#er processing the filter specifica!on, we have a possibly reduced list of usernames.
Once the search results have been found and filtered, the sor"ng specifica"on describes how the results should be sorted. Prior to the sor!ng step, the order of usernames does not ma"er.
Example query
Here is an example query:
SEARCH
tomCruise
following
following
following
FILTER
following c
location-includes a
SORT
popularity
The search specifica!on in this par!cular query has four steps.
1. Start with a list containing the username to start the search from; i.e. ['tomCruise'] . Let's call that list L1.
2. The search keyword 'following' says to replace each username p in L1 with the usernames of the users who p is following. This yields a new list, L2.
3. For the next 'following' keyword, we start with L2 and repeat the same opera!on as in the previous step, yielding another list, L3.
4. For the final 'following' keyword, we start with L3 and repeat that opera!on one last !me, yielding list L4.
No!ce that each step yields a list of zero or more usernames that is the input to the next step. There should be no duplicates in the final results list. Duplicates should be removed a#er each step.
The Twi"er data file diagram_data.txt in the provided files contains the follower/following rela!onships as represented by this diagram.
For those rela!onships, the search specifica!on above would yield this list of usernames: ['i', 'j', 'h', 'k', 'tomCruise'] . Make sure that you can see how the four lists, ending with this final one, are generated. No!ce that the final list contains the users you can get to in three "steps" of the "following" rela!onship, star!ng from 'tomCruise' .
The final list generated by the search specifica!on becomes the input to the filter specifica!on. For our current example, the filter specifica!on says that the list should be filtered in this way: a user should be kept only if they are following user 'c' and has a loca!on that includes the string 'a' .
Based on the query above, the final filtered list would be ['tomCruise'] .
The sor!ng specifica!on says to order the users according to their popularity.
Now let's look in detail at the exact format and meaning of a query.
Overall format of a query
The format of the query data file is as described below, in this order:
A line containing the keyword SEARCH .
The search specifica!on.
A line containing the keyword FILTER .
The filter specifica!on.
A line containing the keyword SORT .
The sor!ng specifica!on.
No!ce that the keywords above all act as separators in this file and are in all capital le"ers. There are other keywords (such as following which can be part of a search specifica!on) that are not separators; they are not capitalized.
Format and meaning of the search specifica!on
The format of a search specifica!on is as shown below, in this order:
A line containing a single username
Zero or more lines, in any order, each containing one of:
the keyword followers
the keyword following
Each step in the search specifica!on involves crea!ng a new list of users by going through each user in the current list and finding their followers or who they are following. Importantly, at each step, we want to create the new results list by replacing each user from the current list with the users that they are following or who are their followers.
The possible search opera!ons for a given user are:
followers : this will get all usernames for users who are following the given user (i.e. that have the given username in the value list paired with the key 'following' in their data in the Twi"erverse dic!onary -- more on this later).
following : this will get all usernames for users who the given user is following (i.e. the value list paired with the key 'following' in the data associated with the given user in the Twi"erverse dic!onary).
Note that, if the list of usernames is ['A'] , and we perform a 'followers' or 'following' opera!on, 'A' will not be in the resul!ng list, because users cannot follow themselves. If 'A' is one of mul!ple usernames in the current list, then 'A' can appear in the results list, but only if one of the other usernames produces 'A' from the search opera!on.
The final list of results should not contain any duplicates. Duplicates should be removed a#er each opera!on is performed. The order of the final list of results from the search opera!on does not ma"er.
Format and meaning of the filter specifica!on
The format of a filter specifica!on is:
Zero or more lines, in any order, each containing one of the following:
the keyword name-includes , a space, and a string to match
the keyword location-includes , a space, and a string to match
the keyword bio-includes , a space, and a string to match
the keyword follower , a space, and a string which is a valid username
the keyword following , a space, and a string which is a valid username
Each of the lines has exactly one space within it (the space separa!ng the keyword from the next string), and each keyword will appear at most once per filter specifica!on.
The filters are "addi!ve" in that each filter step should be applied to the list that resulted from the previous filter step. That is, you start with the list from the search specifica!on, filter it based on the first filter step to get a new list L1, filter L1 in the second filter step to get L2, and so on.
The 'following' and 'follower' filters will add users in the new list if they are following a par!cular user, or if a par!cular user is their follower. These filters are defined as:
The 'following' filter adds only users who are following provided username.
The 'follower' filter adds only users who the provided username is following.
The filters that have 'includes' as part of the keyword will do a simple substring search on the string represen!ng the relevant data for the user in the Twi"erverse dic!onary. This means that if the given string occurs anywhere in a user's name (for a 'name-includes' filter) or a user's loca!on (for a 'location-includes' filter), then that user is to be kept in the list. The substring search should be case-insensi!ve. For example, if the filter specifies users whose loca!ons include the string "USA", then users with loca!on "USA", "Modesto, California, USA", "kansas, usa" or "Musala, Bulgaria" would be kept, and users with loca!on "United States of America" would be excluded. This is far from perfect, but don't try to improve on it.
The order of the final list of results from the filter opera!on does not ma"er.
Format and meaning of the sor!ng specifica!on
The output from your program must include every user whose username is s!ll in the list a#er the search and filter specifica!ons have been processed.
The sor!ng specifica!on describes how the final list of results should be ordered. It consists of one line containing one of these keywords: 'username' , 'name' , 'popularity' .
The users in the final results list must be ordered as indicated by the keyword of the sor!ng specifica!on. Output that is sorted by username or name should be in alphabe!cal order, so that strings star!ng with 'a' are at the beginning of the output. See the sec"on "Sor"ng your output" later in this page for informa"on on how to sort. Although usernames are unique, names are not. If any users have the same name, sort them by username. When sor!ng users by popularity, the most popular users should come first. A user's popularity is defined to be the number of followers they have. (Not the number of people they are following!) If any users are !ed for popularity, sort them by username.
We can use Python's built-in sort and/or string comparison for sor!ng by username. However, we'll have to find another way to sort by username name or popularity. There is more informa!on on sor!ng is later in the "Sor!ng your output" sec!on.
Data Structures
To increase the readability of our code, we will use dic!onaries to represent both the Twi"er data and the query in our program. In this sec!on, we'll look at the format of the data and query dic!onaries.
The Twi"erverse Data Dic!onary
The type of the Twi"erverse dic!onary is Dict[str, Dict[str, object]] . (We use the type object here to indicate that the values in the inner dic!onary can have different types.) More specifically, each key in the Twi"erverse dic!onary is the username of a Twi"er user, and the value associated with that key is a dic!onary containing addi!onal informa!on about the user. Each of these inner dic!onaries will contain the keys 'name' , 'location' , 'web' , 'bio' , and 'following' . The value associated with the 'following' key is a list of zero or more strings, and the values associated with the rest of the keys are strings (and may be the empty string).
For example, if the following user informa!on is included in our Twi"er data file...
tomCruise
Tom Cruise
Los Angeles, CA
http://www.tomcruise.com
Official TomCruise.com crew tweets. We love you guys!
Visit us at Facebook!
ENDBIO
katieH
NicoleKidman
END
... then this key/value pair will be in our Twi"erverse dic!onary.
'tomCruise': {'name': 'Tom Cruise',
'bio': 'Official TomCruise.com crew tweets. We love you guys!\nVisit us at Facebook
!',
'location': 'Los Angeles, CA',
'web': 'http://www.tomcruise.com',
'following': ['katieH', 'NicoleKidman']}
No!ce that the newlines at the end of each line are removed from the data stored in the Twi"erverse dic!onary, with the excep!on of the bio informa!on, which needs to keep its inner newlines so that we could reconstruct the bio format the user chose. In general, you should remove leading and trailing whitespace from each line.
The usernames in the following list should be in the same order as in the data file.
And another example: if the following user informa!on is included in our Twi"er data file (note the blank lines for name, loca!on, and web, and zero lines for bio and following)...
quietTweeter
ENDBIO
END
... then this key/value pair will be in our Twi"erverse dic!onary.
'quietTweeter': {'name': '',
'bio': '',
'location': '',
'web': '',
'following': []}
Note that dic!onaries are not ordered, so don't worry if your dic!onary output is in a different order, as long as the items are the same.
The Query Dic!onary
We will also use dic!onaries to represent queries in our program. The query dic!onary contains three items; the key 'search' , whose value is a search specifica!on dic!onary, the key 'filter' , whose value is a filter specifica!on dic!onary , and the key 'sorting' , whose value is a sor!ng specifica!on dic!onary.
The search specifica!on dic!onary contains the keys 'username' and 'operations' , whose values are the string represen!ng the username to start at, and a list of strings represen!ng the opera!ons to perform, respec!vely. Note that the list that is the value associated with the key 'operations' may be empty, if there is only one line between the SEARCH and FILTER keywords in the query file. If there is only one line between SEARCH and FILTER, then it represents the username to start at, and there are no opera!ons to be performed in the search step. The opera!ons must be performed in the order that they are in the list.
The filter specifica!on is a dic!onary with strings represen!ng filter keywords as keys, and the strings to match as values. Note that this dic!onary may be empty if there are no filters to apply, and that each filter keyword will appear at most once in the filter sec!on of a query file.
The sor!ng specifica!on is a single string value represen!ng the order to sort the results in.
For example, if our query file looks like this...
SEARCH
tomCruise
following
following
following
FILTER
following KatieH
location-includes USA
SORT
popularity
... then the query dic!onary will have these key/value pairs.
{ 'search': {'username': 'tomCruise', 'operations': ['following', 'following', 'following']},
'filter': {'following': 'KatieH', 'location-includes': 'USA'},
'sorting': {'sort-by': popularity'}}
Required Func!ons
In the starter code file twitterverse_functions.py , complete the following func!on defini!ons. In addi!on, you will need to add some helper func!ons to aid with the implementa!on of these required func!ons.
The main program
Once you have correctly implemented the func!ons in twitterverse_functions.py , execu!on of the main program ( twitterverse_program.py ) will:
1. Read a data file and produce the Twi"erverse data dic!onary.
2. Read a query file and produce the query dic!onary.
3. Compute the results of the search opera!ons.
4. Apply the filters to the search results.
5. Sort the final results.
You may try running this file on the provided data and query files, as well as your own files that you have created for tes!ng. Note that just running the twitterverse_program.py is NOT sufficient to test your code.
Required Tes!ng ( unittest )
Write and submit a set of uni"ests for the all_followers func!on in the file test_all_followers.py .
You should create Twi"erverse dic!onaries as inputs for your test cases inside the test_all_followers.py file. Do NOT call open open in your test file, or anywhere else in your code.
Files to Download
All of the files included in the download for the assignment are listed in this sec!on. These files must all be placed in the same folder.
Please download the Assignment 3 files and extract the zip archive. The following paragraphs explain the files you have been given.
Python Code
The starter code for this assignment is in the file twitterverse_functions.py . This is the primary file you will be edi!ng.
The main program file is twitterverse_program.py . It is complete and must not be changed.
The starter code for the required uni"ests is in the file test_all_followers.py . You will edit and submit this file as well.
We are providing a checker as usual: a3_checker.py . It is complete and must not be changed.
Sample Twi"er Data and Sample Queries
We have provided a number of text files containing Twi"er data, and queries on that data. All files in the provided zip file that contain data in their filename are Twi"er data files, and all files in the provided zip file that contain query in their filename are query files. Put these files in the same directory as your other A3 files.
The query files query1.txt , query2.txt , and query3.txt are queries on the data in data.txt , and the file query4.txt is a query on the data in rdata.txt .
a3_checker.py
We are providing a type-check module that can be used to test whether your func!ons in twitterverse_functions.py have the correct parameter and return types. To use the type checker, place a3_checker.py in the same folder (directory) as your twitterverse_functions.py and run it.
If the type-checks pass: the output will tell you that the typechecker passed (and what it means for the typechecker to pass!). If the typechecker passes, then the parameters and return types match the assignment specifica!on for each of the func!ons.
If the type-checks fail: Look carefully at the message provided. One or more of your parameter or return types does not match the assignment specifica!on. Fix your code and re-run the tests. Make sure the tests pass before submi&ng.
We will do a much more thorough job of tes!ng your code once you hand it in, so be sure that you have thoroughly tested it yourself. With the func!ons in this assignment, there are many more possible cases to test (and cases where your code could go wrong). If you want to get a great mark on the correctness of your func!ons, do a great job of tes!ng your func!ons under all possible condi!ons. Then we won't be able to find any errors that you haven't already fixed!
Addi!onal Requirements
Do not change any of the exis!ng code. Add to it as specified in the comments.
Do not call print , input , or open in the code that you write, except within the if __name__ ==
'__main__' block. No!ce that two of the required func!ons take an open file, not a string.
Do not use any break or continue statements. Any func!ons that do will receive a mark of zero.
We are imposing this restric!on (and we have not even taught you these statements) because they are very easy to "abuse," resul!ng in terrible code.
Do not modify or add to the import statements provided in the starter code.
Do not add any code outside a func!on defini!on.
Do not mutate func!on parameters unless specified.
Do not use Python language features for sor!ng that we have not covered in the course, like the
op!onal parameter key or the func!on sorted .
Error checking
You may assume that the files actually contain a valid Twi"er data file and a valid query file, respec!vely. You do not need to check either file to make sure that it is a valid Twi"er or query file.
You may be wondering how you can be sure that the content of the Twi"er data file is valid. For example, there are probably rules about legal Twi"er usernames and there are certainly rules about valid URLs. Don't confirm that the username, URL, or any other elements of the Twi"er data file are valid. Simply read whatever string is in the relevant posi!on in the file and use it as is.
Reading the Twi"er data file
Typically, the easiest way to read through a data file is with a for-loop, but because of the structure of a Twi"er data file, you will need to use while loops. We recommend making sure you are very comfortable with the exercises from Weeks 7 and 8 before beginning this assignment.
Watch out for how you handle \n (newline) characters. If you use readline to get a line of the file, it will have a \n on the end (except possibly on the last line in the file, although that line may also have a \n -- your code should work in both cases). You need to be aware of that before you try to compare it to keywords like END . On the other hand, there are !mes when you really do need a \n .
For instance, when you store the contents of a user's bio, it must include any newline characters that appear in the middle of the bio, so that the original format is maintained.
Sor!ng your output
Processing the sor!ng specifica!on requires you to sort the list of users according to username, name, or popularity. Most likely you thought of using list.sort to accomplish the task. However, by default list.sort uses simple < to compare pairs of list elements. We know that < on strings compares them alphabe!cally. So list.sort will simply sort the strings into alphabe!cal order. In some cases, that is not the ordering you want. Clarifica"on: Python does not actually sort strings in alphabe"cal order. The correct term here is lexicographical order, which is not a term we expect you to know. The inten"on is that you are just using the comparison operators to sort your strings.
For example, if we have a list of users ['user1', 'user2'], with user1's name being 'jacqueline' and user2's name being 'Jen', then if the list is sorted by name, the result should be ['user2', 'user1'], because 'Jen' < 'jacqueline' according to Python.
Fortunately, there are ways to get around this. In the starter code for twitterverse_functions.py , we have provided our own implementa!on of a sor!ng func!on that has a parameter that you can use to control the ordering. All you have to do is send tweet_sort a func!on that takes two parameters of the same type as the elements in your list (in this assignment, the two parameters will be strings
represen!ng usernames) and returns:
-1 if the first one should appear before the second one in a sorted list 1 if the first one should appear a#er the second one in a sorted list, and 0 if they are "!ed", so that it doesn't ma"er which one comes first.
We have also provided some func!ons that return -1, 1, or 0 for each possible sor!ng case.
The simple module sort_demo.py demonstrates how this can be done. It is a good example of passing a func!on to a func!on. We strongly recommend you spend some !me understanding this code before you work on the sor!ng specifica!on.
The file sort_demo.py also contains some examples of wri!ng docstring examples that span mul!ple lines.
Constants
Unlike in the other assignments this term, we are not providing you with constants to use, or requiring the use of constants. However, you are welcome and encouraged to create any constants you think you need to make your code clean and readable.
Marking
These are the aspects of your work that may be marked for A3:
Coding style (20%):
Make sure that you follow Python style guidelines that we have introduced and the Python coding conven!ons that we have been using throughout the semester. Although we don't provide an exhaus!ve list of style rules, the checker tests for style are complete, so if your code passes the checker, then it will earn full marks for coding style with one excep!on:
docstrings for any helper func!ons you add may be evaluated separately. For each occurrence of a PyTA error, one mark (out of 20) deduc!on will be applied. For example, if a C0301 (line-toolong) error occurs 3 !mes, then 3 marks will be deducted.
All func!ons you design and write on your own (helper func!ons), should have complete
docstrings including precondi!ons when you think they are necessary.
Correctness (70%):
Your func!ons should perform as specified. Correctness, as measured by our tests, will count for the largest single por!on of your marks. Once your assignment is submi"ed, we will run addi!onal tests not provided in the checker. Passing the checker does not mean that your code will earn full marks for correctness.
Tes"ng (10%):
We will run the uni"ests that you submit on a series of flawed (incorrect) implementa!ons we have wri"en. Your tes!ng mark will depend on how many of the flawed implementa!ons your uni"ests catch, whether they successfully pass a working (correct) implementa!on, and whether your test files contain redundant (unnecessary) tests.
How should you test whether your code works
First, run the checker and review ALL output — you may need to scroll. Remember that the checker
ONLY shows you style feedback, and that your func!ons take and return the correct types. Passing the checker does not tell you anything about the correctness of your code.
A3 Checker
Note: the checker program for Assignment 3 only checks your twitterverse_functions.py file. It is up to you to make sure your test_all_followers.py file runs. Note that your test_all_followers.py file will not be marked for style.
We are providing a checker module ( a3_checker.py ) that tests two things:
whether your code follows the Python style guidelines, and whether your func!ons are named correctly, have the correct number of parameters, and return the correct types.
To run the checker, open a3_checker.py and run it. Note: the checker file should be in the same directory as your twitterverse_functions.py , as provided in the starter code zip file. When you run your own checker, be sure to scroll up to the top and read all messages.
If the checker passes for both style and types:
Your code follows the style guidelines.
Your func!on names, number of parameters, and return types match the assignment specifica!on.
This does not mean that your code works correctly in all situa"ons. We will run a different set of tests on your code once you hand it in, so be sure to thoroughly test your code yourself before submi&ng.
If the checker fails, carefully read the message provided:
It may have failed because your code did not follow the style guidelines. Review the error descrip!on(s) and fix the code style. Please see the PyTA documenta!on for more informa!on about errors.
It may have failed because:
you are missing one or more func!on,
one or more of your func!ons is misnamed,
one or more of your func!ons has the incorrect number or type of parameters, or
one of more of your func!on return types does not match the assignment specifica!on, or
2020 Fall
Home
Announcements
Start of Term FAQ
Lectures
Piazza
Office Hours
Videos and Exercises
Assignments
Test Informa!on
Modules
So#ware Installa!on
MarkUs
Teaching Lab Accounts and Computer Labs
Bb Collaborate
Account
Dashboard
Courses
Groups
Calendar
Inbox
History
Course
Evals
Help
your .py file is misnamed or in the wrong place.
Read the error message to iden!fy the problema!c func!on, review the func!on specifica!on in the handout, and fix your code.
Make sure the checker passes before submi&ng.
Running the checker program on Markus
In addi!on to running the checker program on your own computer, run the checker on MarkUs as well. You will be able to run the checker program on MarkUs once every 12 hours (note: we may have to revert to every 24 hours if MarkUs has any issues handling every 12 hours). This can help to iden!fy issues such as uploading the incorrect file.
First, submit your work on MarkUs. Next, click on the "Automated Tes!ng" tab and then click on "Run Tests". Wait for a minute or so, then refresh the webpage. Once the tests have finished running, you'll see results for the Style Checker and Type Checker components of the checker program (see both the Automated Tes!ng tab and results files under the Submissions tab). Note that these are not actually marks -- just the checker results. This is the same checker that we have provided to you in the starter code. If there are errors, edit your code, run the checker program again on your own machine tocheck that the problems are resolved, resubmit your assignment on MarkUs, and (if !me permits)
a#er the 24 hour period has elapsed, rerun the checker on MarkUs.
No Remark Requests
No remark requests will be accepted. A syntax error could result in a grade of 0 on the assignment.
Before the deadline, you are responsible for running your code and the checker program to iden!fy and resolve any errors that will prevent our tests from running.
What to Hand In
The very last thing you do before submi!ng should be to run the checker program one last "me.
Otherwise, you could make a small error in your final changes before submi&ng that causes your code to receive zero for correctness.
Submit twitterverse_functions.py and test_all_followers.py on MarkUs by following the instruc!ons on the course website. Remember that spelling of filenames, including case, counts: your
file must be named exactly as above.
Future Thoughts
To make your program realis!c, the only piece that is missing is code to get the Twi"er data directly from Twi"er, rather than through a data file. As you do more computer science, you will get more experience using APIs, like the one used to make the connec!on to Twi"er.
While the query format we used in this assignment is fairly general, there are many things you can't express with our queries. Even the simple query "who do I follow who doesn't follow me?" can't be expressed because we have no way to negate. You may find it interes!ng to think about other sorts of queries that can't be expressed. The format we invented for our queries defines a kind of formal "language". (We call it a formal language to contrast with "natural" languages like Cantonese or
French: a language that humans grow up speaking.) The most popular query language in computer science is called SQL. It is much more powerful than our query language, and you will learn about it if you take CSC343 (Introduc!on to Databases).
Due: Monday, December 7 by 4:00pm (Toronto !me)
November 24, 2020: Added a clarifica"on to the "Sor"ng your output" sec"on regarding the ordering of strings in Python.
Goals of this Assignment
Prac!ce working with structured files.
Prac!ce building and using dic!onaries.
Write uni"ests for a func!on.
Use top-down design to break a problem down into subtasks and implement helper func!ons to complete those tasks.
Note the last goal in the list -- in this assignment, you will be doing significantly more design of func!ons than in the previous ones. We will give you a smaller number of func!ons to write, and you will write helper func!ons to solve the subtasks of each of those func!ons.
Introduc!on: The Twi"erverse
This handout page explains the problem you will be solving, the data you will be working with, and the tasks you need to complete. Please read it carefully -- you will need to read all of the informa!on here to complete the assignment successfully.
Twi"er is a social networking website where users can post very short messages known as "tweets".
Each Twi"er user can choose to "follow" other users, which means that they see those users' tweets.
A Twi"er user sees the tweets of users they are "following", and their tweets are seen by their "followers" (the users who follow them).
All the "follow" connec!ons define a network among Twi"er users, and it's quite interes!ng to look for pa"erns in the connec!ons. Tools like Twiangulate let you explore ques!ons like "what connec!ons do two users have in common?". In this assignment, you'll write func!ons that let you ask
ques!ons (or "queries") about a Twi"er dataset.
While it is possible to get data directly from Twi"er, we have simplified things for you and provided the data already stored in a file.
Your Tasks
For this assignment, you are required to:
1. Write the 6 required func!ons described later in this handout.
2. Write helper func!ons as needed. All helper func!ons should have complete docstrings, but there are no other requirements for your helper func!ons.
3. Write unittest s for one of the required func!ons, all_followers .
This is your first experience designing a program of this size. You will have to do far more planning than in the previous two assignments. It is likely you will need to break most of the required func!ons down into subtasks and write helper func!ons for those subtasks.
The Twi"er Data File
A Twi"er data file contains a series of one or more user profiles, one a#er the other. Each user profile has the following elements, in this order:
A line containing a non-blank, non-empty username. You may assume that usernames are unique (i.e. a single username will not occur more than once in the file), and that usernames do not contain any whitespace. Usernames are case sensi!ve.
A line for the user's actual name. If they did not provide a name, this line will be blank.
A line for the user's loca!on, or a blank line if they did not provide one.
A line for the URL of a website, or a blank line if they did not provide one.
Zero or more lines for the user's bio, then a line with nothing but the keyword ENDBIO on it. This marks the end of the bio, and is not considered part of it. (You may assume that no bio has the string ENDBIO within it.) If the user did not provide a bio, the ENDBIO line will come immediately a#er the website line, with no blank line in between.
Zero or more lines each containing a single username of someone that this user is following, then a line with the keyword END on it. (You may assume that no one has END as their username.) A user cannot follow themselves. You may assume that every user that appears in this sec!on of the file has a user profile in the Twi"er data file.
A blank line is one that contains only whitespace characters. For this assignment, whitespace characters include spaces, tabs, and newlines.
No!ce that the keywords ( ENDBIO and END ) act as separators in this file. All of their le"ers are capitalized, and the keywords contain no punctua!on. You may assume that no keyword separators will appear as any part of the content of a user profile.
Examples
Here is a sample user profile that might occur among many in a file:
tomCruise
Tom Cruise
Los Angeles, CA
http://www.tomcruise.com
Official TomCruise.com crew tweets. We love you guys!
Visit us at Facebook!
ENDBIO
katieH
NicoleKidman
END
The file data.txt in the provided starter files is a small example of a complete Twi"er data file (and was made by hand). The file rdata.txt is a much larger example (and is made from real data extracted from Twi"er). These should help you confirm your understanding of the file format and will be useful in tes!ng your program.
Cycles in the data
Although a user cannot follow themselves, there can be "cycles" such as this: user A can be following B who is following A. This is the shortest possible cycle. Of course, cycles can be longer.
The Query File
Note that the word "query" just means "ques!on". In computer science, we use it to specify a request for informa!on.
For this assignment, a query will be provided in a file. Below we will review the high-level parts of the
query, look at an example, and then describe the query file format.
Overview
A query has three components: a search specifica!on, a filter specifica!on, and a sor!ng specifica!on.
The search specifica"on describes how to generate a list of Twi"er usernames, star!ng with an ini!al username (a list of length one) and then finding their followers or people they are following, then people that are those people's followers or who they are following, and so on. When processing the search specifica!on, don't try to do anything to avoid cycles. For instance, if the search specifica!on says to find the people who user A is following, and from there the people they are following, you could find yourself back at user A. Don't try to avoid that.
A#er processing the search specifica!on, we have a list of Twi"er usernames. Its length could be zero. For example, if the ini!al username is 'dianelynnhorton' and the search specifica!on contains a single 'followers' keyword, then the number of results of the search will be zero if 'dianelynnhorton' has no followers.
The filter specifica"on describes how to filter the list of usernames produced by the search specifica!on. The filtering can be based on whether or not they are following a par!cular user, whether or not a par!cular user is their follower, whether or not their name contains a par!cular string (case-insensi!ve), or whether or not their loca!on contains a par!cular string (case-insensi!ve).
A#er processing the filter specifica!on, we have a possibly reduced list of usernames.
Once the search results have been found and filtered, the sor"ng specifica"on describes how the results should be sorted. Prior to the sor!ng step, the order of usernames does not ma"er.
Example query
Here is an example query:
SEARCH
tomCruise
following
following
following
FILTER
following c
location-includes a
SORT
popularity
The search specifica!on in this par!cular query has four steps.
1. Start with a list containing the username to start the search from; i.e. ['tomCruise'] . Let's call that list L1.
2. The search keyword 'following' says to replace each username p in L1 with the usernames of the users who p is following. This yields a new list, L2.
3. For the next 'following' keyword, we start with L2 and repeat the same opera!on as in the previous step, yielding another list, L3.
4. For the final 'following' keyword, we start with L3 and repeat that opera!on one last !me, yielding list L4.
No!ce that each step yields a list of zero or more usernames that is the input to the next step. There should be no duplicates in the final results list. Duplicates should be removed a#er each step.
The Twi"er data file diagram_data.txt in the provided files contains the follower/following rela!onships as represented by this diagram.
For those rela!onships, the search specifica!on above would yield this list of usernames: ['i', 'j', 'h', 'k', 'tomCruise'] . Make sure that you can see how the four lists, ending with this final one, are generated. No!ce that the final list contains the users you can get to in three "steps" of the "following" rela!onship, star!ng from 'tomCruise' .
The final list generated by the search specifica!on becomes the input to the filter specifica!on. For our current example, the filter specifica!on says that the list should be filtered in this way: a user should be kept only if they are following user 'c' and has a loca!on that includes the string 'a' .
Based on the query above, the final filtered list would be ['tomCruise'] .
The sor!ng specifica!on says to order the users according to their popularity.
Now let's look in detail at the exact format and meaning of a query.
Overall format of a query
The format of the query data file is as described below, in this order:
A line containing the keyword SEARCH .
The search specifica!on.
A line containing the keyword FILTER .
The filter specifica!on.
A line containing the keyword SORT .
The sor!ng specifica!on.
No!ce that the keywords above all act as separators in this file and are in all capital le"ers. There are other keywords (such as following which can be part of a search specifica!on) that are not separators; they are not capitalized.
Format and meaning of the search specifica!on
The format of a search specifica!on is as shown below, in this order:
A line containing a single username
Zero or more lines, in any order, each containing one of:
the keyword followers
the keyword following
Each step in the search specifica!on involves crea!ng a new list of users by going through each user in the current list and finding their followers or who they are following. Importantly, at each step, we want to create the new results list by replacing each user from the current list with the users that they are following or who are their followers.
The possible search opera!ons for a given user are:
followers : this will get all usernames for users who are following the given user (i.e. that have the given username in the value list paired with the key 'following' in their data in the Twi"erverse dic!onary -- more on this later).
following : this will get all usernames for users who the given user is following (i.e. the value list paired with the key 'following' in the data associated with the given user in the Twi"erverse dic!onary).
Note that, if the list of usernames is ['A'] , and we perform a 'followers' or 'following' opera!on, 'A' will not be in the resul!ng list, because users cannot follow themselves. If 'A' is one of mul!ple usernames in the current list, then 'A' can appear in the results list, but only if one of the other usernames produces 'A' from the search opera!on.
The final list of results should not contain any duplicates. Duplicates should be removed a#er each opera!on is performed. The order of the final list of results from the search opera!on does not ma"er.
Format and meaning of the filter specifica!on
The format of a filter specifica!on is:
Zero or more lines, in any order, each containing one of the following:
the keyword name-includes , a space, and a string to match
the keyword location-includes , a space, and a string to match
the keyword bio-includes , a space, and a string to match
the keyword follower , a space, and a string which is a valid username
the keyword following , a space, and a string which is a valid username
Each of the lines has exactly one space within it (the space separa!ng the keyword from the next string), and each keyword will appear at most once per filter specifica!on.
The filters are "addi!ve" in that each filter step should be applied to the list that resulted from the previous filter step. That is, you start with the list from the search specifica!on, filter it based on the first filter step to get a new list L1, filter L1 in the second filter step to get L2, and so on.
The 'following' and 'follower' filters will add users in the new list if they are following a par!cular user, or if a par!cular user is their follower. These filters are defined as:
The 'following' filter adds only users who are following provided username.
The 'follower' filter adds only users who the provided username is following.
The filters that have 'includes' as part of the keyword will do a simple substring search on the string represen!ng the relevant data for the user in the Twi"erverse dic!onary. This means that if the given string occurs anywhere in a user's name (for a 'name-includes' filter) or a user's loca!on (for a 'location-includes' filter), then that user is to be kept in the list. The substring search should be case-insensi!ve. For example, if the filter specifies users whose loca!ons include the string "USA", then users with loca!on "USA", "Modesto, California, USA", "kansas, usa" or "Musala, Bulgaria" would be kept, and users with loca!on "United States of America" would be excluded. This is far from perfect, but don't try to improve on it.
The order of the final list of results from the filter opera!on does not ma"er.
Format and meaning of the sor!ng specifica!on
The output from your program must include every user whose username is s!ll in the list a#er the search and filter specifica!ons have been processed.
The sor!ng specifica!on describes how the final list of results should be ordered. It consists of one line containing one of these keywords: 'username' , 'name' , 'popularity' .
The users in the final results list must be ordered as indicated by the keyword of the sor!ng specifica!on. Output that is sorted by username or name should be in alphabe!cal order, so that strings star!ng with 'a' are at the beginning of the output. See the sec"on "Sor"ng your output" later in this page for informa"on on how to sort. Although usernames are unique, names are not. If any users have the same name, sort them by username. When sor!ng users by popularity, the most popular users should come first. A user's popularity is defined to be the number of followers they have. (Not the number of people they are following!) If any users are !ed for popularity, sort them by username.
We can use Python's built-in sort and/or string comparison for sor!ng by username. However, we'll have to find another way to sort by username name or popularity. There is more informa!on on sor!ng is later in the "Sor!ng your output" sec!on.
Data Structures
To increase the readability of our code, we will use dic!onaries to represent both the Twi"er data and the query in our program. In this sec!on, we'll look at the format of the data and query dic!onaries.
The Twi"erverse Data Dic!onary
The type of the Twi"erverse dic!onary is Dict[str, Dict[str, object]] . (We use the type object here to indicate that the values in the inner dic!onary can have different types.) More specifically, each key in the Twi"erverse dic!onary is the username of a Twi"er user, and the value associated with that key is a dic!onary containing addi!onal informa!on about the user. Each of these inner dic!onaries will contain the keys 'name' , 'location' , 'web' , 'bio' , and 'following' . The value associated with the 'following' key is a list of zero or more strings, and the values associated with the rest of the keys are strings (and may be the empty string).
For example, if the following user informa!on is included in our Twi"er data file...
tomCruise
Tom Cruise
Los Angeles, CA
http://www.tomcruise.com
Official TomCruise.com crew tweets. We love you guys!
Visit us at Facebook!
ENDBIO
katieH
NicoleKidman
END
... then this key/value pair will be in our Twi"erverse dic!onary.
'tomCruise': {'name': 'Tom Cruise',
'bio': 'Official TomCruise.com crew tweets. We love you guys!\nVisit us at Facebook
!',
'location': 'Los Angeles, CA',
'web': 'http://www.tomcruise.com',
'following': ['katieH', 'NicoleKidman']}
No!ce that the newlines at the end of each line are removed from the data stored in the Twi"erverse dic!onary, with the excep!on of the bio informa!on, which needs to keep its inner newlines so that we could reconstruct the bio format the user chose. In general, you should remove leading and trailing whitespace from each line.
The usernames in the following list should be in the same order as in the data file.
And another example: if the following user informa!on is included in our Twi"er data file (note the blank lines for name, loca!on, and web, and zero lines for bio and following)...
quietTweeter
ENDBIO
END
... then this key/value pair will be in our Twi"erverse dic!onary.
'quietTweeter': {'name': '',
'bio': '',
'location': '',
'web': '',
'following': []}
Note that dic!onaries are not ordered, so don't worry if your dic!onary output is in a different order, as long as the items are the same.
The Query Dic!onary
We will also use dic!onaries to represent queries in our program. The query dic!onary contains three items; the key 'search' , whose value is a search specifica!on dic!onary, the key 'filter' , whose value is a filter specifica!on dic!onary , and the key 'sorting' , whose value is a sor!ng specifica!on dic!onary.
The search specifica!on dic!onary contains the keys 'username' and 'operations' , whose values are the string represen!ng the username to start at, and a list of strings represen!ng the opera!ons to perform, respec!vely. Note that the list that is the value associated with the key 'operations' may be empty, if there is only one line between the SEARCH and FILTER keywords in the query file. If there is only one line between SEARCH and FILTER, then it represents the username to start at, and there are no opera!ons to be performed in the search step. The opera!ons must be performed in the order that they are in the list.
The filter specifica!on is a dic!onary with strings represen!ng filter keywords as keys, and the strings to match as values. Note that this dic!onary may be empty if there are no filters to apply, and that each filter keyword will appear at most once in the filter sec!on of a query file.
The sor!ng specifica!on is a single string value represen!ng the order to sort the results in.
For example, if our query file looks like this...
SEARCH
tomCruise
following
following
following
FILTER
following KatieH
location-includes USA
SORT
popularity
... then the query dic!onary will have these key/value pairs.
{ 'search': {'username': 'tomCruise', 'operations': ['following', 'following', 'following']},
'filter': {'following': 'KatieH', 'location-includes': 'USA'},
'sorting': {'sort-by': popularity'}}
Required Func!ons
In the starter code file twitterverse_functions.py , complete the following func!on defini!ons. In addi!on, you will need to add some helper func!ons to aid with the implementa!on of these required func!ons.
|
Func"on name: (Parameter types) -> Return type |
Full Descrip"on (paraphrase for your docstring) |
|
process_data: (TextIO) -> TwitterverseDict |
The parameter represents a Twi"er data file that is already open for reading. Read the file and return the data in the Twi"erverse dic!onary format. Note: in the docstring, do not provide example calls for func!ons that read files. |
|
process_query: (TextIO) -> QueryDict The parameter represents a query file that isalready open for reading. Read the file and return the query in the query dic!onary format. Note: in the docstring, do not provide example calls for func!ons that read files. |
|
|
all_followers: (TwitterverseDict, str) -> List[str]: The first parameter represents a Twi"erverse dic!onary, and the second parameter represents a username. Iden!fy all the usernames that are following the user specified by the second parameter and return them as a list. This is a helper func!on you will likely find helpful in some of the later required func!ons. |
|
|
get_search_results: (TwitterverseDict, SearchDict) -> List[str]: The first parameter represents a Twi"erverse dic!onary format, and the second parameter represents a search specifica!on dic!onary. Perform the specified search on the given Twi"er data, and return a list of strings represen!ng usernames that match the search criteria. The parameters must not be modified by the func!on. |
|
|
get_filter_results: (TwitterverseDict, List[str], FilterDict) -> List[str] The first parameter represents a Twi"erverse dic!onary format, the second parameter represents a list of usernames, and the third parameter represents a filter specifica!on dic!onary. Apply the specified filters to the given username list one at a !me to produce the filtered list, and return the resul!ng list of usernames. The parameters must not be modified by the func!on. |
|
|
get_sorted_results: (TwitterverseDict, List[str], SortingDict) -> List[str]: The first parameter represents a Twi"erverse dic!onary format, the second parameter represents a list of usernames, and the third parameter represents a sor!ng specifica!on dic!onary. Sort the results based on the given sor!ng specifica!on and return the final results list. The parameters must not be modified by the func!on. |
Once you have correctly implemented the func!ons in twitterverse_functions.py , execu!on of the main program ( twitterverse_program.py ) will:
1. Read a data file and produce the Twi"erverse data dic!onary.
2. Read a query file and produce the query dic!onary.
3. Compute the results of the search opera!ons.
4. Apply the filters to the search results.
5. Sort the final results.
You may try running this file on the provided data and query files, as well as your own files that you have created for tes!ng. Note that just running the twitterverse_program.py is NOT sufficient to test your code.
Required Tes!ng ( unittest )
Write and submit a set of uni"ests for the all_followers func!on in the file test_all_followers.py .
You should create Twi"erverse dic!onaries as inputs for your test cases inside the test_all_followers.py file. Do NOT call open open in your test file, or anywhere else in your code.
Files to Download
All of the files included in the download for the assignment are listed in this sec!on. These files must all be placed in the same folder.
Please download the Assignment 3 files and extract the zip archive. The following paragraphs explain the files you have been given.
Python Code
The starter code for this assignment is in the file twitterverse_functions.py . This is the primary file you will be edi!ng.
The main program file is twitterverse_program.py . It is complete and must not be changed.
The starter code for the required uni"ests is in the file test_all_followers.py . You will edit and submit this file as well.
We are providing a checker as usual: a3_checker.py . It is complete and must not be changed.
Sample Twi"er Data and Sample Queries
We have provided a number of text files containing Twi"er data, and queries on that data. All files in the provided zip file that contain data in their filename are Twi"er data files, and all files in the provided zip file that contain query in their filename are query files. Put these files in the same directory as your other A3 files.
The query files query1.txt , query2.txt , and query3.txt are queries on the data in data.txt , and the file query4.txt is a query on the data in rdata.txt .
a3_checker.py
We are providing a type-check module that can be used to test whether your func!ons in twitterverse_functions.py have the correct parameter and return types. To use the type checker, place a3_checker.py in the same folder (directory) as your twitterverse_functions.py and run it.
If the type-checks pass: the output will tell you that the typechecker passed (and what it means for the typechecker to pass!). If the typechecker passes, then the parameters and return types match the assignment specifica!on for each of the func!ons.
If the type-checks fail: Look carefully at the message provided. One or more of your parameter or return types does not match the assignment specifica!on. Fix your code and re-run the tests. Make sure the tests pass before submi&ng.
We will do a much more thorough job of tes!ng your code once you hand it in, so be sure that you have thoroughly tested it yourself. With the func!ons in this assignment, there are many more possible cases to test (and cases where your code could go wrong). If you want to get a great mark on the correctness of your func!ons, do a great job of tes!ng your func!ons under all possible condi!ons. Then we won't be able to find any errors that you haven't already fixed!
Addi!onal Requirements
Do not change any of the exis!ng code. Add to it as specified in the comments.
Do not call print , input , or open in the code that you write, except within the if __name__ ==
'__main__' block. No!ce that two of the required func!ons take an open file, not a string.
Do not use any break or continue statements. Any func!ons that do will receive a mark of zero.
We are imposing this restric!on (and we have not even taught you these statements) because they are very easy to "abuse," resul!ng in terrible code.
Do not modify or add to the import statements provided in the starter code.
Do not add any code outside a func!on defini!on.
Do not mutate func!on parameters unless specified.
Do not use Python language features for sor!ng that we have not covered in the course, like the
op!onal parameter key or the func!on sorted .
Error checking
You may assume that the files actually contain a valid Twi"er data file and a valid query file, respec!vely. You do not need to check either file to make sure that it is a valid Twi"er or query file.
You may be wondering how you can be sure that the content of the Twi"er data file is valid. For example, there are probably rules about legal Twi"er usernames and there are certainly rules about valid URLs. Don't confirm that the username, URL, or any other elements of the Twi"er data file are valid. Simply read whatever string is in the relevant posi!on in the file and use it as is.
Reading the Twi"er data file
Typically, the easiest way to read through a data file is with a for-loop, but because of the structure of a Twi"er data file, you will need to use while loops. We recommend making sure you are very comfortable with the exercises from Weeks 7 and 8 before beginning this assignment.
Watch out for how you handle \n (newline) characters. If you use readline to get a line of the file, it will have a \n on the end (except possibly on the last line in the file, although that line may also have a \n -- your code should work in both cases). You need to be aware of that before you try to compare it to keywords like END . On the other hand, there are !mes when you really do need a \n .
For instance, when you store the contents of a user's bio, it must include any newline characters that appear in the middle of the bio, so that the original format is maintained.
Sor!ng your output
Processing the sor!ng specifica!on requires you to sort the list of users according to username, name, or popularity. Most likely you thought of using list.sort to accomplish the task. However, by default list.sort uses simple < to compare pairs of list elements. We know that < on strings compares them alphabe!cally. So list.sort will simply sort the strings into alphabe!cal order. In some cases, that is not the ordering you want. Clarifica"on: Python does not actually sort strings in alphabe"cal order. The correct term here is lexicographical order, which is not a term we expect you to know. The inten"on is that you are just using the comparison operators to sort your strings.
For example, if we have a list of users ['user1', 'user2'], with user1's name being 'jacqueline' and user2's name being 'Jen', then if the list is sorted by name, the result should be ['user2', 'user1'], because 'Jen' < 'jacqueline' according to Python.
Fortunately, there are ways to get around this. In the starter code for twitterverse_functions.py , we have provided our own implementa!on of a sor!ng func!on that has a parameter that you can use to control the ordering. All you have to do is send tweet_sort a func!on that takes two parameters of the same type as the elements in your list (in this assignment, the two parameters will be strings
represen!ng usernames) and returns:
-1 if the first one should appear before the second one in a sorted list 1 if the first one should appear a#er the second one in a sorted list, and 0 if they are "!ed", so that it doesn't ma"er which one comes first.
We have also provided some func!ons that return -1, 1, or 0 for each possible sor!ng case.
The simple module sort_demo.py demonstrates how this can be done. It is a good example of passing a func!on to a func!on. We strongly recommend you spend some !me understanding this code before you work on the sor!ng specifica!on.
The file sort_demo.py also contains some examples of wri!ng docstring examples that span mul!ple lines.
Constants
Unlike in the other assignments this term, we are not providing you with constants to use, or requiring the use of constants. However, you are welcome and encouraged to create any constants you think you need to make your code clean and readable.
Marking
These are the aspects of your work that may be marked for A3:
Coding style (20%):
Make sure that you follow Python style guidelines that we have introduced and the Python coding conven!ons that we have been using throughout the semester. Although we don't provide an exhaus!ve list of style rules, the checker tests for style are complete, so if your code passes the checker, then it will earn full marks for coding style with one excep!on:
docstrings for any helper func!ons you add may be evaluated separately. For each occurrence of a PyTA error, one mark (out of 20) deduc!on will be applied. For example, if a C0301 (line-toolong) error occurs 3 !mes, then 3 marks will be deducted.
All func!ons you design and write on your own (helper func!ons), should have complete
docstrings including precondi!ons when you think they are necessary.
Correctness (70%):
Your func!ons should perform as specified. Correctness, as measured by our tests, will count for the largest single por!on of your marks. Once your assignment is submi"ed, we will run addi!onal tests not provided in the checker. Passing the checker does not mean that your code will earn full marks for correctness.
Tes"ng (10%):
We will run the uni"ests that you submit on a series of flawed (incorrect) implementa!ons we have wri"en. Your tes!ng mark will depend on how many of the flawed implementa!ons your uni"ests catch, whether they successfully pass a working (correct) implementa!on, and whether your test files contain redundant (unnecessary) tests.
How should you test whether your code works
First, run the checker and review ALL output — you may need to scroll. Remember that the checker
ONLY shows you style feedback, and that your func!ons take and return the correct types. Passing the checker does not tell you anything about the correctness of your code.
A3 Checker
Note: the checker program for Assignment 3 only checks your twitterverse_functions.py file. It is up to you to make sure your test_all_followers.py file runs. Note that your test_all_followers.py file will not be marked for style.
We are providing a checker module ( a3_checker.py ) that tests two things:
whether your code follows the Python style guidelines, and whether your func!ons are named correctly, have the correct number of parameters, and return the correct types.
To run the checker, open a3_checker.py and run it. Note: the checker file should be in the same directory as your twitterverse_functions.py , as provided in the starter code zip file. When you run your own checker, be sure to scroll up to the top and read all messages.
If the checker passes for both style and types:
Your code follows the style guidelines.
Your func!on names, number of parameters, and return types match the assignment specifica!on.
This does not mean that your code works correctly in all situa"ons. We will run a different set of tests on your code once you hand it in, so be sure to thoroughly test your code yourself before submi&ng.
If the checker fails, carefully read the message provided:
It may have failed because your code did not follow the style guidelines. Review the error descrip!on(s) and fix the code style. Please see the PyTA documenta!on for more informa!on about errors.
It may have failed because:
you are missing one or more func!on,
one or more of your func!ons is misnamed,
one or more of your func!ons has the incorrect number or type of parameters, or
one of more of your func!on return types does not match the assignment specifica!on, or
2020 Fall
Home
Announcements
Start of Term FAQ
Lectures
Piazza
Office Hours
Videos and Exercises
Assignments
Test Informa!on
Modules
So#ware Installa!on
MarkUs
Teaching Lab Accounts and Computer Labs
Bb Collaborate
Account
Dashboard
Courses
Groups
Calendar
Inbox
History
Course
Evals
Help
your .py file is misnamed or in the wrong place.
Read the error message to iden!fy the problema!c func!on, review the func!on specifica!on in the handout, and fix your code.
Make sure the checker passes before submi&ng.
Running the checker program on Markus
In addi!on to running the checker program on your own computer, run the checker on MarkUs as well. You will be able to run the checker program on MarkUs once every 12 hours (note: we may have to revert to every 24 hours if MarkUs has any issues handling every 12 hours). This can help to iden!fy issues such as uploading the incorrect file.
First, submit your work on MarkUs. Next, click on the "Automated Tes!ng" tab and then click on "Run Tests". Wait for a minute or so, then refresh the webpage. Once the tests have finished running, you'll see results for the Style Checker and Type Checker components of the checker program (see both the Automated Tes!ng tab and results files under the Submissions tab). Note that these are not actually marks -- just the checker results. This is the same checker that we have provided to you in the starter code. If there are errors, edit your code, run the checker program again on your own machine tocheck that the problems are resolved, resubmit your assignment on MarkUs, and (if !me permits)
a#er the 24 hour period has elapsed, rerun the checker on MarkUs.
No Remark Requests
No remark requests will be accepted. A syntax error could result in a grade of 0 on the assignment.
Before the deadline, you are responsible for running your code and the checker program to iden!fy and resolve any errors that will prevent our tests from running.
What to Hand In
The very last thing you do before submi!ng should be to run the checker program one last "me.
Otherwise, you could make a small error in your final changes before submi&ng that causes your code to receive zero for correctness.
Submit twitterverse_functions.py and test_all_followers.py on MarkUs by following the instruc!ons on the course website. Remember that spelling of filenames, including case, counts: your
file must be named exactly as above.
Future Thoughts
To make your program realis!c, the only piece that is missing is code to get the Twi"er data directly from Twi"er, rather than through a data file. As you do more computer science, you will get more experience using APIs, like the one used to make the connec!on to Twi"er.
While the query format we used in this assignment is fairly general, there are many things you can't express with our queries. Even the simple query "who do I follow who doesn't follow me?" can't be expressed because we have no way to negate. You may find it interes!ng to think about other sorts of queries that can't be expressed. The format we invented for our queries defines a kind of formal "language". (We call it a formal language to contrast with "natural" languages like Cantonese or
French: a language that humans grow up speaking.) The most popular query language in computer science is called SQL. It is much more powerful than our query language, and you will learn about it if you take CSC343 (Introduc!on to Databases).
2020-12-01