Prerequisites

Prior deep learning experience (e.g. ELEC_ENG/COMP_ENG 495 Deep Learning Foundations from Scratch) and familiarity with the Python programming language.

REQUIRED TEXT:  R. S. Sutton and A. C. Barto, Reinforcement Learning, MIT Press, 2nd edition, 2020.

COURSE OUTLINE:

Intro

Multi-Armed Bandits

Markov Decision Processes

Dynamic Programming

Temporal Difference Learning (Q-learning)

Planning and Learning with Tabular Methods

Function Approximation (polynomials, NNs)

Approximate Solution Methods I (on policy prediction)

Approximate Solution Methods II (on-policy control, eligibility traces)

Policy gradient methods

Applications

PROBLEM SETS:

Weekly pencil-and-paper and computer problems will be assigned and graded. Homeworks will be typically assigned on Mondays and will be due on Monday a week later at midnight. No late homeworks will be accepted.

PROJECT:

A literature survey or computer project is required.  Please submit a short description (a few     paragraphs, less than a page) of the course project you would like to work on by Monday May 9, 2022, midnight. You can work with other students in the class (the maximum size of a group is 4 people.)

Project Scope:

Your choice of project is up to you (and your team). All we ask is that it be related to the course material, and that you pick something that interest you. If you are having trouble deciding on a   project idea talk with one of us (TAs and me).

Deliverables:

Your final deliverables for the course project will be

1) A short Python Jupyter notebook (less than 5 pages long) explaining and illustrating your project topic, including any fundamental Python code.

2) A short Youtube video (3 - 5 minutes in length maximum) summarizing what you / your team pursued, any trials and setbacks along the way, and conclusions.

Both of these items (a URL to the Youtube video) are to be uploaded to canvas. Due date: Wed 6/8/22 at midnight.

COURSE GRADE:

Final grades for the course will be based on the homework assignment grades (65%) and the project (35%).

Academic Integrity Statement

Students in this course are required to comply with the policies found in the booklet, "Academic             Integrity at Northwestern University: A Basic Guide". All papers submitted for credit in this course must be submitted electronically unless otherwise instructed by the professor. Your written work may be       tested for plagiarized content. For details regarding academic integrity at Northwestern or to download

the guide, visit:https://www.northwestern.edu/provost/policies/academic-integrity/index.html

Accessibility Statement

COVID-19 Classroom Expectations Statement

Students, faculty, and staff must comply with University expectations regarding appropriate classroom behavior, including those outlined below and in theCOVID-19 Code of Conduct. With respect to            classroom procedures, this includes:

•    Policies regarding masking and social distancing evolve as the public health situation changes. Students are responsible for understanding and complying with current masking, testing,        Symptom Tracking, and social distancing requirements.

•    In some classes, masking and/or social distancing may be required as a result of an Americans    with Disabilities Act (ADA) accommodation for the instructor or a student in the class even when not generally required on campus. In such cases, the instructor will notify the class.

•    No food is allowed inside classrooms. Drinks are permitted, but please keep your face covering on and use a straw.

•    Faculty may assign seats in some classes to help facilitate contact tracing in the event that a student tests positive for COVID-19.  Students must sit in their assigned seats.

If a student fails to comply with theCOVID-19 Code of Conductor other University expectations related to COVID-19, the instructor may ask the student to leave the class. The instructor is asked to report the incident to the Office of Community Standards for additional follow-up.

COVID-19 Testing Compliance Statement

To protect the health of our community, Northwestern University requires unvaccinated students who are in on-campus programs to be tested for COVID-19 twice per week.

Students who fail to comply with current or future COVID-19 testing protocols will be referred to the Office of Community standards to face disciplinary action, including escalation up to restriction from campus and suspension.

Exceptions to Class Modality

Class sessions for this course will occur in person. Individual students will not be granted permission to attend remotely except as the result of an Americans with Disabilities Act (ADA) accommodation as      determined by AccessibleNU.

Maintaining the health of the community remains our priority.  If you are experiencing any symptoms of COVID do not attend class and update your Symptom Tracker application right away to connect with       Northwestern’s Case Management Team for guidance on next steps. Also contact the instructor as soon as possible to arrange to complete coursework.

Students who experience a personal emergency should contact the instructor as soon as possible to arrange to complete coursework.

Should public health recommendations prevent in person class from being held on a given day, the instructor or the university will notify students.

Prohibition of Recording Classes by Students

Unauthorized student recording of classroom or other academic activities (including advising    sessions or office hours) is prohibited. Unauthorized recording is unethical and may also be a    violation of University policy and state law. Students requesting the use of assistive technology as an accommodation should contactAccessibleNU. Unauthorized use of classroom recordings – including distributing or posting them – is also prohibited. Under the University’sCopyright   Policy, faculty own the copyright to instructional materials – including those resources created specifically for the purposes of instruction, such as syllabi, lectures and lecture notes, and          presentations. Students cannot copy, reproduce, display, or distribute these materials.               Students who engage in unauthorized recording, unauthorized use of a recording, or                  unauthorized distribution of instructional materials will be referred to the appropriate               University office for follow-up.

Support for Wellness and Mental Health

Northwestern University is committed to supporting the wellness of our students. Student Affairs has multiple resources to support student wellness and mental health.  If you are feeling distressed or