CS B659: Reinforcement Learning

News:

Here is a document with more details and intuitions about Bellman equations
New page available on the Reinforcement learning code used for this course.
Project page is up!
Here are two projects from last year that were well done: here and here
A great reference to brush up on statistics is the book All of Statistics

Basic info:

Class meets Tuesday & Thursday 2:30pm - 3:45 pm, Ballantine Hall Room 330
Instructor: Adam White

office: 201-I Lindley Hall
office hours: wed 1-3pm
email: adam@adamwhite.ca
web: adamwhite.ca

AI: Matthew Schlegel

office: Lindley Hall 406
office hours: (Tentative) *: 4-6, Wed: 4-6 (* - The day homework is due)
email: mkschleg@umail.iu.edu

AI: Su Zhang

office: Lindley Hall 004
office hours: (Tentative) *: 4-6, Mon: 4-6 (* - The day homework is due)
email: zhangsu@indiana.edu

Class schedule

This schedule is tentative and may change throughout the semester.

Readings and assignment questions are from the main course text: Sutton and Barto

Supplementary material can be found in Szepesvari's book.

These slides will be released progressively.

Week	Date	Lecture	Readings and Deadlines
1	Jan 10	Chapter 1: Introduction	Read Chapter 1 of Sutton and Barto Assignment #1 Released
	Jan 12	Chapter 2: Evaluative feedback	Read Chapter 2 of Sutton & Barto
2	Jan 17	Chapter 2: Evaluative feedback continued	Two thought questions due monday night.
	Jan 19	Chapter 3: The reinforcement learning problem	Read Chapter 3
3	Jan 24	Chapter 3 continued	One thought question due monday night.
	Jan 26	Chapter 3 completed. Start of Chapter 4 Dynamic programming	Assignment #1 Due
4	Jan 31	Finish Chapter 4: Dynamic programming	Read Chapter 4 Thought question on Ch4 due Monday 1 min to Midnight Assignment #2 released
	Feb 2	Chapter 5: Monte Carlo methods
5	Feb 7	No class: work on assignments and projects
	Feb 9	Guest lecture: David Crandall
	Feb 14	Monte Carlo methods
6	Feb 16	Monte Carlo methods
	Feb 21	Chapter 6: Temporal difference learning Project info	Assignment #2 due
7	Feb 23	Chapter 6:TD learning	Assignment #3 released
	Feb 28	Chapter 7: n-step TD methods
8	Mar 2	Quiz Review Chapter 8: Planning and learning
	Mar 7	Thoughts on Research Chapter 8: Planning and learning
9	Mar 9	Chapter 9: Function approximation
	Mar 14	No classes: Spring break
10	Mar 16	No classes: Spring break	Assignment #3 Due Assignment #4 Released
	Mar 21	Chapter 9:Function approximation
11	Mar 23	Policy evaluation with function approximation	30 minute in class quiz (5% of total grade)
	Mar 28	Policy evaluation with function approximation
12	Mar 30	Control with function approximation
	April 4	lambda-returns and Eligibility traces
13	Apr 6	Advanced topics (LSTD and Off-policy gradient TD)	Assignment #4 Due Assignment #5 released
	Apr 11	Lecture Cancelled
14	Apr 13	Lecture by Martha White Advanced topics (gradient TD and Policy gradient methods)
	Apr 18	Finish gradient TD and Policy gradient methods
15	Apr 20	RL and Psychology
	Apr 25	Review Lecture	Thought question Ch14 due
16	Apr 27	Practice Problems	Assignment #5 due
	May 2	Final Exam	2:45- 4:45 p.m., Tues., May 2
17	May 4	Exam Period	Research projects due