Mathematical Machine Learning -- Reinforcement Learning


Course Information

The information on this website is preliminary and will be updated continuously. Additional information will soon be available publicly in the LSF information system of the university.

Description

This seminar is addressed to master students of Scientific Computing and Mathematics. Together we will learn about reinforcement learning, from the very basics until more advanced concepts like policy gradient methods.

Topics

In the following, an excerpt of the list of contents of “Reinforcement Learning - An Introduction” by Sutton and Barto are listed:

Presentation number Date Time Chapter
1 14.05.2024 18:00 C3 - Finite Markov Decision Processes
2 21.05.2024 18:00 C4 - Dynamic Programming
3 28.05.2024 18:00 C5 - Monte Carlo Methods
4 04.06.2024 18:15 C6 - Temporal Difference Learning
5 11.06.2024 18:00 C7 - n-step Bootstrapping
6 18.06.2024 18:00 C8 - Planning and Learning with Tabular Methods
7 02.07.2024 18:15 C9 - On-policy Prediction with Approximation
8 09.07.2024 18:00 C10 - On-policy Control with Approximation
9 16.07.2024 18:00 C12 - Eligibility Traces

Chapters 1 & 2 should be read in preparation before the start of the seminar. Chapters 3 through 12 (excluding 11) will be presentation topics, one for each of the 9 participating students.

Exam / Presentations

Participant will prepare and present a presentation on one of the sections of “Reinforcement Learning - An Introduction” by Sutton and Barto, as listed in Topics. The presentations are prepared and presented alone, i.e., no group work, and should be approximately one hour long. Grading is based on the presentation and the discussion thereafter. If you have questions, you can contact us.

Dates and Timeline

The organizational meeting was on Tuesday 16.04.2024 at 6pm in SR 10. During this meeting, participants chose one of the available topics.

Starting KW20 (20th week of the year), we will meet regularly once a week for each presentation. In the organizational meeting we decided on a regular meeting time. The regular meetings will be on Tuesdays 18:00 in SR 10.

(Pre-)registration

The registration is closed, the participants are fixed, and have an assigned presentation topic.

Resources

  • Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.

  • Slides of the organizational meeting here

Material developed during the course

  • Slides of the first presentation on Finite Markov Decision Processes here