Principles of Reinforcement Learning

Atelier thématique · Université Lumière Lyon 2 · 2026

Course material: github.com/pegahani/Atelier-thematique-Principles-of-Reinforcement-Learning

Overview

Reinforcement learning studies how an agent learns to make good decisions by interacting with an environment. In this atelier you learn it by building agents: short theory blocks, then coding in pairs, from planning on a small grid world to deep RL agents that land a spacecraft. There is no exam.

Format: 4 sessions, one of 2h and three of 4h. Sessions 3 and 4 end with a LunarLander leaderboard.

Sessions

#	Topic	Hands-on
1	Markov decision processes, Bellman equations, dynamic programming	Value iteration on FrozenLake
2	Learning without a model: Monte Carlo, TD learning, SARSA, Q-learning	Tabular agents
3	Deep reinforcement learning	DQN from scratch
4	Policy gradient methods, PPO, reinforcement learning for language models	PPO on LunarLander

Learning objectives

Model a decision problem as a Markov decision process and solve it by dynamic programming
Implement and compare value-based and policy-based algorithms
Implement DQN and PPO and make them train reliably
Understand how reinforcement learning is used to train large language models (RLHF, GRPO)

Prerequisites

Python and NumPy, basic probability and linear algebra. Some familiarity with PyTorch helps but is not required. Setup instructions are in the course repository.

References

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., 2018. Free online
D. Silver, UCL course on reinforcement learning (videos)
Hugging Face Deep RL course

Contact

Questions about the course: pegah.alizadeh@univ-lyon2.fr. Reinforcement learning is also my research area: students interested in an internship are welcome to get in touch.