Projects

Robust, pluralistic and efficient alignment of large language models, with applications to digital humanities

2025 – · Chaire de professeur junior, Université Lumière Lyon 2, funded by the French National Research Agency (ANR)

Large language models are aligned with human preferences through reinforcement learning and preference optimization (RLHF, DPO). My chair studies the decision-making foundations of this step, and its use in the humanities, along four lines:

• Robustness: keeping preference optimization reliable when feedback is noisy or corrupted, and when deployment data differs from training data.
• Pluralistic alignment: aligning models with several legitimate, possibly conflicting, preference distributions instead of a single consensus, and letting personalized models learn from each other.
• Efficiency: reducing the computational and sample cost of reinforcement learning post-training, so that it can run on modest hardware.
• Digital humanities: using language models as research tools in the humanities and social sciences, where interpretations are plural and sources are noisy, which makes robust and pluralistic alignment essential.

Code: SHIFT-Lab, a framework for studying distribution shifts and lightweight corrections in LLM-based decision pipelines

People: Emma Carpanin (Master's student, 2026); Mohammad Amanlou, Yasaman Amou Jafari, Amir MalekHosseini (Bachelor's students, 2026)

Reinforcement learning for energy management systems

2021 – 2026

Learning control policies for energy systems that anticipate future conditions and adapt quickly to new environments, using temporal modeling and meta-reinforcement learning.

People: Théo Zangato (PhD), Aomar Osmani

Papers: Meta-RL with Shared Representations Enables Fast Adaptation in Energy Systems, PAKDD 2026 [arXiv]; Enhancing Decision-Making in Energy Management Systems Through Action-Independent Dynamics Learning, ECAI 2024 [DOI]

Thesis: Améliorer la généralisation et la capacité de prévision en apprentissage par renforcement

Offline reinforcement learning for network control

2025 –

Learning to control mobile communication networks from logged data only: choosing which offline reinforcement learning algorithm to deploy, and applying offline RL to mobility robustness optimization (handover parameters).

People: Nicolas Helson (Master's student), Vasileios Koutsoukis (Master's student)

Papers: Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control, IEEE NOMS 2026 [arXiv]; Offline Reinforcement Learning for Mobility Robustness Optimization, arXiv 2025 [arXiv]

Learning and optimization for demand-responsive transport

2020 – 2023

Combining machine learning with operations research to plan dynamic demand-responsive transport in suburban areas: predicting demand and learning heuristics for routing decisions.

People: Louis Zigrand (PhD), Roberto Wolfler Calvo, Emiliano Traversi

Papers: Optimization-driven Demand Prediction Framework for Suburban Dynamic Demand-Responsive Transport Systems, IJCAI 2023 [DOI]; Machine Learning Guided Optimization for Demand Responsive Transport Systems, ECML/PKDD 2021 [DOI]

Thesis: Learnheuristics for dynamic demand-responsive transport systems

Imitation learning from multiple experts

2022

Learning a policy from the trajectories of several experts of different quality, by merging their trajectories.

People: Sammy Taleb (Master's student), Aomar Osmani

Papers: Improving Imitation Learning by Merging Experts Trajectories, CIKM 2022 [DOI]

Reinforcement learning for expert finding

2024

Locating experts from web search results: an agent learns, with deep Q-learning, which search results to explore to find information about a person.

People: Sammy Taleb (Master's student), Jorge García Flores, Iván Meza

Papers: Reinforcement Learning for Expert Finding from Web Search Results, Studies in Computational Intelligence, 2024 [DOI]

Code: dospordos

Learning for semidefinite relaxations of optimal power flow

2022

Using machine learning to choose clique decompositions that speed up semidefinite relaxations of the optimal power flow problem.

People: Charly Alizadeh (Master's student), Miguel F. Anjos, Lucas Létocart, Emiliano Traversi

Papers: How to Learn the Optimal Clique Decompositions in Solving Semidefinite Relaxations for OPF, IJCNN 2022 [DOI]

Code: OPFSDP.jl, SDPowerModelsDataset.jl, MergeDQN

Place embeddings across cities

2022

Learning representations of places from location-based social networks that transfer from one city to another.

People: Christophe Haikal (Master's student), Christophe Rodrigues, Chongke Bi

Papers: Place embedding across cities in location-based social networks, ACM SAC 2022 [DOI]

Decision-making in Markov decision processes with imprecise rewards

PhD (2016) and follow-up work

When the rewards of an MDP are only partially known, which policy should we choose, and which questions should we ask the user to learn the rewards that matter? This line of work, started in my PhD, computes policies that minimize the maximum regret and studies when deterministic policies are enough.

Papers: Deterministic policies based on maximum regrets in MDPs with imprecise rewards, AI Communications, 2021 [DOI]

Thesis: Elicitation and planning in Markov decision processes with unknown rewards, 2016

Code: Optimization