avatar

Clément Romac

clement[dot]romac[at]gmail[dot]com


About Me

Hi, I am Clément Romac. I recently completed my PhD (manuscript) jointly supervised by Pierre-Yves Oudeyer (FLOWERS team, Inria Bordeaux) and Thomas Wolf (Hugging Face), studying how curiosity-driven RL can help ground Large Language Models. I am currently holding a research engineer position at Inria Bordeaux studying curiosity models, LLMs, and automatic curriculum learning for educational technologies.

Previously, I worked as a Machine Learning Engineer at Weenove and as a Research Engineer in the FLOWERS team working on Automatic Curriculum Learning for Deep RL and automated scientific discovery in complex systems. Apart from research, I have also been involved in various dissemination initiatives, in particular towards education (e.g., I am co-leading the development of the “ChatGPT explained in 5 minutes” series of videos that is used in French classrooms and now embedded in official trainings for workers from all French ministries). I am also the co-organizer of the Machine Learning Meetup group of Bordeaux.

Research Interests

My research focuses on artificial agents’ knowledge and skill acquisition through curiosity-driven Reinforcement Learning (RL). My PhD work notably focused on how curiosity-driven RL can ground Large Language Models (LLMs) through online interactions with an environment. I also studied how artificial (autotelic) agents can select their own goals using Automatic Curriculum Learning.

News

Publications

  1. Image RLDM
    WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-making
    Guillaume Levy, Cedric Colas, Pierre-Yves Oudeyer, Thomas Carta, Clement Romac
    Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM) 2025
    PDF Code Poster

  2. Image ICML
    MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces
    Loris Gaven, Thomas Carta, Clément Romac, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2025.
    PDF Code Poster

  3. Image NAACL Findings
    Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting
    Mohamed Salim Aissi, Clement Romac, Thomas Carta, Sylvain Lamprier, Pierre-Yves Oudeyer, Olivier Sigaud, Laure Soulier, Nicolas Thome
    NAACL Findings 2025.
    PDF Poster

  4. Image IMOL Workshop
    SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling
    Loris Gaven, Clément Romac, Thomas Carta, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    IMOL Workshop, NeurIPS 2024.
    PDF Poster

  5. Image ARLET Workshop
    Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent
    Quentin Gallouédec, Edward Beeching, Clément Romac, Emmanuel Dellandrea
    ARLET Workshop, ICML 2024.
    PDF Code Project Page Poster

  6. Image ICML
    Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
    Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2023.
    PDF Code Project Page Poster

  7. Image ICML
    TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
    Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2021.
    PDF Code Project Page Poster

  8. Image
    Meta Automatic Curriculum Learning
    Rémy Portelas, Clément Romac, Katja Hofmann, Pierre-Yves Oudeyer
    Preprint, 2020
    PDF Code Project Page

  9. Image
    Deep Recurrent Q-Learning vs Deep Q-Learning on a simple Partially Observable Markov Decision Process with Minecraft
    Clément Romac, Vincent Beraud
    Preprint, 2019
    PDF Code

Projects

  1. Image
    ChatGPT explained in 5 minutes
    A series of short videos explaining Large Language Models.
    Project Page

  2. Image
    Lamorel
    Lamorel is a Python library designed for RL practitioners eager to use Large Language Models (LLMs).
    Code

  3. Image
    Automated Discovery Tool
    A software for assisted and automated discovery of patterns in the exploration of complex systems.
    Code

  4. Image
    Interactive Deep RL Demo
    In-browser interactive demo of Deep Reinforcement Learning agents' adaptation to unknown 2D tasks.
    Project Page Code

Other stuff


Powered by Jekyll and Minimal Light theme.