1. X
  2. Alexandre Ramé
Log inSign up
Alexandre Ramé
906 posts
Image
user avatar
Alexandre Ramé
@ramealexandre
Senior research scientist @GoogleDeepMind. Previously PhD @Sorbonne_Univ_. Post-training Gemma LLMs: distillation, RL and merging.
alexrame.github.io
Joined May 2011
790
Following
1,997
Followers
RepliesRepliesMediaMedia

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up
  • Pinned
    user avatar
    Alexandre Ramé
    @ramealexandre
    Mar 12, 2025
    Welcome Gemma 3, our new open-weight LLM from @GoogleDeepMind. All sizes (1B, 4B, 12B and 27B) excel on benchmarks, but the key result may be the 27B reaching 1338 on LMSYS. For this, we scaled post-training, with our novel distillation, RL and merging strategies. Happy building!
    Image
    21K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Jan 23, 2024
    Introducing DeepMind's Weight Averaged Reward Model (WARM) for alignment via RLHF! We merge multiple reward models into one that's more reliable and robust. WARM efficiently captures the best of each to mitigate reward hacking. A thread 🧵 below.
    arXiv logo
    arxiv.org
    WARM: On the Benefits of Weight Averaged Reward Models
    Aligning large language models (LLMs) with human preferences through reinforcement learning (RLHF) can lead to reward hacking, where LLMs exploit failures in the reward model (RM) to achieve...
    75K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Sep 2, 2024
    Usually not sharing personal things on X, but I’m making an exception for my fantastic wedding this weekend, and spread my happiness and love with you all.
    Image
    25K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Jun 25, 2024
    Introducing Weight Averaged Rewarded Policies (WARP), Google DeepMind's latest RLHF alignment method using the magic of model merging. By scaling alignment like pre-training was scaled, WARP learns sota Gemma LLM surpassing previous releases. A 🧵below.
    arXiv logo
    arxiv.org
    WARP: On the Benefits of Weight Averaged Rewarded Policies
    Reinforcement learning from human feedback (RLHF) aligns large language models (LLMs) by encouraging their generations to have high rewards, using a reward model trained on human preferences. To...
    36K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Sep 9, 2021
    Surprised #deeplearning methods were not massively used to detect Covid? Some tried, but most failed: e.g. they rely on patients' age instead of 'truly' analyzing the medical scans. This is because networks learn correlation and not causation. Our preprint Fishr tackles this. 1/4
    Image
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Mar 29, 2024
    Devoured all papers related to model merging & rarely does anything shine like "Model Stock" arxiv.org/abs/2403.19522. Clear findings, wise insights and splendid figures. For anyone merging models or interested in finetuning, this is your must-read. TLDR; interpolate towards init.
    arXiv logo
    arxiv.org
    Model Stock: All we need is just a few fine-tuned models
    This paper introduces an efficient fine-tuning method for large pre-trained models, offering strong in-distribution (ID) and out-of-distribution (OOD) performance. Breaking away from traditional...
    22K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Oct 10, 2024
    An AI will win a Nobel price someday✨. Yet currently, alignment reduces creativity. Our new @GoogleDeepMind paper "diversity-rewarded CFG distillation" improves quality AND diversity for music, via distillation of test-time compute, RL with a diversity reward, and model merging.
    arXiv logo
    arxiv.org
    Diversity-Rewarded CFG Distillation
    Generative models are transforming creative domains such as music generation, with inference-time strategies like Classifier-Free Guidance (CFG) playing a crucial role. However, CFG doubles...
    32K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Jun 8, 2023
    Human-aligned AI is a multi-objective problem. Yet, current RLHF prioritizes certain values when aligning LLMs, resulting in a lack of transparency and unfair representation of minorities. In our latest paper arxiv.org/abs/2306.04488), we embrace the diversity of human values.
    41K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Mar 4, 2024
    Today was my first day as a Research Scientist at @GoogleDeepMind ! 🚀 Feeling both proud and excited about the learning and collaboration ahead. My main hope: to help better align AI models with the world, in all its diversity.
    22K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Jul 26, 2024
    Replying to @jm_alexia
    EMA also helps with LLMs: e.g., was used during the pretraining of Llama 3.1.
    Image
    3.5K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Dec 21, 2022
    Ready to give your deep models a second life? Introducing model ♻️ recycling (arxiv.org/abs/2212.10445), improving generalization by reusing weights fine-tuned on various vision tasks. Just like you recycle your bottles and cardboards, it's time to start recycling your models too!
    14K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Oct 10, 2023
    Exciting news 🎓! I'm defending my PhD on "Diverse & Efficient Ensembling of Deep Networks" tomorrow at 13h30 CEST. If you're in Paris and can join, DM me. Or catch the live stream on YouTube: youtube.com/watch?v=DTD7qt…. Wish me luck!
    Image
    8.8K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Aug 1, 2023
    Congrats @MustafaShukor1 for this super work, extending multimodality to academic budgets, and the recycling ideas from my previous ratatouille. We confirm that multimodal weights finetuned on diverse image/video/language tasks (captioning, VQA, etc) can be linearly interpolated.
    user avatar
    AK
    @_akhaliq
    Aug 1, 2023
    Unified Model for Image, Video, Audio and Language Tasks paper page: huggingface.co/papers/2307.16… Large Language Models (LLMs) have made the ambitious quest for generalist agents significantly far from being a fantasy. A key hurdle for building such general models is the diversity
    Image
    GIF
    21K
  • user avatar
    Alexandre Ramé
    @ramealexandre
    Jul 25, 2024
    Presenting WARM (Weight Averaged Reward Models) poster #1205 at @icmlconf right now, where we merge multiple reward models for more efficient, reliable and robust RLHF. Crazy recent news was WARM actually being used in LLama 3.1! See you there.
    Image
    user avatar
    Alexandre Ramé
    @ramealexandre
    Jan 23, 2024
    Introducing DeepMind's Weight Averaged Reward Model (WARM) for alignment via RLHF! We merge multiple reward models into one that's more reliable and robust. WARM efficiently captures the best of each to mitigate reward hacking. A thread 🧵 below. arxiv.org/abs/2401.12187
    3.4K
Advertisement
Advertisement