Ahmad Beirami ✈️ ICML (@abeirami) / X

Ahmad Beirami ✈️ ICML

4,836 posts

Ahmad Beirami ✈️ ICML

@abeirami

stealth // ex Gemini RL+Inference @GoogleDeepMind // Chat AI @AIatMeta // RL Agents @EA // ML+Information Theory @MIT+@Harvard+@GeorgiaTech

{NYC, SFO, YYZ}

Joined December 2018

Replies Articles Media

Pinned
Ahmad Beirami ✈️ ICML
@abeirami
Jul 4
I've been saying for a while that aggregate benchmark scores often hide what actually matters. When building and evaluating a model, an agent, or a system, a few things are crucial: 1. Evaluation rubrics should be derived and stated unambiguously from the spec or policies. Any
Ahmad Beirami ✈️ ICML
@abeirami
Oct 21, 2025
They should have broken the 10k to 10-100 stacks of $100-1k and given them to the identical copies of the same model to be able to see anything remotely meaningful. Right now we are looking at noise!
This post is unavailable.
Ahmad Beirami ✈️ ICML
@abeirami
Aug 25, 2025
What are the founders going to own? 🤔
Ahmad Beirami ✈️ ICML
@abeirami
Aug 25, 2025
There is no free lunch here. Pruning hurts robustness and capabilities other than those captured by the success metric used to prune.
Avi Chawla
@_avichawla
Aug 25, 2025
I removed 74% of neurons from a neural network. It dropped the accuracy by just 0.50%. Here's a breakdown (with code):
Ahmad Beirami ✈️ ICML
@abeirami
Jun 2, 2025
After three incredible years, today is my last day at Google DeepMind! I am truly grateful to the amazing colleagues who made the journey 1000x more fruitful and enjoyable! I am forever indebted to my collaborators who showed me how to be better at everything via demonstrations.
Ahmad Beirami ✈️ ICML
@abeirami
Sep 10, 2025
This is a great example of what good research looks like. You start with a real problem. You peel it layer by layer to find the root cause. You form a new hypothesis and keep digging. At the end, you have something insightful to share!
Thinking Machines
@thinkymachines
Sep 10, 2025
Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference” We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to
Ahmad Beirami ✈️ ICML
@abeirami
Aug 12, 2025
The best AI researchers zoom at three abstraction levels: - High: paper-level ideas & math - Mid: code-level implementation - Low: GPU/TPU reality (kernels/memory) Low exposes bottlenecks. High accelerates exploration. Mid makes it real. The job is to translate between them!
Ahmad Beirami ✈️ ICML
@abeirami
Aug 9, 2025
The main ingredient that led to GRPO's performance leap is the calibration of the reward/value via multiple rollouts per prompt. Let me elaborate on what I mean by that and a cheaper way of doing it offline.
Ahmad Beirami ✈️ ICML
@abeirami
May 27, 2025
As we go through a lot of excitement about RL recently with lots of cool work/results, here is a reminder that RL with a reverse KL-regularizer to the base model cannot learn new skills that were not already present in the base model. It can only amplify the existing weak skills.
Ahmad Beirami ✈️ ICML
@abeirami
Feb 4, 2025
A very nice blogpost on GRPO (the method that was used to train R1) by Youssef Mroueh
Ahmad Beirami ✈️ ICML
@abeirami
Aug 10, 2025
Post-training research was fueled by the KL-regularized RL mathematical foundation. That led to a lot of algorithmic research and a ton of progress over a few years. This helped us learn how to "distill" metrics back into models. But today we are optimizing workflows/agents.
Ahmad Beirami ✈️ ICML
@abeirami
Sep 6, 2025
Unpopular opinion: When a paper has a senior mentor and a junior mentee, the senior author must make sure the claims are correct and well supported. They must check every claim and gate the submission until it meets that bar. The junior author is the generator. The senior author
Ahmad Beirami ✈️ ICML
@abeirami
Apr 13, 2022
The question that a reviewer should ask themselves is: Does this paper take a gradient step in the right direction? Is the community better off with this paper published? If the answer is yes, then the recommendation should be to accept.
Ahmad Beirami ✈️ ICML
@abeirami
Oct 4, 2025
This post should have been titled "Finetuning without regret" Finetuning models for specific narrow use cases to save costs, and to keep data on prem during inference, makes sense in general. However, that is bottlenecked by two main issues IMO: 1. Today, OSS models are lacking
Thinking Machines
@thinkymachines
Oct 1, 2025
Introducing Tinker: a flexible API for fine-tuning language models. Write training loops in Python on your laptop; we'll run them on distributed GPUs. Private beta starts today. We can't wait to see what researchers and developers build with cutting-edge open models!