Jeremy Cohen

@deepcohen

Research fellow at Flatiron Institute, working on understanding optimization in deep learning. Previously: PhD in machine learning at Carnegie Mellon.

New York, NY

jmcohen.github.io

Joined September 2011

Replies Reposts Media

Pinned
Jeremy Cohen
@deepcohen
Oct 1, 2025
Replying to @deepcohen
Part 1: How does gradient descent work? centralflows.github.io/part1/ Part 2: A simple adaptive optimizer centralflows.github.io/part2/ Part 3: How does RMSProp work? centralflows.github.io/part3/
centralflows.github.io
How does gradient descent work?
1
Jeremy Cohen
@deepcohen
Jun 12
Did Anthropic get more gains out of model scaling than other labs thought was possible? It reminds me of an interesting recent paper, which showed that deep layers in open LLMs are not doing much, and that this can be fixed by scaling the LayerNorm output.
arxiv.org
The Curse of Depth in Large Language Models
In this paper, we introduce the Curse of Depth, a concept that highlights, explains, and addresses the recent observation in modern Large Language Models (LLMs) where nearly half of the layers are...
9
Jeremy Cohen
@deepcohen
Jun 8
The recent Microsoft AI report noted that too much learning rate decay during pretraining hurts post-RL performance. This is actually just the latest of several papers this year pointing out that small learning rates can be harmful in LLM pretraining. (Thread)
5
Jeremy Cohen
@deepcohen
Apr 22
Looking forward to attending ICLR and giving a talk on Sunday at 9am at the Science of Deep Learning workshop: scienceofdlworkshop.github.io/2026/. Message me if you want to chat about deep learning optimizer dynamics at the conference!
3
Jeremy Cohen
@deepcohen
Feb 9
Isn’t it a little ironic that this argument for why LLMs aren’t truly intelligent is based on … pattern matching?
Big Brain AI
@realBigBrainAI
Feb 8
AMI Labs founder Yann LeCun on why LLMs are fooling us the same way AI has for decades: He argues that every generation of AI scientists has made the same mistake: confusing task performance with real intelligence. LeCun's core challenge to the current hype: "We're fooled into
00:00
1