Yi Zhang (@YiZhangZZZ) / X

Yi Zhang

60 posts

Yi Zhang

@YiZhangZZZ

plant fruits@meta Prev. @xAI, @Apple, @MSFTResearch, PhD @princeton

Menlo Park, CA

Joined June 2022

Yi Zhang
@YiZhangZZZ
Jul 8, 2025
stay tuned for the smartest AI humanity's ever seen🚀
Elon Musk
@elonmusk
Jul 7, 2025
Grok 4 release livestream on Wednesday at 8pm PT @xai
Yi Zhang
@YiZhangZZZ
Dec 16, 2022
Very nice visualization by @TheGradient of an example from our paper on 'edge of stability' (arxiv.org/abs/2212.07469). Gradient flow gets trapped in sharp local minima, but the discontinuous dynamics of the 'edge of stability' escape them and settle in flat regions.
Hossein Mobahi
@TheGradient
Dec 16, 2022
Replying to @SebastienBubeck
Intriguing work! Not sure if this a coincidence, but plotting the loss in your task (Eq 1.2) reveals a narrow ravine & a wide minimum. The solution that generalizes (i.e. learns threshold neuron) is the flat one. Looks like you cannot realize a threshold neuron at sharper ravine.
Yi Zhang
@YiZhangZZZ
Apr 5, 2025
LLMs are sensitive to changes in prompt style — even the best ones, like GPT-4o. In this work with my amazing intern @VectorZhou, we may have found a cure.
Runlong Zhou
@VectorZhou
Apr 4, 2025
🧠 Ever notice how LLMs struggle with familiar knowledge in unfamiliar formats? Our new paper "CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models" tackles this head-on! 🔍 Our findings: - Created a qualitative pipeline demonstrating problem we call
Yi Zhang
@YiZhangZZZ
Dec 15, 2023
Many thanks @BingbinL! for your amazing work🫡
Bingbin Liu
@BingbinL
Dec 15, 2023
Please come to our poster at the MATH-AI workshop! 😊 (Room 217-219, 3-4pm)
Yi Zhang
@YiZhangZZZ
Mar 24, 2023
Replying to @martin_gorner
Hi Martin, thanks for your great appreciation of this example. Yuanzhi and I are in charge of the coding section. The specific example's credit goes to Yuanzhi, but since he has yet to claim a Twitter account, please allow me to try to answer the questions (I'll do it inline).
Yi Zhang
@YiZhangZZZ
Jul 5, 2022
The GitHub repo for LEGO is finally online! Let us know how well your favorite transformer does on it
Sebastien Bubeck
@SebastienBubeck
Jul 5, 2022
After the paper arxiv.org/abs/2206.04301 & the vid youtube.com/watch?v=brmidg…, here comes the Github repo for LEGO, courtesy of our amazing Yi Zhang @YiZhangZZZ! Lots more can be uncovered abt Transformers with LEGO. Play with it & let us know what you find! github.com/yizhangzzz/tra…
Yi Zhang
@YiZhangZZZ
Jun 7, 2025
Replying to @SebastienBubeck
Congrats Seb!
Yi Zhang
@YiZhangZZZ
Jul 7, 2022
The Scientist Who Developed a New Way to Understand Communication quantamagazine.org/mark-braverman… via @QuantaMagazine
quantamagazine.org
Mark Braverman Wins the IMU Abacus Medal | Quanta Magazine
Mark Braverman has spent his career translating thorny problems into the language of information complexity.
Yi Zhang
@YiZhangZZZ
Dec 16, 2022
Replying to @TheGradient and @SebastienBubeck
That's correct, and nice plot btw! Our interpretation of this example is that the edge of stability leads to a discontinuous trajectory that escapes from the bad local minima (i.e. the narrow ravine) while continuous (or small learning rate) gradient dynamics will get trapped.
Yi Zhang
@YiZhangZZZ
Jul 31, 2022
Replying to @SebastienBubeck
wow, congrats Seb！
Yi Zhang
@YiZhangZZZ
Dec 16, 2022
Replying to @deepcohen @TheGradient and @SebastienBubeck
I mean, the *test loss* is worse for the gradient flow trajectory. More conceptually, from the same initialization, gradient flow does not learn the threshold neuron while EOS does.
Yi Zhang
@YiZhangZZZ
Mar 24, 2023
Replying to @martin_gorner and @erichorvitz
Let me know if I was able to answer your questions above. Followups are welcomed 😃
Yi Zhang
@YiZhangZZZ
Mar 24, 2023
Replying to @martin_gorner
Yuanzhi's idea for this line of prompt is that given a scalar hyper parameter d_dim, and each gradient, which is a tensor of any order, has an axis whose dimension is exactly d_dim. Find that axis ('by looping') and then reshape the gradient to 2d by collapsing all the other axes
Yi Zhang
@YiZhangZZZ
Mar 24, 2023
Replying to @martin_gorner
you are absolutely right here, this is technically an incorrect answer from the model. We will highlight it together with the other mistakes in the paper.