Xinyu Yang (@Xinyu2ML) / X

Xinyu Yang

979 posts

Xinyu Yang

@Xinyu2ML

Scaling everything @Kimi_Moonshot. Building open frontier intelligence. Prev. PhD @CarnegieMellon. Opinions are my own. Architect. They/Them

Mountain View, CA

Joined December 2022

Pinned
Xinyu Yang
@Xinyu2ML
10h
Kimi K3 is out, and it's beautiful. Go try it now. Also, a (very) late life update: I joined @Kimi_Moonshot a few months ago to build open frontier intelligence. Honored to ship K3 alongside an incredible team. Only the beginning. Scaling never stops. (Btw, please don't sleep
Kimi.ai
@Kimi_Moonshot
10h
Introducing Kimi K3: Open Frontier Intelligence 🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional
Xinyu Yang
@Xinyu2ML
Sep 29, 2025
These days, LoRA seems less prominent in mainstream discussions compared to full FT. However, the post from @thinkymachines highlights that LoRA can actually match full FT in real-world customization scenarios! One year ago, one of my previous works (arxiv.org/pdf/2412.06289)
Thinking Machines
@thinkymachines
Sep 29, 2025
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
Xinyu Yang
@Xinyu2ML
Oct 21, 2025
Honored to receive the 2025 Amazon AI PhD Fellowship! Thank you @amazon for the award!
Carnegie Mellon University
@CarnegieMellon
Oct 21, 2025
Ten CMU doctoral students pursuing AI research will receive support from @amazon through the company's new AI Ph.D. Fellowship Program. The researchers, from @SCSatCMU and @CMUEngineering, are tackling foundational challenges critical to AI innovation. cs.cmu.edu/news/2025/amaz…
Xinyu Yang
@Xinyu2ML
Aug 11, 2025
What’s particularly striking is that 1B unique tokens trained for 96 epochs can match the performance of 96B unique tokens trained for a single epoch. At first glance, this seems counterintuitive. However, if we randomly mask tokens during training, a sequence of length L can
Jinjie Ni
@NiJinjie
Aug 9, 2025
Token crisis: solved. ✅ We pre-trained diffusion language models (DLMs) vs. autoregressive (AR) models from scratch — up to 8B params, 480B tokens, 480 epochs. Findings: > DLMs beat AR when tokens are limited, with >3× data potential. > A 1B DLM trained on just 1B tokens
Xinyu Yang
@Xinyu2ML
Sep 10, 2025
Thank @_akhaliq for introducing our Parallel-R1! 😀 Parallel thinking has recently emerged as a promising direction for enhancing model performance. However, most existing inference-time approaches rely on heuristic rules to aggregate results, without enabling the model to truly
AK
@_akhaliq
Sep 10, 2025
Parallel-R1 Towards Parallel Thinking via Reinforcement Learning
Xinyu Yang
@Xinyu2ML
Oct 21, 2025
🏆Honored to share that LLM.265 (dl.acm.org/doi/10.1145/37…) received the Best Paper Award at MICRO 2025! 🥳Huge thanks to the whole team! 😅Accidentally deleted the original tweet—posting it again
Xinyu Yang
@Xinyu2ML
Jun 16, 2025
🚀 Super excited to share Multiverse! 🏃 It’s been a long journey exploring the space between model design and hardware efficiency. What excites me most is realizing that, beyond optimizing existing models, we can discover better model architectures by embracing system-level
Infini-AI-Lab
@InfiniAILab
Jun 16, 2025
🔥 We introduce Multiverse, a new generative modeling framework for adaptive and lossless parallel generation. 🚀 Multiverse is the first open-source non-AR model to achieve AIME24 and AIME25 scores of 54% and 46% 🌐 Website: multiverse4fm.github.io 🧵 1/n
GIF
Xinyu Yang
@Xinyu2ML
May 2, 2024
Excited to announce the Workshop on Foundation Models in the Wild at @icmlconf 2024 (hybrid workshop). We welcome submissions! Please consider submitting your work here: icml-fm-wild.github.io (deadline: May 31, 2024, AOE) Hope to see you in Vienna or virtually in July,
Xinyu Yang
@Xinyu2ML
Feb 12, 2025
📢 Announcing our new work "APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding" @iclr_conf 🚀 Enabling the efficient combination of multiple contexts with negligible prefilling cost 💅 Re-using the context window of LLMs to accommodate more and
00:00
Xinyu Yang
@Xinyu2ML
Jul 30, 2025
Huge congrats to the NSA authors on clinching ACL 2025 Best Paper! 🏆 We’ve built a community‑driven, third‑party Triton implementation so you can drop NSA straight into your PyTorch pipelines—no from‑scratch coding required. 🔗 Try our 3rd‑party NSA:
ACL 2026
@aclmeeting
Jul 30, 2025
Replying to @aclmeeting
Best Paper (4/4)
Xinyu Yang
@Xinyu2ML
Apr 24, 2025
We will be presenting "APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding", a novel encoding method that enables: 🚀Pre-caching Contexts for Fast Inference 🐍Re-using Positions for Long Context Our poster session is located in Hall 3 and Hall 2B,
00:25
Xinyu Yang
@Xinyu2ML
Feb 12, 2025
📢 Announcing our new work "APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding" @iclr_conf 🚀 Enabling the efficient combination of multiple contexts with negligible prefilling cost 💅 Re-using the context window of LLMs to accommodate more and
Xinyu Yang
@Xinyu2ML
Jan 26, 2025
Replying to @alexandr_wang
If you are confident in the leadership of US, why you mention tightening export controls on chips. The fact is that you are afraid. It is also hard for me to understand a people with last name “Wang” to say these, you can expect USA to do better, but it doesn’t require you to
Xinyu Yang
@Xinyu2ML
Jul 25, 2025
I used to underestimate the importance of prompt engineering. However, after working on Multiverse, I’ve come to realize that the success of LLMs in solving highly challenging tasks is deeply tied to prompt design. For example, generating each training example for Multiverse
Lin Yang
@lyang36
Jul 24, 2025
Code release! 🚀 Following up on our IMO 2025 results with the public LLM Gemini 2.5 Pro — here’s the full pipeline & general (non-problem-specific) prompts. 👉 [github.com/lyang36/IMO25] Have fun exploring! #AI #Math #LLMs #IMO2025
Xinyu Yang
@Xinyu2ML
Sep 18, 2025
🚀 Excited to share that #Multiverse has been accepted to #NeurIPS 2025! Couldn’t have done it without such incredible collaborators—thank you!!
Infini-AI-Lab
@InfiniAILab
Jun 16, 2025
🔥 We introduce Multiverse, a new generative modeling framework for adaptive and lossless parallel generation. 🚀 Multiverse is the first open-source non-AR model to achieve AIME24 and AIME25 scores of 54% and 46% 🌐 Website: multiverse4fm.github.io 🧵 1/n
GIF