Hongyi Wang

@HongyiWang10

Assist. Prof. @RutgersCS; Head of Infra @genbioai; OSS LLMs @IFM_AI; Ex @mldcmu @WisconsinCS

Piscataway, NJ

hwang595.github.io

Joined August 2019

2,032 Following

2,560 Followers

Replies Reposts Media

Pinned
Hongyi Wang
@HongyiWang10
Sep 3
Really proud of our K2 Horizon release! One message I hope people take from this work is that truly OSS models matter, and “open” does not have to mean weak. True openness should go far beyond downloadable weights. It should include the training data, code, recipes, and
Institute of Foundation Models
@IFM_AI
Sep 3
Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters. - Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B and 7B models setting
5
Hongyi Wang
@HongyiWang10
Oct 4
Proud of my @RutgersCS undergrad @Chen94751623484 for this work! 🎓🔥 No NaNs ≠ correct gradients. Even after fixing the visible training instability, large errors can hide in the attention backward. The culprit: BF16 rounding breaks a zero-sum conservation law. A small
Junlin Chen
@Chen94751623484
Oct 4
We trained an LLM with FlashAttention-3 in BF16 🔥 Training looked healthy for a long time. Then the grad norm shot up 1000× 📈 and the loss ended 0.2 nats above FP32 attention. Not a single NaN 😶 The cause is in the attention backward 🧵 📄 arxiv.org/abs/2609.34272
4
Hongyi Wang
@HongyiWang10
Oct 2
An exciting use case for #K2Horizon, @IFM_AI’s open-source LLMs! 🚀 Big decisions don’t always need big models. In the demo below, our team shows how Jev, powered by a 0.9B language model, combines efficiency, effectiveness, and fast responses. Huge congrats to my student
Junlin Chen
@Chen94751623484
Oct 2
Most decisions don't need words. 🤫⚡ Meet K2-Type-0.9B 🧠: an open decision model. One state + many typed questions in, calibrated probabilities out, in a single forward pass. No text generated. Apache-2.0 weights 👇 huggingface.co/IFM/K2-Type-0.… #OpenSource #LLM #AI
Hongyi Wang
@HongyiWang10
Sep 4
AR quality, diffusion speed, without compromise. Uno adds diffusion weights for parallel drafting, then uses AR verification to preserve the AR distribution. No separate draft model, with gains across tested batch sizes and up to 3× over base AR. Congrats @ssahoo_ and team,
Subham Sahoo
@ssahoo_
Sep 4
Diffusion LLMs have two limitations relative to AR models: (1) Lower quality, and (2) Slower inference at large batch sizes. We address this "Uno" > Retains the AR architecture of LLMs > Each layer has two sets of weights: AR weights and Diffusion weights > Diffusion
2
Hongyi Wang
@HongyiWang10
Sep 3
Another major milestone for fully open-source models! Yes, you can have openness, transparency, and frontier-level capabilities all at once!
Artificial Analysis
@ArtificialAnlys
Sep 3
K2 Horizon 375B A23B, a new open weights model from UAE's MBZUAI, scores 47 on the Artificial Analysis Intelligence Index, with relatively strong agentic performance and a 30 point jump over its predecessor K2 Horizon 375B A23B is an open weights Mixture-of-Experts model with
2

Hongyi Wang

Hongyi Wang’s posts