Ruisi Cai (@ccccrs

Ruisi Cai

27 posts

Ruisi Cai

@ccccrs_0908

Ph.D. student @UTAustin; Research Intern @NVIDIA @CitadelSecurities; BS @USTC; NVIDIA fellowship 2025 recipient

Joined May 2020

Ruisi Cai
@ccccrs_0908
Dec 18, 2024
Excited to share that I have been awarded NVIDIA fellowship! 🎉 Immensely grateful for the recognition and support - this inspires me to continue advancing research in LLM efficiency and AI security.
NVIDIA Awards up to $60,000 Research Fellowships to PhD Students
From blogs.nvidia.com
Ruisi Cai
@ccccrs_0908
Jun 11, 2024
Managing long context is challenging due to quadratic attention memory usage. But what if we could compress growing context information into a fixed-size memory? 🤔 Check out our new ICML paper: "LoCoCo: Dropping In Convolutions for Long Context Compression"! 1/3
Ruisi Cai
@ccccrs_0908
Jun 18, 2024
Tired of training varying-size LLMs to fit various GPU memory and latency requirements? Check out Flextron! Our new ICML (Oral) paper shows how to train one model deployable across GPU series. Learn more: cairuisi.github.io/Flextron/🚀
Ruisi Cai
@ccccrs_0908
Mar 14, 2023
Thrilled to share our latest research project on model merging, now available at arxiv.org/pdf/2302.12480…! Our finds suggest a surprisingly simple way to disentangle "robustness" encoded in the robust model weights, which can be "zero-shot" transferred to some other models.
Ruisi Cai
@ccccrs_0908
Oct 10, 2024
Exciting to see flexible inference being explored on the mamba architecture! Our recent work Flextron tackles similar challenges. Looking forward to seeing how these approaches complement each other! 🚀
Abhinav Shukla
@Abhinav95_
Oct 10, 2024
Announcing MatMamba - an elastic Mamba2🐍architecture with🪆Matryoshka-style training and adaptive inference. Train a single elastic model, get 100s of nested submodels for free! Paper: sca.fo/mmpaper Code: sca.fo/mmcode 🧵(1/10)
Ruisi Cai
@ccccrs_0908
Dec 6, 2024
Layer-wise routers are surprisingly redundant in current MoE. Check out Read-ME for the system-friendly MoE refactorization technique with system co-design!
Yeonju Ro
@j777ro
Dec 5, 2024
(1/n) Do you think token batching in MoE is inefficient? Are you looking for ways to transform pre-trained LLMs into MoEs? Then you should check out Read-ME at NeurIPS'24! 📖 arxiv.org/abs/2410.19123
Ruisi Cai
@ccccrs_0908
Oct 19, 2024
With countless open-source LLM checkpoints available, each specializing in unique domain knowledge, how can we tap into their full potential? Check out Model-GLUE! 🚀 We introduce a framework that integrates model merging, mixture, and stacking to unlock new possibilities.
VITA Group
@VITAGroupUT
Oct 18, 2024
1/ 🌟 Excited to announce #Model-#GLUE (#neurips2024 D&B), a new framework designed by an extensive team from UNC, UMD, UT Austin, HKUST, Google, and CMU to #scale pre-trained LLMs efficiently! 🚀 Tackling the challenge of #aggregating disparate pre-trained LLM, we introduce a
Ruisi Cai
@ccccrs_0908
Jul 16, 2024
Train one - Get many🚀! Check more details about Flextron at cairuisi.github.io/Flextron/
Pavlo Molchanov
@PavloMolchanov
Jul 16, 2024
🚀 Introducing Flextron - a Many-in-One LLM - Oral at ICML! Train one model and get many optimal models for each GPU at inference without any additional retraining. 🌟 🔗 Paper: arxiv.org/abs/2406.10260 Main benefits with only 5% post-training finetuning: ✅ Best model for
GIF
Ruisi Cai
@ccccrs_0908
Jul 27, 2023
Aloha! Third day of #ICML2023, I will present our work: Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights? TL;DR: we unveiled the existence of task vectors in robustness, which can be utilized to inject certain robustness through simple vector arithmetic.
Ruisi Cai
@ccccrs_0908
Jun 11, 2024
Replying to @ccccrs_0908
LoCoCo offers universal compatibility with existing LLM architectures for seamless integration. By injecting convolutional heads, we compressed sequences of up to 3482 tokens into a 128-size KV cache, retaining comparable performance - all with just 104M tokens of tuning! 🚀 2/3
Ruisi Cai
@ccccrs_0908
Jun 18, 2024
Replying to @ccccrs_0908
The Flextron-Llama2-7B model family demonstrates superior MMLU performance compared to both open-source models (including Pythia, OpenLLaMA-v2) and existing post-hoc compression methods (including Sheared-LLaMA, SliceGPT, LLM-Pruner, Compresso, LaCo).
Ruisi Cai
@ccccrs_0908
Jun 18, 2024
Replying to @ccccrs_0908
Flextron optimizes resources with adaptive computation. Using a MoE-like architecture, we route different tokens to different model sizes instead of domain experts. Paper: arxiv.org/pdf/2406.10260
Ruisi Cai
@ccccrs_0908
Jun 18, 2024
Replying to @ccccrs_0908
Grateful to collaborate with @srv_m, Greg Henrich, @yin_hongxu, @VITAGroupUT, @jankautz , and @PavloMolchanov! Excited for more great work ahead! 🙌✨
Ruisi Cai
@ccccrs_0908
Jun 18, 2024
Replying to @ccccrs_0908
In Flextron, we support adaptive model loading: get the best model for every GPU (small and large) without re-training the model. We can dynamically adjust inference speed depending on the GPU load.