Selected Publications

LoLCATs: On Low-Rank Linearizing of Large Language Models 2025
Michael Zhang, Simran Arora, Rahul Chalamala , Benjamin Spector, Alan Wu, Krithik Ramesh, Aaryan Singhal, Christopher Ré
ICLR
LoLCATs (Low-rank Linear Conversion via Attention Transfer) converts softmax attention into linear attention using low-rank adaptation so open-source LLMs generate in linear time and constant memory without retraining from scratch.
arXiv Together blog Hazy blog p1 Hazy blog p2 code
RedPajama: an Open Dataset for Training Large Language Models 2024
Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala , Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
NeurIPS
Spotlight Presentation
RedPajama-V2 compiles over 100B documents across 84 CommonCrawl snapshots with quality signals and deduplicated subsets to support open LLM training.
arXiv Together blog code
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models 2023
Kaiyu Yang, Aidan Swope, Alex Gu, Rahul Chalamala , Peiyang Song, Shixing Yu, Saad Godil, Ryan Prenger, Anima Anandkumar
NeurIPS
Oral Presentation
LeanDojo provides data, models, and tooling for retrieval-augmented theorem proving in Lean, including the ReProver system and a benchmark of nearly 99k theorems.
arXiv project code
Spectrum Safety: Compatibility of NTS-3 Signals with GNSS Signals 2022
Rahul Chalamala , Joanna Hinks
ION Joint Navigation Conference
Oral Presentation
Evaluated spectral separation coefficients to confirm NTS-3 satellite signals do not interfere with GPS and outlined mitigation strategies for edge cases.
abstract