Kairong Luo

@openhonor

PhD Student @ Tsinghua University | Researching LLM

thu-yao-01-luo.github.io

Joined February 2025

Replies Reposts Media

Kairong Luo
@openhonor
22h
How much would it cost you to pretrain a 2B LLM from scratch? $1M? $100K? 🚀Announcing Puro-2B, with a fully open recipe. You can train a model that matches Qwen2-1.5B on RTX 5090s for less than $5090! $4.4K → beats Qwen2-1.5B $6.9K → approaches Qwen2.5-1.5B Check here👇
00:00
Kairong Luo
@openhonor
Apr 23
✈️ Heading to ICLR 🇧🇷 Apr 22–27. Come to our oral on Fri, Apr 24 (10:30 AM–12:00 PM, Room 202 A/B) or find me at our poster (3:15 PM–5:45 PM, P3-#521). We study why LR decay can hurt curriculum-based LLM pretraining — and how to fix it. Happy to chat!
Kairong Luo
@openhonor
Dec 10, 2025
🚀 Announcing PCMind-2.1-Kaiyuan-2B A new frontier for fully open-source models. Not just weights—full pretraining pipeline & recipe. Specs: 2B params, 2.2T tokens Approach: data-centric pretraining Status: SOTA among fully-open models 🤗 HF: huggingface.co/thu-pacman/PCM…
Kairong Luo
@openhonor
Apr 25, 2025
📢 Come meet us at #ICLR2025! We'll be presenting our Multi-Power Law — a new approach to predicting full pretraining loss curves across LR schedules — during the poster session: 🗓 Friday, April 25 🕒 3:00 PM – 5:30 PM CST 📍 Hall 3 + Hall 2B, Poster #237 Expect your feedback!
Kairong Luo
@openhonor
Mar 18, 2025
🔍How does pretraining loss evolve under different LR schedules? 🌟Meet our Multi-Power Law: predicts the full loss curve for various schedules! 🌟Accurate enough to optimize LR schedules directly. 🌟Result? A WSD-like schedule that outperforms the rest! 🔥Accepted at #ICLR2025
Kairong Luo
@openhonor
Apr 24, 2025
📢 Come meet us at #ICLR2025! We'll be presenting our Multi-Power Law — a new approach to predicting full pretraining loss curves across LR schedules — during the poster session: 🗓 Friday, April 25 🕒 3:00 PM – 5:30 PM CST 📍 Hall 3 + Hall 2B, Poster #237 Expect your feedback!