What if we could skip that retraining altogether?

That’s the central idea behind my recent work, “Keep the Alignment, Skip the Overhead: Lightweight Instruction Alignment for Continually Trained LLMs.” This research introduces a simple yet powerful mechanism—Instruction Residuals—to retain and restore instruction-following behavior in LLMs without full instruction fine-tuning.

This work has been accepted at the ICML 2025 Workshop on Test-Time Adaptation: Putting Updates to the Test!

The Core Idea: Instruction Residuals

Instead of re-aligning the entire model after every update, we extract the instruction-following difference between a base LLM and its instruction-tuned version. This instruction residual acts like a plug-and-play adapter.

Think of it as a “cheat sheet” of instruction-following skills that you can add back to your model after it learns new knowledge.

• Plug-and-play alignment: After updating your base model with new knowledge, simply add the residual to recover instruction capabilities.
• Compute-efficient: Avoid repeating the entire instruction tuning process.
• Modular and reusable: Works across architectures and model versions.

Why This Matters

• Continual Pretraining is risky: Updating even a well-aligned LLM can degrade instruction-following ability by up to 10 points.
• Instruction Residuals fix this: Restore—and often improve—instruction behavior with zero extra tuning.
• Architecture-agnostic: Works across model families like LLaMa and Qwen.
• Industry adoption: Already in use by:
  • DeepMind’s GAIA project within the GemmaVerse ecosystem
  • CEIA-UFG’s Gemma-3-Gaia-PT-BR-4b-it multilingual model on HuggingFace

Highlights from Our Findings

• Restores instruction-following: After 1B tokens of continual pretraining, instruction residuals restore performance nearly to original levels, sometimes even exceeding them.
• Cross-model portability: Residuals from LLaMa 3.1 can improve instruction behavior in LLaMa 3—showing backward compatibility.
• Generalizes to domain-tuned models: Applied to domain-specific models like LLaMa-DocChat, instruction residuals boost instruction benchmarks by up to 6 points without harming domain QA ability.

• 2000× Less Compute, Competitive Performance: A detailed analysis revealed that traditional instruction tuning demands nearly 2000× more FLOPs than our residual approach. With instruction residuals, we deliver competitive accuracy on benchmarks like MMLU, IFEval, and GSM8K at a fraction of the computational cost.

What’s Next?

This is just the beginning. Ongoing work focuses on:

• Extending instruction residuals to smaller models (≤1.5B) without quality loss.
• Generalizing across model architectures like Mistral, Mixtral, and future Qwen variants.
• Exploring residual approximation techniques when both the base and instruction-tuned models are not available.

Reducing instruction tuning costs could democratize LLM alignment, making advanced models more accessible to researchers and smaller organizations.

🙏 Acknowledgments

Thanks to my collaborators at Samsung R&D Institute India, and the broader research community for supporting this work. The momentum we’ve gained—both in academia and industry—has been incredible.