This was a great collaboration! Check it out if you are training models in memory constrained settings 🚀
Ever wanted to train your own 13B Llama2 model from scratch on a 24GB GPU? Or fine-tune one without compromising performance compared to full training? 🦙
You now can, with LoQT: Low Rank Adapters for Quantized Training! arxiv.org/abs/2405.16528
1/4
