Great work led by Jiale! NoRA builds on LoRA’s parameterization, with the down-projection matrix normalized. Even more surprisingly, applying this normalization only at initialization is already sufficient to obtain considerable performance gains.
More details in
1/ How can we make Low-Rank Matrices behave more like Full-Rank Matrices—and achieve higher rank efficiency?
We release our new paper: NoRA: Normalized Low-Rank Adaptation.
Paper: arxiv.org/abs/2608.31036
