KV cache is all the rage. Maybe we should just focus on V cache…
Need different model weights? Just change them 🙃
Our new paper VFold does exactly that to enable low-overhead cross-layer value cache compression: half the value cache, >98% of full-cache performance 🧵
📄 arxiv.org/abs/2610.12338
