Hot / global KV stays in GPU HBM. CSA2 + FP4 compression brings this down to about 890 bytes per token (roughly 1/4 of V4-Flash). This is the part the model needs instantly during decode
Amazing!!!😋
💾 Smaller KV cache. Bigger savings.
Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage
Cache-hit charges often account for a large share of agent costs. Compressing the cache cuts those costs significantly.
3/6
💾 Smaller KV cache. Bigger savings.
Compared with the previous generation, V4.1-Flash’s KV cache needs just:
🔹 1/4 the HBM
🔹 1/8 the SSD storage
Cache-hit charges often account for a large share of agent costs. Compressing the cache cuts those costs significantly.
3/6
Rust target directories are getting smaller! 🦀
Cargo is rolling out a nightly experiment that reduces duplicated metadata in build artifacts.
Size reductions of up to 33% in testing 👇
blog.rust-lang.org/inside-rust/20…#rustlang#cargo#nightly
