📣 We've just published our latest research on state-of-the-art speculative decoding in interactivity.
Introducing a new approach that achieves a 4.37× speedup over autoregressive decoding in small batch setting and outperforms the strongest tuned DFlash baseline by 24.7%.
