to prepare for an upcoming release (👀) we dug into the pretokenizer a little more. are people aware that hand-crafted regexes are part of every frontier LLM and doing some pretty heavy lifting?
some quick examples 🧵
transformer inference has had dozens of tricks optimized by the entire ecosystem, there are still many ideas and optimizations waiting to be discovered for alternative architectures!
ReplaySSM now runs in production in GLM-5.3-Flash and Kimi-K3, and ships in TensorRT-LLM, vLLM, FlashInfer, and SGLang!
It speeds up standard decoding (up to 1.47x) and speculative decoding (up to 2.1x), and now also enables a low-precision SSM state whose rounding error never
really excited to see an application built on top of Hydra (bidirectional Mamba arxiv.org/abs/2407.09941)!
at small scales we had done some DNA language modeling and found evidence that Mamba is better than Transformers and convolutions; this paper from a serious bio group shows
This week we're releasing a preprint exploring state space models for long-sequence genomic prediction, alongside Cerberus, a set of open PyTorch models built on this work. biorxiv.org/content/10.648…
Introducing Multilingual Voices
We partnered with Baklavastory, our neighbor in the Mission, to show what it sounds like when a business's character and warmth stay consistent, no matter who calls or what language they speak.
It's easy to translate speech, but much harder to
