100% open source framework for realtime voice and multimodal AI. Maintained by @trydaily engineering team with support from the Pipecat developer community.
Phonon: a new open, on-device, text-to-speech model from Gradium. It's really good.
Click through, below, and you can test it in your web browser (enabled by an optimized WebGPU inference stack). Go do it, just to *feel* what it's like when time-to-first-byte is ~10ms.
There
A new frontier for on-device speech.
We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date.
Phonon-31m is available under Apache 2.0.
We have some fun voice AI stuff to talk about on the @NVIDIAAI Nemotron Labs livestream tomorrow: fine-tuning audio and LLM models for specific agents and use cases. Join us live if you want to chat and ask questions. 10:00am Pacific Time on Tuesday.
Sometimes, choosing a voice for your Pipecat agent is about more than just latency. It might be about personality, accent, or how convincingly a voice fits the situation.
We explore voice realism with @usebland Speech v3, available in Pipecat as a maintained service.
📖
🔥 New @SarvamAI STT release — Saaras V4, with Pipecat support on launch day. 22 Indian languages + English. 10 of its Indian languages have no commercial alternative.
Watch @VinayakGavariya@SarvamForDevs demo key features
Saaras V4 is here.
Built for the messy reality of speech, accents, dialects, noise, and mixed-language conversations.
→ 5 modes: Transcribe, Translate, Transliterate, Codemix, Verbatim
→ Keyterm prompting: Guide the model toward the words that matter
→ 22 Indian languages +
17h
