Polyglo

Most creators ship in 2 languages. The other 18 die in QA hell. Polyglo fixes that one story, auto-illustrated, narrated, verified, and published across 20 languages. No human listeners required.

Comment

Inspiration

Making content in 20 languages sounds exciting until QA becomes the nightmare. Someone has to listen to hundreds of audio clips to catch robotic glitches and mispronunciations. We built Polyglo to automate that entire verification loop so creators can ship in 20 languages as easily as two.

What it does

Polyglo takes one story and fans it out across multiple languages illustrated scenes, narrated audio, and ready-to-post 9:16 vertical videos (Reels, TikToks, Shorts).

  • One story, many languages: Illustrations are generated once and shared across all languages via SHA-256 content hashing on Backblaze B2. Only audio differs per language, cutting costs by 37–90%.
  • Self-Healing QA Gate: After generating audio, Polyglo transcribes it back and checks the Word Error Rate. Bad takes are retried automatically on a fallback voice only verified audio ships.
  • Short-Form Video Engine: Scenes become 9:16 motion videos with Ken Burns zoom, ambient music, and burned-in subtitles ready to post.
  • Live Telemetry: Every pipeline event logs to a Parquet lake on B2, queried live with DuckDB for dedup ratios, latency, and retry stats.
  • Cryptographic Provenance: Every asset has a SHA-256 Genblaze manifest, verifiable live at /verify.

How we built it

Python + FastAPI + htmx frontend, anchored by Genblaze and Backblaze B2.

  • B2 Storage: Content-addressed blobs via boto3 S3 API with Object Lock and SSE-B2 encryption. SQLite snapshots go straight to B2 so data survives Render restarts.
  • Chat & Authoring: Llama 3.1 via NVIDIA NIM + OpenRouter (Nemotron 120B) for CEFR-level scene splitting and multilingual translation.
  • Images: FLUX.1-dev (NVIDIA NIM) and Seedream 4.5 (OpenRouter) with reference conditioning for cross-scene character consistency.
  • Audio: Mistral Voxtral Mini TTS via OpenRouter, with Gemini 2.0 Flash ASR as the independent QA verifier.
  • Video: imageio-ffmpeg with zoompan motion filters and drawtext subtitle rendering.

Challenges we ran into

  • Character Consistency: Early runs gave us a different-looking character every scene. Fixed by anchoring a style_guide prompt at authoring time and seeding image generation deterministically per story.
  • NVIDIA Silent Refusals: NVIDIA returns a solid black JPEG with HTTP 200 when it moderates a prompt. We built a pixel variance checker to detect flat images and retry with a softened prompt automatically.
  • Windows File Locks: DuckDB held Parquet files open during telemetry writes on Windows. Solved with gc.collect() retry fallbacks.

Accomplishments that we're proud of

  • 37.5%+ real cost savings confirmed live: 32 asset references compressed to 20 unique B2 blobs across 4 scenes × 4 locales. Savings scale to 90% as more languages are added.
  • Self-healing reliability there's a live Chaos Toggle on the dashboard. Disable any model mid-run and the fallback chain recovers without a single 500 error.
  • 494 tests, all green every provider, pipeline step, and edge case covered with exit code 0.

What we learned

QA has to be a blocking step, not an afterthought. Cross-modal verification (TTS → ASR → WER diff) is what lets you actually trust automated content at scale it completely changes how confident you can be when deploying AI media to real users.

What's next for Polyglo

  • Multi-speaker voice cloning for dialogue-driven stories with distinct character voices.
  • Kinetic word-highlight subtitles timed to ASR timestamps (Alex Hormozi-style animated captions).
  • Direct publishing to YouTube Shorts, TikTok, and Instagram APIs from the dashboard.

Built With

  • backblaze-b2
  • duckdb
  • fastapi
  • ffmpeg
  • flux-1-dev
  • gemini-api
  • genblaze
  • htmx
  • nvidia-nim
  • openrouter
  • parquet
  • python
  • s3
Share this project:

Updates