Real-Time Talking Coach

RTC is a real-time, privacy-first desktop AI that gives live feedback on how you speak — helping users communicate more clearly during presentations, interviews, and Zoom calls.

Unlike existing tools that analyze recordings after you’re finished, RTC provides feedback while you’re speaking, when it actually matters.

Inspiration

Most people spend years learning what to do — but almost no time learning how to communicate it.

Students present research, engineers pitch ideas, teachers lecture complex material, yet very few are ever trained to speak clearly, confidently, and effectively. Outside of a handful of public speaking classes, most people are expected to “just figure it out.”

The cost of this gap is massive. Industry estimates suggest that communication failures cost companies $1.2 trillion annually, averaging $12,506 per employee. As work, learning, and collaboration increasingly move online, the ability to communicate ideas clearly has become more critical than ever.

Teaching makes this especially clear. Professors and instructors give hour-long lectures that are, in practice, speeches — yet unlike high school educators, many never receive formal training in communication or delivery. The same is true for professionals presenting to teams, clients, or classrooms.

Existing tools don’t help much. Most analyze speech after it’s finished, when the moment to improve has already passed.

We wanted to build something different: a real-time speaking coach that helps you improve as you speak. Whether you’re practicing a presentation at home, lecturing in a classroom, or presenting on Zoom, this tool gives live feedback on your delivery — pacing, clarity, and structure — while letting you read or reference your script naturally. No screen recording. No distractions. Just better speaking, learned through practice.

What it does

RTC is a real-time desktop communication coach that:

Listens to live speech through the microphone
Tracks pacing (words per minute), pauses, and filler words
Displays clear, real-time visual feedback (green / yellow / red indicators)
Surfaces actionable hints like “Slow down”, “Add a pause”, or “Reduce filler words” while you’re speaking
Generates a post-session coaching summary enhanced by AI, highlighting strengths and concrete improvements

RTC works during live presentations, interviews, and Zoom calls — not just practice recordings.

How we built it

RTC was built as a solo project using a systems-first approach:

Rust powers the real-time audio pipeline for performance, safety, and low latency
Tauri packages the app as a cross-platform desktop application
A React + Vite frontend displays live feedback and coaching hints
An event-driven architecture streams audio metrics from Rust to the UI in real time
The Gemini API analyzes session transcripts to generate personalized coaching feedback at the end of each session

All core analysis runs locally, keeping the system responsive and privacy-focused.

Challenges we ran into

Designing a real-time feedback loop without introducing distracting latency
Balancing responsiveness with accuracy for live speech metrics
Learning Rust while building a full desktop application from scratch
Creating feedback that is helpful during a presentation, not overwhelming

Accomplishments that we're proud of

Built the entire system solo
First project using Rust, chosen intentionally for performance and safety
Implemented a live, end-to-end feedback loop from microphone input to UI
Created a clean, demo-ready desktop application that works during real presentations