Grok+1

Ever found yourself zoned out or distracted from a 3 hour podcast on AI or that lecture that you really wanted to learn something from? Say no more!

Comment

Inspiration

Podcasts, lectures, meetings—you spend hours with them, pausing to jot notes or drifting into rabbit holes, and end up missing the best moments. The stuff that should’ve stayed in your head just… doesn’t.

I wanted something that sits with me while I'm listening to this stuff, something that I pester constantly with my questions (like I would to my friend), something that really picks up on what I care about, and something remembers the things future-me will actually need.

What it does

Grok+1 is a listen-along companion, that you can just turn-on and forget. Its like your friend whom you can just interrupt midway and ask annoying questions about things you don't understand. Actually, it might be better than your friend, because as you listen, this one takes live notes about the things you might be interested in based on the stuff you explore on X. And also suggests relevant tweets you might want to look at and bookmark!

Best part, you can always come back and ask grok any questions about a particular recording.

How we built it

Currently, the MVP is an electron-based desktop app that continuously streams audio to Grok voice via WebSocket to a FastAPI backend. The backend brokers Grok Voice real-time transcription which keeps appending to storage. Additionally, we allow the user to interrupt and ask question mid-way while the recording is in progress. This uses the Grok language models to answer the questions and use TTS to respond back to the user.

The app uses X OAuth integration to authenticate you and build an understanding of your recent topics of interest on X.

At certain intervals, Grok will analyze the recent chunks of the conversation and based off of your interests on X, jot down notes and suggest tweets which are sent to the client via the WebSocket connection for real-time discovery.

Challenges we ran into

  • Working with audio signals is quite tricky, especially when they're streamed and there's scope for multiple sources i.e firstly the external audio and the secondly the user interrupting in mid-way to ask questions.

  • I initially started out with building a chrome extension, which did not pan out so well due to constant tussle with microphone permission issues on chrome. Although, this led to me thinking if a desktop app is more convenient especially when built using electron.

Accomplishments that we're proud of

Working with Grok Voice APIs and building something I've always wanted for myself. And something that I can quite literally go home and use it today !!

What we learned

  • Working with Grok APIs, which I've done before.
  • Dealing with audio stream processing.
  • Gain mastery on prompt engineering :)

What's next for Grok+1

  • Add multi-speaker diarization instead of having the user manually signal when they're about to speak.
  • Expand integrations beyond just X
  • Use the mobile phone as the primary surface, since its much more handy.

Built With

Share this project:

Updates