Inspiration

Have you ever been in an awkward social situation? Outright danger? Multitasking discretely? So many situations where we need help getting bailed out without drawing attention to ourselves. The old-school way is to make an excuse to use the bathroom or pocket dial a friend - but this isn't always possible. I want to speak commands discretely, in natural language, and have some message sent to myself or friends without being noticed at all.

For example - on a date, I might want to say, "I wonder if they have cake" - and without it being obvious at all, have it send a message to my friends who know to call me and get me out of that situation, or maybe even pick me up.

After that primary use-case, I realized that the integration here is actually more generally useful - e.g. to speak and send out a reminder to a group of friends about an upcoming event, or kick off a pre-arranged delivery, etc.

What it does

You login with Telegram and link a bot (@OmiSmartBot) to your DM's or any group.

After that link is established, you can manage actions on the website. Actions include prompts and messages. The backend is constantly receiving live transmissions from the Omi, and as soon as it detects a prompt, it sends your message the the destination.

Technically - it operates over Cloudflare Workers (and D1, KV, etc.) and Telegram, for cost-effective massive scalability, but these are implementation details.

UX is a fully hosted application to let you manage all the secret phrases, messages, and delivery options.

As of right now, the site is optimized for desktop - but it works on the phone too. Needs a designer ;)

Behind the scenes, Omi sends the live transcriptions to a Cloudflare worker, which does the appropriate database lookup on D1, and sends the message via a Telegram bot. Auth is managed with KV store and Telegram.

Telegram was only chosen for now due to not needing a registered business to run in production or at scale, but it could easily be extended to WhatsApp, Twilio, etc. Auth is the same way for the same reasons, but is already abstracted to separate tables and a canonical user id, would be straightforward to add in any other oAuth or email etc.

How we built it

Hands-on, everything in Rust, including the frontend and backend. All the code is handwritten by me. This builds on some previous work I've done in the space- but I had to solve a lot of problems for the first time here.

It's fully production ready, I'm paying for the hosting and fine with massive users as long as it remains cost effective (it would take incredible levels of adoption or abuse to make that a problem).

Challenges we ran into

Time in the day, family, a new puppy, and doing fullstack Rust has ... let's say unique challenges :)

Accomplishments that we're proud of

It's useful and a technological accomplishment.

Bottom line - it works, the code is elegant imho, and it can be scaled very very high for very very cheap.

What we learned

I've never written a Telegram Bot before, or used Omi. Learned lots!

Also had to learn more about Cloudflare Workers and think carefully about the codebase, and tackle new problems in Rust

What's next for Omi Assist

More delivery channels, primarily. Maybe more integrations besides text delivery (e.g. "dispense dog food")

Built With

Share this project:

Updates