Banter | Devpost

Inspiration

It's a story everyone knows: you're stuck in traffic or walking home, and you feel like you could be doing something. But it's hard when you barely lack the attention to read a book or even listen to a podcast. What can you do?

In comes Banter.

What it does

With Banter, users can conveniently interact, navigate, and ask related information about any book they upload.

How we built it

Our front-end interface runs on React and Typescript, which incorporates an embedded voice recognition model to get user input. Data is seamlessly transfered using the Fetch API to our Tornado backend, where we subsequently leverage OpenAI's GPT 3.5 model to generate accurate responses.

Challenges we ran into

Distinguishing computer and user speech during the conversation when on speaker
Adding on commands and features for specific use cases
Being limited by the OpenAI API
Handling HTTP requests to integrate the backend and frontend
Unanticipated Typescript behaviors

What we achieved

Learned new languages (Typescript, HTML/CSS, Python)
Gained collaboration skills
Seamlessly integrated backend and frontend

What's next for Banter

🎙️ Voice Annotations & Bookmarks:

Enable users to add voice notes, comments, and bookmarks at specific points in the audiobook so they can interact and reflect on the content.

⏪ Adjustable Playback Speed:

Allow users to control the playback speed through voice commands like "Speed up" or "Slow down" to suit their listening preferences.

🗨️ Multi-Language Support:

Implement multilingual voice recognition and narration to cater to a diverse audience.

📚 Voice-Controlled Library Management:

Enable users to add, remove, or organize audiobooks in their library using voice commands like "Add a new audiobook" or "Create a 'Favorites' folder."

🧠 Smart Recommendations:

Utilize AI algorithms to suggest audiobooks based on the user's preferences and reading history, which can be refined through voice input.

🎮 Gamification for Engagement:

Introduce voice-controlled challenges, quizzes, or rewards to make the audiobook experience more engaging and interactive.

Built With

Submitted to

HackRice 13

Created by

I designed the visual identity and video presentation for Banter. I also worked on the user-interface components to make for a polished user experience.

I learned a lot about how large language models can be used to efficiently make these sorts of machines. While I was ready to hard-code a decision-tree of predefined responses, my teammates had already developed a working model of a bot that responds to questions about a provided book. It's crazy how fast things are changing these days!

Thomas Kaldahl
I’ve developed the backend handlers and methods and implemented the llm (openai api) which process the book and do the chatting functionality. I’ve also helped with some parts of the frontend, especially the voice recognition part.

This is the first time that I encounter typescript frontend project, and I’m determined to learn more of this “magic”.

Barbara Su
I developed the frontend interface & the connection to the backend. I definitely tightened up my webdev skills - especially Typescript syntax.

Emily Huynh
- Brainstorming the app ideas with Thomas
- Spearheading discussions of features, and user stories
- Navigating around and debugging small block of codes between frontend and backend

gizaquoc197 Nguyen