Invincibot

Invincibot is a themed AI summarizer that accepts text, audio, and file input.

Comment

Inspiration

Invincibot is inspired by the hit TV series, Invincible, and it ties into the overall superhero/comic book hackathon theme. Invincibot can help those with visual impairments understand class notes better by providing text-to-speech capabilities and reading every response aloud.

What it does

Invincibot is an AI text summarizer that accepts user-entered text and file input for .txt files and .mp3 files. The AI has been trained with ElevenLabs to replicate the voice of the Invincible characters, making the program more interactive for users. Since the AI works with text-to-speech and speech-to-text, it would be helpful for users with low vision who prefer auditory input and output.

How we built it

For the front-end, we used ReactJS and Tailwind CSS. On the backend, we built it with Python, JavaScript, and LLMs like Google Gemini, BART, and ElevenLabs.

Challenges we ran into

We ran into challenges like using A2A protocols and developing the UI. Instead of using A2A, we decided to have the LLMs directly talk to one another. Regarding the UI, we found it difficult to make the application fully responsive and ran into issues when developing the chat window.

Accomplishments that we're proud of

We successfully designed a Figma prototype and developed an AI web summarizer using the following tools: LLMs (Elevenlabs, Facebook BART Model, and Google Gemini API), React, and Tailwind. We’re proud that we brought our ideas and prototype to life, which is themed around our current favorite TV series: Invincible. We also successfully replicated the voices of Invincible, Atom Eve, and Omni-Man, who are characters from the show. After attaching a file or sending a message in the chat bubble, the response outputs as text in the selected character’s voice. Overall, we’re glad we completed a challenging technical project within three days.

What we learned

During this project, we learned an extensive amount about LLMs, backend architecture, and frameworks. Using Google Gemini and the BART summarizer meant we had to learn how to get the LLMs to talk to each other, we developed a unique algorithm for determining how often and when the two models communicate found in our GitHub. It was also our first time connecting the back-end of an application to the front-end. It took some time to navigate this process, but we handled it smoothly. This project was our first time using Tailwind CSS as well.

What's next for Invincibot

In the future, we plan to upgrade Invincibot to allow users to record audio directly on the platform and upload it. We would also consider adding more characters to the platform for users to interact with. Additionally we want to expand the modality of the model, allow for image and video input, more file input, concatenation of varying elements, and a more refined summarizing agent that we can fine tune as opposed to using a pretrained model.

Built With

Share this project:

Updates