Echo Notes | Devpost

The main page of our web app.
A screenshot of the terminal during the voice recognition process.
Different Features of the app which can be accessed by scrolling using arrow keys.
A code snippet of the speech recognition feature.

Inspiration

Our team drew inspiration for this project from the increasing dependence on technology, especially AI, to perform tasks efficiently in today's world. Over recent years, online notes and AI have revolutionized how we study. However, upon encountering the hackathon prompt, we realized that not everyone enjoys equal access to these tools. Traditionally, blind or visually impaired individuals relied on braille (a system of touch reading and writing for blind persons in which raised dots represent the letters of the alphabet). With the majority of notes moving online, braille has become less relevant. Therefore, we created Echo Notes, a platform that allows visually impaired users to listen to their notes in audio format.

What it does

Our solution is a website meticulously designed for the visually impaired, featuring a user interface tailored to their needs and enhancing their educational experience. Utilizing the space bar as a hot-key in conjunction with voice commands, users can effortlessly navigate the platform. They can listen to their notes, adjust the audio speed, translate content between languages, pose questions to AI models in real-time, generate quiz questions for active recall practice, and receive summaries for last-minute revision—all without the necessity of visual interaction.

Using our solution is straightforward. By pressing the space bar—chosen for its ease of location without visual aid—and issuing a voice command, users can seamlessly interact with the platform. Upon loading the program, clients are guided through the available voice commands. This enables them to effortlessly access all features, thereby maximizing their educational potential.

How we built it

We built the main app using python with streamlit to make it a web app. Additionally we used the chat gpt api and deepl api to include the translation, quiz, queries and summarization feature. Additionally, we used various other packages including speech recognition using sr and OCR feature using pytesseract.

Challenges we ran into

Overall, the hackathon went according to plan. Initially, we did encounter a few challenges, such as finding appropriate tools for voice recognition and converting image files to text files using OCR. However, once we resolved those issues, we were able to complete our project successfully. Another challenge we faced was during the idea formation process. We invested significant time in brainstorming each feature and ensuring its functionality.

Accomplishments that we're proud of

We're proud of successfully implementing all the features we had envisioned for this project. Moreover, we're pleased with the exceptional performance of the voice recognition system and our effective utilization of OpenAI and DeepL. Additionally, we take pride in creating a user-friendly interface where blind users can simply activate voice commands by pressing the spacebar, eliminating the need for any external assistance.

What we learned

This was our first experience working with the ChatGPT API and DeepL API, so we had to learn how to implement them. Additionally, prior to this hackathon, we had no experience with voice recognition, so learning how to use that was also a new challenge for us.

What's next for Echo Notes

Firstly, we plan on adding a feature that allows users to select specific points in their notes to navigate to and then study from so that users have full access of their notes and complete flexibility.

Next, we'll add a feature that allows users to edit the notes they upload using our software through the voice commands that we already use.

Finally, we plan on extending the usability of this app to various other devices such as iPhones, Ipads and even android devices.