super adventure

Inspiration

- newest AI tools are able to provide help in all aspects, we can make work and study easier by using these tools to build things with voice, gesture, provided wanted styles and videos

What it does

- generate notes based on the video/recording provided, in your own style of writting/designing
- control the page/redirect to other tabs using gestures
- auto voice coach for real time inquiries about current content
- code analyser
- locate each kownledge to corresponding minute second of the video

How we built it

frontend: - Next.js - Node.js - React - Konva

backend: - ElevenLabs - Gemini - MediaPipe - OpenAI - JavaScript

Challenges we ran into

- there has been one error that kept preventing us from enabling the camera
- the hand gesture recognizations were not accurate (mistake trigerring)
- displaying the note on the note pad instead of just a file
- makeing the text match whichever background we choose was tough
- piecing every components of the solution together
- passing context between models

Accomplishments that we're proud of

-  making the gesture recognization work
- generate clear notes using the platform
-  Pulling an all-nighter

What we learned

-  Voice agent architecture
- 2D mapping library
- Image generator
- gesture recognition principles

What's next for super adventure

• live recording
• graphing
• real time analysing

Built With

• elevenlabs
• genmini
• javascript
• next.js
• node.js
• openai
• react