Inspiration
When our team member Sherry volunteered at a senior center in Vaughan during Grade 9, she noticed that seniors often knew what they wanted to do on a laptop, but the steps to get there were confusing. Small text, crowded keyboards, and complex sequences made even simple tasks like sending emails or booking appointments overwhelming.
This inspired ELDA, the Elderly Learning Digital Agent. The real barrier was not learning new technology, but completing tasks confidently and comfortably. ELDA provides clear, step-by-step guidance while also addressing usability challenges.
What it Does
Elda is an intelligent voice agent specifically designed to help elderly users navigate technology with confidence. Our MVP centers on three key features:
- Voice-guided task assistance: Seniors can ask ELDA for help, and it breaks down complex tasks into clear, step-by-step instructions.
- Visual guidance & zoom: ELDA highlights exactly where to click and can zoom in for improved visibility.
- Audio and brightness control: Seniors can adjust their computer’s volume and screen brightness using simple voice commands.
Together, these features help seniors navigate technology independently, reduce frustration, and build confidence.
How We Built It
We began by pinpointing the main challenges seniors face with digital tools. Based on these insights, we developed a minimum viable product (MVP) that combines voice recognition for natural interaction, visual overlays for guidance, and system-level audio and brightness controls. ELDA sequences tasks step by step, allowing seniors to follow along confidently without feeling overwhelmed. Our design prioritizes accessibility, with a clear, simple, and intuitive interface that is easy to navigate.
Technical Implementation
Frontend: React-based Electron app with modern UI/UX Backend: Python Flask server for tutorial generation AI Integration:
- OpenAI Whisper for speech recognition
- Google Gemini for intent classification Communication & Data Flow:
- WebSocket integration between components
- HTTP/REST for data fetching between React frontend and Flask backend
Challenges We Ran Into
- Bringing together voice recognition, visual guidance, zoom, and system-level controls into a single, smoothly functioning system proved complex and required careful design and testing.
- Learning Electron to create ELDA’s popup interface involved understanding its structure, APIs, and integration with our Python backend.
- Implementing reliable text-to-speech required selecting the right libraries, handling asynchronous processing, and ensuring timely, clear audio feedback.
- Designing ELDA to provide immediate visual cues, such as highlighting areas to click and zooming in, required precise coordination between the interface and task sequencing.
Accomplishments That We're Proud Of
We are most proud that we were able to bring all of ELDA’s features together into a single, fully functional system. ELDA doesn’t just run separate parts on their own; everything works together seamlessly so it feels like a real digital assistant. The fact that seniors can speak to ELDA and see it respond right away, just like Siri, shows that we were able to create something smooth, connected, and reliable. This was our biggest accomplishment because it proved our idea could truly work in practice.
What We Learned
Building ELDA taught us a lot because it was our first time developing a voice-controlled agent and using Electron to create a desktop interface. We faced many challenges, like figuring out how to make the voice commands trigger the right actions and ensuring the interface responded smoothly to user input. There were moments of trial and error, and sometimes progress was slow, but we learned how to connect multiple technologies in a cohesive system. We also gained hands-on experience with integrating AI task sequencing, visual guidance, and system-level controls. Even with the difficulties, it was incredibly rewarding to see all the pieces come together into a fully functional assistant that seniors can actually use.
What's next for ELDA
- Multi-language Support: The MVP is currently built in English, but we plan to use services like Amazon Comprehend, Eleven Lab's tools to scale language support. This would allow ELDA to understand and respond in over 100 languages, making it more inclusive and accessible to diverse communities.
- Database Scalability Task Memory: ELDA can keep a record of previous requests and actions by connecting user authentication, and database management with Postgresql, AWS S3 Bucket, triggers etc. Seniors can review past instructions or steps, making it easier to repeat tasks, track progress, and avoid having to ask the same question multiple times.
- Workflow Integration APIs: ELDA could connect with apps like email, social media, and banking to screate AI agentic workflow tasks. Seniors could complete everyday tasks more easily and without navigating complicated menus.
- Offline Capabilities: Reduce dependency on internet connectivity
Log in or sign up for Devpost to join the conversation.