HeartTalk | Devpost

Create an account with HeartTalk to login.
The HeartTalk dashboard will display an intuitive overview of health activity!
This page is for tracking medical and/or daily records.
You can add new records...
... or modify saved ones!
This page keeps an organized history of HeartTalk conversations.
This page is for chatting with your HeartTalk device!
Click the start button to start talking. Stop whenever you're done.
This is our smart HeartTalk device!

Inspiration

HeartTalk is inspired by the growing challenges faced by aging populations in developed countries, where many seniors live alone, increasing the risk of social isolation and reduced mental and physical well-being. Dementia research highlights the importance of early detection, cognitive stimulation, and support systems, yet stigma and lack of awareness often hinder progress. Drawing from these insights and leveraging advancements in conversational AI and companion robotics, HeartTalk aims to create an accessible and empathetic solution that fosters connection, independence, and proactive care for seniors, filling in for when caretakers cannot be there. The project also incorporates ideas from companion robotics research and the personal experiences of families navigating care for aging loved ones. HeartTalk is not meant to replace caretaker duties, but to support their routines.

What it does

HeartTalk aims to promote both physical and mental health through a variety of features such as a chatbot, health data maintenance, and journaling. By attaching sensors to a Raspberry Pi, we sent data to a backend server where a LLM model could analyze this data and provide feedback not only to the user, but a web application where this data can be viewed remotely by the patient as well as caretakers.

How we built it

In terms of the web application, to ensure robust security in our web application, we utilized the MERN stack and enhanced the 1Password Passage Complete authentication by integrating JWT tokens. This approach provides tamper-proof, stateless authentication by verifying tokens on every request, ensuring only authenticated users access sensitive resources. Additionally, we enforce strict token expiration, forcing users to reauthenticate periodically, which minimizes the risk of unauthorized access from stale sessions. Moreover, We utilized Defang.io for backend deployment, which streamlined the process by bootstrapping the backend development environment and enabling rapid and efficient deployment. This approach reduced setup complexities, minimized deployment time, and ensured a reliable infrastructure for the backend. The other half of the project revolves the hardware system built with Raspberry Pi, a microphone and speaker and pulse sensor for conversational LLM capability and health monitoring features. The Raspberry Pi sends data to the MongoDB database as well as interacting with our backend endpoints for LLM and health data.

Challenges we ran into

One of the biggest challenges was working with the hardware as well as limitations of our resources. A large portion of our time was spent debugging and setting up our Raspberry Pi and sourcing hardware for our necessary features. Additionally, we were constrained by the memory resources on our hardware and backend server when considering LLM models.

Accomplishments that we're proud of

We managed to create an intuitive user interface with robust user authentication to protect users’ sensitive data. We also created a compact, portable smart device.

What we learned

Through our troubleshooting of hardware, we learned more about topics such as IP addresses, circuits and limitations of the Raspberry Pi itself. We also explored and implemented new technologies to enhance the efficiency of software development and streamline workflow.

What's next for HeartTalk

HeartTalk has a lot of potential for improving healthcare strategies. Although the current model has basic functionality, many new features could be implemented. In the future, we would like to facilitate an even greater scope of senior care by adding a reminder system for notifying seniors and their caretakers when to take medication, take a walk, or take a nap. Additionally, we hope to update the HeartTalk device to have improved responses, more functionality (e.g. providing ambient music), and more intuitive user interface. It is also important to note that further research is needed to polish legal policies. Our team at HeartTalk values user privacy and safety. Before subsequent releases, the product will have extra safeguards and security permissions to access certain features. Our users are always our top priority!