Parakeet.ai | Devpost

Pear the Parakeet

Inspiration

Our project was inspired by CareYaya's ImmortalAI project, which seeks to create a digital avatar of a person that can be used to interact with their loved ones after they pass away. An issue we identified was that the process of collecting enough rich, personal data is a difficult and boring process if done incorrectly. Our goal was to design an application that would make the process of collecting data as easy and fun as possible for the elderly.

What it does

Parakeet.ai accomplishes this by providing a simple, easy-to-use interface that allows users to share their stories, mannerisms, and personality. Engaging prompts encourage a wide variety of responses, and users can respond via text or speech-to-text. Our product engages all generations of users, from the elderly to their children and grandchildren. The "curator" - our term for the storyteller sharing their life - can share their stories with their loved ones, who are able to read the responses like a journal.

For us, the AI digital avatar is an afterthought, not the main focus. We believe that the process of sharing stories should be fun and engaging. Families are always free to use the data collected in whatever way they wish to. If the curator wishes to use that data to create an AI avatar, they are free to do so. If they wish to simply share their stories with their loved ones, they are free to do so as well. All data will be stored securely and privately, and we do not sell any data to third parties.

How we built it

We built a web app, using a Firestore database to store login credentials, entries from the curator, and journal prompts. We used Express.js to interact with the database, and React to tie it all together in the front-end. Additionally, to improve accessibility for our users, we incorporated speech-to-text with the react-speech-recognition library. Finally, we deployed our website using Firebase.

Challenges we ran into

Two of our three main coders had no web development experience. To deal with this, our more experienced coder had to teach the basics of React and Express.js on the fly, and our other coders worked hard using documentation and online examples to learn quickly.
Speech-to-text was a stretch goal, but one we really wanted to accomplish since it would make our program much more accessible to visually impaired/mobility impaired users. In addition, in speaking with Neal from CareYaya we learned that many patients with dementia have an easier time speaking than typing. Therefore, we spent a lot of time making sure the system worked well, even though the libraries were difficult to set up.

Accomplishments that we're proud of

Successfully implementing speech-to-text
Getting two team members with no web development experience to quickly learn enough to build a usable website in 1.5 days, using a full, modern tech stack
Automatically sending onboarding emails to family members, as well as periodic summaries of the curator's stories
Setting permissions intelligently to allow principals (think caretakers, children) to set up accounts for elderly users to minimize friction onboarding

What we learned

We found it interesting to consider the accessibility features of our application, especially as most of our previous coding projects tended to assume our users were able-bodied and tech-savvy. Accessibility was an aspect of programming that none of us had much experience working with, but in dealing with a user-base of elderly people, we each gained a newfound appreciation for its importance.

What's next for Parakeet.ai

We hope to expand this project to provide better service to our users while generating even more robust data to unlock the potential for highly accurate Immortal AIs. Our roadmap going forward includes but is not limited to:

Integrating chatbots that can continue the conversation with our curators.
An embedded video speedfriending social media platform for our curators to socialize while opening a vector towards real likeness and personality modeling.
Adding an audio recorder that records the actual voices during speech-to-text such that families have the option of authentic voice replication.
Integrating game-changing health features such as using ML models to non-invasively screen each audio recording for signs of early onset dementia or Alzheimer's based on vocal patterns.
Integrating user data with an LLM to create a personalized avatar.

Using Parakeet.ai

Principal

The principal is in charge of setting up the account for the curator and inviting viewers. They can also view the curator's responses. This person may be a family member or a caretaker, for example.

Go to parakeet.ai and click on "Register Curator"
Enter all the personal information for the curator (the person sharing their stories). Click submit
Enter all the personal information for the principal (the person setting up the account). Click submit
In settings, click on "Invite Viewer" and enter the email of the viewer you wish to invite. Click submit when you are done
Go to parakeet.ai and login using your login credentials next time you use the app. You can view the curator's responses in the 'Read' tab and invite more viewers/change settings in the 'Settings' tab.

Curator

The curator is the person sharing their stories. They can respond to prompts and are able to view their responses. This person may be an elderly person, for example.

Go to parakeet.ai and login using the credentials provided by the principal
That's it! You can respond to prompts in the 'Write' or 'Audio' tabs. You can view your responses in the 'View' tab.

Viewer

The viewer is the person viewing the curator's responses. They can view the responses, but do not have privileges to change settings for the curator like the principal does. This person may be a child or grandchild, for example.

Use the link provided by the principal (sent via email) to create an account
Go to parakeet.ai and login using your login credentials next time you use the app. You can view the curator's responses in the 'Read' tab.