RoloTim.ai

Spinning life's Rolodex, with each card holding the map of time with your precious ones.

Comment

Inspiration

Introducing Tim - your digital memory guardian and AI Assistant.

Physical memories often hold immense sentimental value, yet they are prone to fading with time and vulnerable to loss through unforeseen circumstances. Amidst our hectic schedules, it's all too easy to lose track of the countless conversations, trips, and cherished moments we've shared with our loved ones over the years.

However with RoloTim.ai, forgetting becomes a thing of the past. It's your digital memory bank, carefully cataloging every nostalgic interaction with a design that brings to mind the old-fashioned Rolodex. Tim has all your mementos stored securely for you to access whenever you need them. Say goodbye to missed memories and hello to a journey into your past.

What it does

Suppose you are talking to Harry at UofTHacks:

  • You can activate RoloTim.ai with the sentence “Tim begin.”.
  • This starts recording the conversation. It also snaps a photo of Harry and vectorizes it using OpenCV.
  • Next, Tim compares this picture to your personal contacts database on Firebase to identify Harry.
  • If Harry is recognized, a new event is added to his timeline. If not, a new timeline is created for him.
  • To stop recording the conversation, you simply say “Tim end.”.
  • After your conversation, details like date, location, event name, and summary are extracted by AI and stored in the event details. Conversation summaries are generated using Cohere.
  • You can also add personalized notes about UofTHacks and your conversation with Harry
  • Finally, If you want to search for something, you can use the app’s AI rerank functionality which will give you an ordered list of the most to least relevant events.

How we built it

  • React.js and Tailwind CSS for the frontend
  • Flask for backend servers
  • Firebase to host the database on the cloud
  • OpenAI for extraction of single line text
  • Cohere to summarise transcripts and rank search results

Challenges we ran into

  • Linking the frontend with the backend
  • Getting images as vectors to store in Firebase and retrieving them to compare with the other vectors
  • Training our face recognition model to accurately recognize different faces
  • Not having a mockup on Figma caused back-and-forth

Accomplishments that we're proud of

  • Finishing the whole project
  • Completing almost all of the features we initially wanted in the app
  • Learning how to extract a single piece of data from a transcript
  • Learning how to train and implement a face recognition machine learning model

What we learned

  • How to use the Cohere API and OpenAI API
  • How to use Tailwind CSS and Flask
  • Implementing dynamic components and modules like Webcam in React
  • How to train machine learning models

What's next for RoloTim.ai

  • Implementing User Authentication
  • Deploying the website
  • Creating a mobile app for improved portability
  • Creating a wearable device (i.e. glasses) with the app running on it so that users can record their pictures and conversations without needing to pass a device around

Built With

Share this project:

Updates