Calls4Mental

Call an ACTUAL, TALKING, and SENTIENT AI, 2D therapist (3D is creepy).

Comment 1

Calls4Mental

LOC (lines of code)

Image

GRANDE (big/large in spanish?) idea ☁️☁️☁️

Everyone knows that theres a problem in mental health related deaths. (I bet that theres gonna be like 10 projects out there that have this as an "inspiration").

Current technology in a "virtual therapist" kinda suck 😢

  • If you argue that you will just text a LLM and it would be better than speaking with a rando AI model, you would be horribly wrong. Directly speaking and narrating your emotions are proven to be much more effective at helping oneself find inner peace
  • If you argue that you will just find a AI model therapist that can talk, you would struggle to find one.
  • If you somehow found the AI therapist that can speak, it would definitely not have a talking face.
  • If you somehow, out of a million chances found a AI therapist that 1. can speak, 2. has a face, I can gaurentee you that the talking face is probably creepy. What I mean is it is probably lip synced to some 3D render of a human's face without the body... ew 🤢.

On a broader scale,

  • There is also a lack in available therapists
  • Early mental illness diagnosis is not normally seen (Read below)

Therefore, I created Calls4Mental, a calling app that will connect you to different 2D avatars (you will see in my devpost video) that will chat with you as a professional therapist. This app will also diagnosis you and connect you with real therapists, increasing mental illness diagnosis accessibility.

Features ☀️

Entire app

  • Camera capabilites ✅
  • Microphone capabilites ✅
  • 1 (summerizer), 2 (therapist), 3 (emailer), 4 (sentiment detection), 5 (emotion detection), 6 (Speech to Text), 7 (Text to speech), 8 (object detection + image analysis). 8 AI models used. ✅
  • On Windows (only for pc for now) ✅
  • Local Data storage ✅
  • CUSTOMIZABLE 2D AVATARS This means you can put anything, (yes, anything) you want to to be the therapist when talking to you. ✅ 😏😏😏
  • Sick logo ✅
  • SUPER DUPER CUTE avatars? ✅

Overview

UI (Frontend) Server (Backend)
Login screen Can use locally hosted TTS AI model or openai key
You can choose which 2D avatar is talking to you (WOW TALKING AVATARS??? VTUBER ALERT??💥💥) 2D avatar is customizable
You can choose which voice the avatar uses to talk you Voice is customizable in the code... (means just harder to customize)
You can chat with and without your camera Invovles python threading package and callbacks to manage microphone and video call inputs, basically creating a mini-zoom (woooo complex right!!! Im sure that no one in this hackathon used that 😎)
Microphone input level Uses AI to analyze conversation to output mental illness index for early mental illness detection
Conversation summary and history display via graph (using matplotlib) and table Emails therapists about mental illness if detected
Neighborhood GPS map using openstreetmaps -> these display therapist locations and therapist phone numbers Uses pygame 🦆 to play audio (unexpected right!??)
Simple UI -> VERY EASY TO USE

Future features

  • Make login feature better
  • Make the gps actually return therapist locations LOL
  • Send emails to actual therapists
  • Deploy to GLOBAL scale 🎙️

Challenges faced 💀

I work with flutter/dart not this thing 😢. I swear if I ever have to touch tkinter once again I will be using the app frfr.

^TDLR; Tkinter is an extremely rudimentary coding package that basically required me to build everything from ground up and it was a pain to work with because of its weird tweaks and lack-of-support-for-other-coding-packages. However, I would like to thank tkinter for easily connecting python backends to the frontend. Lowkey I don't think I could have programmed this much STUFF on my flutter/dart project. LOL

Built With

Share this project:

Updates