Mentalsense

💡 Inspiration As college students juggling classes, jobs, and social lives, we often don’t realize how stressed or burnt out we are until it's too late. I wanted to build something that didn’t just ask how you're feeling—but knew. MentalSense was born from the idea that our environment could respond to our emotional state in real time, offering subtle nudges toward balance, calm, and clarity.

🛠️ What it does MentalSense’s Mood Mirror is a passive mood detection system that listens to how you speak, type, and interact during the day, infers your current emotional state, and adjusts your environment accordingly. It can:

Detect mood from voice tone, typing patterns, and facial expressions.

Display mood trends over time in a dashboard.

Automatically adjust Spotify playlists, recommend breaks, and change lighting or screen themes to match your emotional needs.

⚙️ How I built it I used a combination of:

Frontend: React + TailwindCSS for a clean, minimal dashboard UI.

Backend: Node.js with Express to manage user sessions and mood logs.

Mood Detection:

Whisper for transcribing voice and analyzing tone.

A small fine-tuned transformer to label mood categories.

Keystroke pattern tracking using a custom browser script.

(Optional) Webcam-based emotion recognition via mediapipe or OpenFace.

Integrations:

Spotify API for music adjustments.

Philips Hue API for lighting changes.

Google Calendar for break suggestions.

🚧 Challenges I ran into Getting accurate emotion classification from noisy data like keyboard inputs was tough.

Facial expression tracking posed ethical/privacy concerns, so I had to make that an opt-in feature.

Timing and syncing Spotify/light/environment changes without overwhelming the user took iteration.

🏆 Accomplishments that I'm proud of Built a real-time mood detection loop using three different passive input channels.

Integrated a fully working Spotify playlist tuner that responds to emotional shifts.

Created a dashboard that visualizes emotional trends with actionable insights.

📚 What I learned Emotion detection is far more nuanced than I expected; context matters deeply.

Users are more open to passive mental health tools when given transparency and control.

Micro-changes in environment can have macro-effects on productivity and well-being.

🚀 What's next for MentalSense Integrating with Zoom and Slack to auto-adjust notification settings based on stress levels.

Adding journaling and reflection tools tied to mood data.

Exploring partnerships with wellness platforms or therapists to provide personalized support.

Building a mobile version for on-the-go emotion tuning.

Built With

• next
• perplexity
• shadcn
• tailwind
• vapi