A project for Snap Spectacles to assist with a variety of mental health issues - being developed for Reality Hack 2026 MySight is an augmented‑reality experience for Snap Spectacles that aims to support users with mental‑health challenges. Built during the MIT Reality Hack 2026 hackathon, MySight explores how wearable AR can provide discreet, hands‑free assistance wherever you are. The project’s goal is to make mental‑health tools more accessible and user‑friendly by harnessing the capabilities of Spectacles and on‑device machine‑learning models.
Inspiration
Mental‑health difficulties—such as PTSD, anxiety, depression and stress—affect millions of people. Even when resources are available, it can be hard to find the time or motivation to engage with conventional tools like journaling apps or mindfulness exercises. Our team wanted to create a companion that users could turn to any time without having to pull out their phone. Snap Spectacles provide a lightweight platform where AR overlays can guide the user through grounding techniques, mood tracking and breathing exercises. This hackathon gave us a chance to explore how ambient computing and thoughtful design could make mental‑health support feel natural and personal.
What it does
MySight runs entirely on Snap Spectacles. When you put the glasses on, the app starts listening for cues—either via hand‑gesture input or voice commands—to launch short exercises designed to improve mood or reduce stress. Key capabilities include:
Emotion recognition and logging: The app uses on‑device neural networks (exported as ONNX models) to analyse facial expressions and infer the user’s current emotional state. This information helps tailor the exercises and provides a private mood log for future reflection.
AR‑guided breathing and mindfulness: MySight overlays gentle animations and prompts in your field of view, guiding you through paced breathing, mindfulness meditations or grounding techniques without breaking immersion.
Cognitive‑behavioural prompts: When the system detects signs of anxiety or low mood, it offers short cognitive‑behavioural therapy (CBT) prompts and reframing exercises to help the user challenge negative thoughts.
Hands‑free interaction: All interactions are designed for Spectacles. The user can trigger exercises using simple gestures (like tapping on the frame) or voice commands, making the tool truly hands‑free.
How we built it
MySight is a Lens Studio project. Snap’s template .gitignore shows that Lens Studio generates a number of cache and workspace files that we exclude from version control. We leveraged several key components:
SnapML for emotion recognition: We trained a convolutional neural network on facial‑expression datasets and exported it as a set of ONNX models (best_export_224.onnx, best_export_640.onnx and their fp16 variants) that run efficiently on Spectacles. The models are loaded through SnapML and run locally, preserving user privacy.
Lens Studio scripting: The project is written in TypeScript and JavaScript, using Lens Studio’s scripting environment to orchestrate the UI and call SnapML models. Scene objects—such as text overlays and animated breathing visuals—are managed via script components.
State management: A simple state machine tracks whether the user is idle, in an exercise or logging a mood. This allows the app to switch exercises and prompts smoothly based on detected emotions or user input.
Version control: Only source files, model files and configuration scripts are committed. Lens Studio’s caches and workspaces are ignored according to the .gitignore.
Features
The current prototype demonstrates several features that we plan to refine and expand:
Mood‑aware interventions: Real‑time facial‑expression analysis adapts the app’s suggestions. If the user appears stressed, the app prioritises breathing exercises; if they look sad, it offers uplifting prompts.
Discrete UI: The interface uses subtle translucent overlays and gentle animations to avoid overwhelming the user. A progress ring around the periphery of your vision indicates exercise duration without requiring you to look away.
Offline operation: All models run on‑device, so the app works without an internet connection. This is crucial for privacy and for use in situations where connectivity is limited.
Customisable routines: Users can pre‑configure a list of preferred exercises and set reminders. A simple configuration file defines the sequence and timing of prompts.
To run or modify MySight you’ll need:
A pair of Snap Spectacles (dev edition) with the latest firmware.
Lens Studio installed on your computer.
Git (for cloning this repository).
Running the project
Clone the repository:
git clone https://github.com/Nmagic144/Niche.git cd Niche
Open the project in Lens Studio:
Launch Lens Studio and choose Open Project.
Navigate to the cloned repository and select the project file (e.g., project.lns or project.json).
Connect your Spectacles:
Ensure your Spectacles are set up as a developer device.
Use the Preview button in Lens Studio to test the experience in the simulator or deploy it to the glasses.
Contributing
Contributions are welcome! To suggest a feature or submit a pull request, please open an issue first so we can discuss your idea. Make sure to test changes on Spectacles and include reproducible instructions.
