DREAMAIL

A spatial AR interface for everyday digital tools, powered by AI-driven interaction.

Comment

DREAMAIL

Inspiration

Reimagining Email in Virtual Reality

As we venture into augmented reality, we reimagine how essential everyday digital tools can exist within spatial environments, and how AI-driven interactions reshape the way we perceive, organize, and engage with them.

DREAMAIL takes the traditional email experience and turns into an immersive 3D environment where emails become interactive spheres you can grab, toss, and organize naturally in VR space.

What it does

DREAMAIL is a revolutionary VR email client for Meta Quest that brings your inbox to life in three dimensions. Instead of scrolling through endless lists, emails appear as colorful spheres floating around you in virtual space.

Core Features:

  • Real-time Email Visualization: Emails from your Gmail inbox materialize as 3D spheres in VR, updating in real-time via Google Cloud Pub/Sub
  • AI-Powered Summaries: Each email is automatically analyzed and summarized using Google Gemini AI, so you can instantly understand what matters
  • Voice-First Interaction: Use speech-to-text to dictate quick email responses without typing
  • AI Email Generation: Speak a brief message like "schedule meeting next week" and DREAMAIL generates a complete professional email with proper formatting and signature
  • Intelligent Categorization: Emails are automatically classified (Work, Personal, Promotions, Social) and prioritized (1-5) using AI
  • Natural VR Interactions: Grab spheres to read emails, throw them toward your view to snap them into focus, or toss them away to dismiss
  • Audio Summaries: Text-to-speech narration of email summaries for hands-free review

How we built it

Platform & Framework:

  • Built in Unity for Meta Quest VR headsets
  • Oculus Integration SDK for hand tracking and VR interactions
  • C# for all game logic and API integrations

Cloud & AI Services:

  • Google Cloud Pub/Sub: Real-time email notification pipeline
  • Gmail API: Email fetching and management
  • Google Gemini AI: Email summarization and categorization
  • Google Speech-to-Text API: Voice recognition for hands-free commands
  • ElevenLabs API: Neural text-to-speech for audio summaries

Key Technical Systems:

  1. Email Sphere System (EmailSphere.cs): Physics-based 3D objects with grab mechanics, velocity-based snapping, and focus states
  2. Speech Recognition (SpeechRecognition.cs): Records microphone input, converts to WAV format, and sends to Google Speech API
  3. AI Email Generation (GenerateEmail.cs): Transforms brief voice input into full professional emails with proper structure
  4. Email Analysis (EmailAITool.cs): Multi-faceted AI analysis including summarization, categorization, and priority scoring
  5. Real-time Sync (GCloudPubSubManager.cs): Maintains live connection to Gmail via Pub/Sub for instant notifications
  6. State Management: Global state system for managing focused spheres and UI states

Architecture:

  • Singleton pattern for core services (Audio, SpeechRecognition, EmailAITool)
  • Event-driven communication using UnityEvents
  • Coroutine-based async API calls
  • Component-based design for modular email sphere features

Challenges we ran into

Too many (multiple apis, hand tracking, meta UI… )

Accomplishments that we're proud of

We slept

What we learned

Sleeping is important

(But) for realizies:

VR Design Principles:

  • Physical metaphors work best in VR (throwing to dismiss, grabbing to read)
  • Visual state feedback is crucial (Selecting → Detecting → Generating → Displaying)
  • Minimize text entry in VR - voice input is far more natural

Cloud Architecture:

  • Pub/Sub provides reliable real-time notifications without polling
  • API key authentication is simpler than OAuth for prototypes
  • Always monitor API costs in production scenarios

Built With

Share this project:

Updates