Sighted people have Lovable. Blind creators have Hearcraft — voice in, voice out, build apps by speaking. By them, for them.
Hearcraft is a voice-first iOS app that lets visually impaired creators build, run, and share voice agents ("crafts") entirely by speaking. No screen. No code. Just conversation.
Creation should be joyful and accessible. Every no-code builder today assumes you can see a screen. Hearcraft assumes you can't — and builds the entire experience around voice as the only interface.
- Speak your idea. Describe the voice app you want in natural language.
- Hearcraft builds it. The Gemini Live API captures your intent, and Gemini 3 Flash transforms it into a structured craft configuration (JSON) — system prompt, tools, voice personality, and capabilities.
- Run it. Your craft launches as a real-time voice agent you can talk to immediately.
- Share it. Crafts are portable configs that can be shared with other creators.
- Voice-first creation — Speak your idea, get a working voice app. No typing, no visual UI required.
- Real-time voice agents — Each craft runs as its own live conversational agent with custom system prompt, personality, tools, and voice selection.
- Camera mode — Camera-enabled crafts stream live video to Gemini, describing surroundings in real time.
- Web research — Live Google Search via function calling, with a verbal cue so the user knows a search is happening.
- Location awareness — Google Maps API integration for geographically relevant context.
- Accessibility-first UI — Large touch targets, minimal navigation depth, VoiceOver-optimized flows, verbal confirmations at every step.
- Craft management — Manage mode for safe, accessible editing and deletion.
| Craft | Description |
|---|---|
| Accessible Travel Scout | Plans trips using real blind/low-vision travel experiences |
| Top Grade Persuasion | A 10-round persuasion game — can you convince the AI? |
| Local News Flash | Starts immediately with a 5-item local news brief |
┌─────────────────────────────────────────────┐
│ User speaks an idea │
│ ↓ │
│ Gemini Live API (real-time voice session) │
│ → bidirectional audio streaming │
│ → generates conversation transcript │
│ ↓ │
│ Gemini 3 Flash (structured output) │
│ → transcript → JSON craft config │
│ ↓ │
│ Craft runs as a live voice agent │
│ → own system prompt, tools, voice │
│ → camera / search / location capabilities │
└─────────────────────────────────────────────┘
| Component | Technology |
|---|---|
| Framework | SwiftUI (iOS 17+) |
| Real-time voice | Gemini Live API |
| Craft generation | Gemini 3 Flash (structured JSON output) |
| Audio + camera | AVFoundation |
| Location | Google Maps API |
- Clone the repo and open
Hearcraft.xcodeprojin Xcode. - Build and run on an iOS 17+ device or simulator.
- Enter your Gemini API key when prompted on first launch.
Note: For the full experience (camera, microphone), test on a physical device.
Hearcraft/ → App source code
doc/ → Product specs + API notes
scripts/ → Local test helpers
- API keys are entered in-app and stored locally on-device.
- No keys are committed to this repository.
- Community sharing — Publish, browse, and remix crafts from other creators.
- More capabilities — Calendar, document scanning, smart home, contacts.
- Cross-platform — Android and web to reach the full VI community.
- Cloud infrastructure — Migrate from local storage to cloud DB for sync and sharing.
- Smart glasses — Ray-Ban Meta integration for hands-free craft experiences.
TBD
