Web-based 3D AI virtual companion with voice interaction, lip sync, and emotional expressions.
Codia is a web application that brings AI companions to life through 3D avatars. Inspired by CODE27 Character Livehouse, it provides an immersive AI companion experience accessible directly from your browser.
- 3D VRM Character Rendering - Load and display VRM models with expressions and animations
- AI Conversation - Multi-LLM support (OpenAI, Claude, Ollama)
- Voice Interaction - Text-to-Speech (Kokoro/ElevenLabs) and Speech Recognition (Whisper)
- Lip Sync - Real-time mouth animation synchronized with speech
- Expression System - 6 basic emotions that respond to conversation context
- Privacy-First - Local-first data storage with IndexedDB
| Category | Technologies |
|---|---|
| Framework | Next.js 14+, TypeScript |
| 3D Rendering | Three.js, @react-three/fiber, @pixiv/three-vrm |
| AI/LLM | OpenAI, Anthropic Claude, Ollama |
| TTS | Kokoro (browser-local), ElevenLabs |
| ASR | Whisper.js (browser WASM) |
| State | Zustand + IndexedDB |
| Styling | Tailwind CSS |
| Document | Description |
|---|---|
| 01-Product-Discovery | Market research, user personas, competitive analysis |
| 02-PRD | Product requirements, user stories, acceptance criteria |
| 03-Technical-Specification | Architecture, module design, API specifications |
| 04-UI-UX-Design | Design system, components, interaction flows |
| 05-Testing-Strategy | Test pyramid, automation, CI/CD integration |
| 06-Implementation-Plan | Sprint planning, milestones, task breakdown |
Phase: Planning & Design Complete
- Product Discovery
- PRD (Product Requirements Document)
- Technical Specification
- UI/UX Design System
- Testing Strategy
- Implementation Plan
- Development (8 weeks planned)
Development not started yet. See Implementation Plan for roadmap.
# Coming soon
npm install
npm run dev┌─────────────────────────────────────────────────────────────┐
│ Browser (Client) │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ UI Layer │ │ 3D Renderer │ │Audio Engine │ │
│ │ (React) │ │ (Three.js) │ │ (Web Audio) │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ └────────────────┼────────────────┘ │
│ │ │
│ ┌───────────┴───────────┐ │
│ │ Core Services │ │
│ │ • VRMService │ │
│ │ • ChatService │ │
│ │ • TTSService │ │
│ │ • ASRService │ │
│ └───────────┬───────────┘ │
│ │ │
│ ┌───────────┴───────────┐ │
│ │ State (Zustand) │ │
│ │ Storage (IndexedDB) │ │
│ └───────────────────────┘ │
└──────────────────────────┬───────────────────────────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ LLM APIs │ │ TTS APIs │ │ CDN │
│OpenAI/ │ │ElevenLabs│ │ (VRM) │
│Claude │ │ │ │ │
└──────────┘ └──────────┘ └──────────┘
- Lonely Young Adults - Need emotional companionship
- Virtual Character Enthusiasts - Anime/VTuber fans
- VTuber Creators - Low-cost creation tool
| Metric | Target |
|---|---|
| DAU | 10,000 |
| Avg Session | ≥15 min |
| D7 Retention | ≥25% |
| NPS | ≥40 |
MIT
- Inspired by CODE27 Character Livehouse
- Reference implementation: Amica
