MultiChat AI

Real-time group chat where multiple users interact with the same AI character.

Comment

💡 Inspiration

Current AI chatbots are designed for one-on-one conversations. But what happens when a group of people wants to brainstorm with AI? Or when a study group needs an AI tutor that can respond to multiple students? We realized there was a fundamental limitation: AI can't handle group dynamics.

Traditional chatbots don't know when to speak up, who to address, or how to maintain context when multiple people are talking simultaneously. We wanted to solve this problem and create a truly multiplayer AI experience that feels natural, engaging, and intelligent.

🤖 What it does

MultiChat AI is a real-time chatroom where multiple users can interact with an AI character named Nomi simultaneously. The system intelligently decides:

  • When to respond: Detects @mentions, conversation lulls, direct questions, or after 3+ human messages
  • Who to address: Recognizes which users are active and references them by name
  • What to say: Maintains both group conversation summaries and individual user memory to provide contextually relevant responses

Users see messages stream in real-time, typing indicators show who's active, and the AI responds naturally as if it's a participant in the conversation—not just a question-answering bot.

🛠 How we built it

Architecture:

  • Frontend: React + Vite for the UI, Socket.io Client for real-time WebSocket connections
  • Backend: Node.js + Express server with Socket.io for WebSocket management
  • AI Integration: JanitorAI's JLLM API with 25,000-token context window for streaming responses
  • Deployment: Vercel for frontend hosting, Railway for backend WebSocket server

Key Technical Components:

  1. Context Manager: Maintains per-room conversation history, user presence tracking, and rolling group summaries to stay within token limits

  2. Response Scheduler: Implements smart turn-taking heuristics that analyze message patterns to decide when the AI should respond without interrupting human conversations

  3. Streaming Pipeline: Server-Sent Events (SSE) parser that receives tokens from the JLLM API and broadcasts them to all connected users in real-time

  4. Rate Limiting: In-memory per-socket rate limiter prevents spam and ensures fair usage

  5. CORS Configuration: Properly configured cross-origin resource sharing to allow Vercel frontend to communicate with Railway backend

😅 Challenges we ran into

1. WebSocket Deployment Complexity
Initially tried Google Cloud Run, but discovered it doesn't handle persistent WebSocket connections well across multiple instances. Had to pivot to Railway, which natively supports long-lived WebSocket connections.

2. Multi-Origin CORS Issues
Getting the frontend (Vercel) to connect to the backend (Railway) required careful CORS whitelisting for both development and production environments. Took several iterations to get Socket.io's dual CORS config (Express + Socket.io) working correctly.

3. Context Window Management
With 25k tokens available, we had to design smart summarization strategies to prevent context overflow during long conversations with many participants. Implemented async room summarization that runs in the background without blocking responses.

4. AI Turn-Taking Logic
Finding the right balance for when the AI should respond was tricky. Too aggressive = interrupts humans; too passive = feels unresponsive. Ended up implementing multiple trigger conditions (@mentions, silence thresholds, message counts) to make it feel natural.

5. Real-Time Streaming Synchronization
Ensuring all users see the same AI response streaming simultaneously required careful event orchestration—we emit tokens as they arrive but also send a final "done" event with the complete text to handle any dropped packets.

🎉 Accomplishments that we're proud of

✅ It Actually Works in Production! Multiple users can join from anywhere in the world and have coherent group conversations with AI

✅ Smart Context Management - The AI remembers who said what and references previous messages accurately across users

✅ Real-Time Streaming - Responses appear token-by-token for all users simultaneously, creating an engaging "thinking out loud" experience

✅ Clean Architecture - Separation of concerns with dedicated services for context, scheduling, and streaming makes the codebase maintainable

✅ Deployed and Scalable - Using modern cloud infrastructure (Vercel + Railway) means we can handle multiple concurrent rooms without manual server management

✅ User Experience Polish - Typing indicators, presence tracking, emoji reactions, and connection status all contribute to a polished chat experience

📚 What we learned

Technical Skills:

  • How to properly configure WebSocket servers for production deployment
  • Advanced Socket.io patterns for room-based broadcasting and presence management
  • Strategies for managing large LLM context windows efficiently
  • Server-Sent Events (SSE) parsing for streaming API responses
  • Cross-origin resource sharing (CORS) for full-stack deployments

Product Thinking:

  • The importance of AI "turn-taking" in group conversations—it's not just about answering questions, but knowing when to stay silent
  • User experience details (typing indicators, streaming responses) dramatically improve the perceived intelligence of the AI
  • Designing for multi-user scenarios requires fundamentally different prompting strategies than 1-on-1 chatbots

Deployment Lessons:

  • Not all cloud platforms handle WebSockets equally—choosing the right hosting matters
  • Environment variable management across platforms is crucial for cross-origin communication
  • Rate limiting and graceful degradation are essential for public-facing real-time apps

🚀 What's next for MultiChat AI

Voice Integration
Add LiveKit SDK to enable voice channels where users can speak with the AI, creating a Discord-like experience with intelligent voice responses using text-to-speech.

Persistent User Profiles
Implement user accounts so the AI remembers individuals across sessions—their interests, conversation history, and preferences.

Room-Specific AI Personalities
Allow users to create custom rooms with different AI characters (e.g., "Study Buddy," "Creative Brainstormer," "Code Reviewer") each with specialized system prompts.

Enhanced Context Strategies
Integrate vector databases (like Chroma or Pinecone) for semantic search over conversation history, enabling the AI to recall relevant past discussions even from weeks ago.

Analytics Dashboard
Build admin tools to visualize room activity, popular conversation topics, and AI response quality metrics to continuously improve the prompting strategies.

Mobile App
Create native iOS/Android apps using React Native to bring the multiplayer AI chat experience to mobile devices with push notifications for mentions.

Enterprise Use Cases
Develop features for businesses: customer support rooms where AI assists multiple agents, virtual workshops with AI facilitators, and collaborative research sessions.

GitHub: https://github.com/KillMonga130/janitor-ai-multichat Live Demo:https://2man.vercel.app/ Category: Computer Science + AI (Coding Track)

Built With

Share this project:

Updates