Inspiration
We love platforms like Character.ai, where people can chat with virtual characters. But what if these characters weren't just confined to text? We wanted to make virtual characters you could actually see, interact with, and bring into your world as interactive 3D beings.
What it does
Embodi transforms any image or sketch into an interactive, lifelike 3D character. Beyond just a static 3D model, our characters come alive, you can talk to them, ask them questions, and engage with them just like you would a friend. It’s like Character.ai, but visual, immersive, and far more interactive.
How we built it
We combined TripoSR to convert uploaded images into detailed 3D models, integrating OpenAI's Whisper for speech-to-text and Google's Gemini for natural language understanding and conversation. For the frontend, we used React and Three.js to provide seamless, interactive experiences. Everything runs smoothly in the cloud with GPU acceleration, allowing rapid and responsive interactions.
Challenges we ran into
Ensuring seamless integration of speech recognition, conversational AI, and real-time 3D rendering was challenging. Handling tensor shapes correctly was another significant obstacle, as was creating smooth, realistic conversations with AI-generated 3D characters without noticeable latency.
Accomplishments that we're proud of
We're thrilled we created a fully interactive pipeline, going from a single uploaded image to a fully conversational and interactive 3D character. Users can now experience AI characters visually and conversationally, breaking the boundary between traditional chatbots and immersive experiences.
What we learned
We deepened our skills in full-stack AI integration, learned the intricacies of managing multiple complex APIs simultaneously, and refined our understanding of real-time 3D rendering and conversational AI nuances.
What's next for Embodi
We plan to enhance interactions further by adding gesture recognition, facial expressions, and emotions, making characters feel even more real. Eventually, we hope to integrate AR so these characters can interact directly in your real-world space.
Log in or sign up for Devpost to join the conversation.