Inspiration
Our inspiration is grounded in the lack of accessible, cost-effective, and easy-to-use solutions for converting 2D images to 3D models. We realized that existing technologies for creating 3D models are limited and expensive, thereby discouraging the average person from utilizing them. AI-dar was created to make 3D model creation more accessible and convenient for everyone.
What it does
AI-dar is a groundbreaking tool that converts objects from any image into 3D models. With a simple picture of any object, AI-dar generates a 3D model asset that can be modified and used in various fields such as fashion design, game development, virtual reality chat, animation, and architecture with support for popular graphics and design software including Blender and Unity. It's revolutionizing how 3D models are created by allowing the average person to easily and quickly convert any object from their surroundings into a 3D model.
How we built it
We have employed cutting-edge AI technologies including CLIP, Point-E and DALL-E, to analyze images and isolate individual objects within them. This isolated data is then used to create 3D models of each object. AI-dar integrates these technologies seamlessly to offer a user-friendly tool for generating 3D models from 2D images.
The pipeline from prompt to generation begins with a set of 2D images of a still object. We use Point-E first to lift point clouds out of the 2D images, and also their built-in methods to convert the pointcloud to mesh with triangulation and normals as a PLY file; the compute for this is hosted on a single EC2 A100 instance. The PLY file is then edited in Blender with materials and physics.
When a final .blend file is created, it is online-converted to a .glb file, and then loaded as a Three.js component for interactive web app render.
Challenges we ran into
The integration of multiple AI technologies into a unified, user-friendly system was a significant challenge. Creating a system that is capable of isolating objects in images and generating accurate 3D models also proved to be a difficult task.
There were many AI tools to our disposal, each one with very different features and APIs, so we had to swiftly read through and experiment with all of them. We had to be very wise managing time around blockers, including finding the right tools and workarounds to triangulate/normal a pointcloud to a mesh before we could do any Blender processing.
Accomplishments that we're proud of
We are incredibly proud of developing a tool that democratizes the process of 3D model creation. With AI-dar, anyone can take a picture and convert it into a 3D model, effectively reducing costs and time invested in the traditional 3D model creation process. Furthermore, we're pleased to have developed a solution that has potential applications across various sectors, including fashion design, game development, and architecture.
What we learned
The project taught us the importance of initial planning and the value of having a clear roadmap to follow. Having specific, time-bound goals at each stage of the project was a critical factor that contributed to our success.
What's next for AI-dar
We plan on leveraging more AI technologies to improve the detailing of the 3D models created by AI-dar. Furthermore, we are also looking into implementing a feature that can convert 2D videos into 3D animations. Every step in the pipeline that we implemented has potential to be automated to create a simple, seamless experience for a general user to upload images/prompts and create 3-dimensional models imbued with interaction and physics.
From a physics standpoint, we also want to explore the potential for LLMs themselves to generate the automation scripts and filters for Blender models. Our use of cloth physics was specifically for garment generation, but different Blender features can apply to different models.
Log in or sign up for Devpost to join the conversation.