Block Designer | Devpost

Inspiration

We were inspired by the need for designers to better visualize their creations and make design choices more interactive. Traditional drawing methods don't always convey a clear sense of depth, so we wanted to create a tool that would let users translate their 2D sketches into interactive 3D models.

What it does

Block Designer allows users to take a picture of their hand-drawn sketches, automatically converts them into a 3D model, and then uses augmented reality (AR) to display the model in their physical environment. This helps the user better understand the design in three dimensions and visualize how it would look in real life.

How we built it

We used Next.js to create an interface and app that dynamically loads the Google Model Viewer library, which is used to display the 3D models. On the backend, we built an AI-powered Python script to convert the 2D sketches into 3D models through computer vision. Finally, for augmented reality, we used Google's Model Viewer library, which supports both WebXR and QuickLook (iOS-specific AR). This allows users to place the 3D model in the real world using their phone’s camera.

Challenges we ran into

We faced several challenges during development, including issues with package downloads and managing dependencies. Slow downloads and timeouts from the package store required us to adjust network settings and use alternative registries. Additionally, Python dependency conflicts made it difficult to set up a stable environment for the AI-driven 3D model generation, leading to extra time spent on managing virtual environments. We also encountered difficulties when switching from A-Frame and AR.js to Google Model Viewer for AR, as it required rebuilding parts of the AR implementation. While the transition delayed progress, it ultimately provided better integration.

Accomplishments that we're proud of

We’re proud that we were able to create a working prototype that can accurately transform hand-drawn designs into 3D models for the timeframe given.

What we learned

We learned how to integrate 3D models with augmented reality (AR) for both web and phone platforms. We also worked with camera applications for real-time AR interactions and focused on web and phone integration for consistent performance. Additionally, we used Visual Studio Code Live Share for collaboration, allowing us to share code and work together remotely.

What's next for Block Designer

If we worked on this project longer, we would add more user design options and create a more seamless interface connection. We would expand the AR features to include interactive feedback. Additionally, we are working on making the app more accessible for different types of users and devices.