Wallscapes

Rediscover your city, as AR-powered muraling reimagines every building and street corner, uniting communities through public, local art. See artists work in real time and connect with your community.

Comment

Inspiration

As much as the internet has promised to bring us closer, the sad truth is that it has more often made us feel lost and isolated, disconnected from those around us. During COVID, for many people our age, it brought us to the edge. Inspiration struck from right here in Seattle. From the thin alleys of Chinatown to the rooftops aglow in the UDistrict's night skyline, muraling reaches deep into the heart of our communities. Murals convey history and culture, are events for congregation, and enhance the vibrancy of the neighborhood. We challenged ourselves to breathe into this art form, which has told our stories since the dawn of humankind, a new life.

What it does

Wallscapes uses cutting edge Augmented Reality to provide a live platform for creating murals on vertical surfaces anywhere in the world. For artists, it is as simple as scanning a wall with their camera--our application determines the wall's features, and creates a canvas of equal size on their device, on which to create beautiful, layered art pieces. Neighbors that encounter this local artist at work can point their phones to the surface and see the mural being painted in real-time, or share their love for the work through likes and comments. Occlusion technology accurately handles blocking objects, people, and renders lighting to match the environment perfectly.

How we built it

The app is built with Swift for iOS, and uses the ARKit 6 library for Augmented Reality. A back-end API was created with Express and MongoDB to handle user, mural, and comment models. Murals are handled as many strokes rendered into a vector graphics image with some position and rotation, which are published via Websockets to all nearby observers for real-time updates.

Challenges we ran into

The most interesting challenge that we ran into was how to localize different users, despite beginning their AR sessions in different locations. If the two users do not agree on a coordinate system, then the murals will be in the wrong location, either floating off in space or clipping behind a wall. The solution that we implemented was to generate a 3d map of the space that is being muraled, which persists in the database. New users fetch the map, and are localized by matching the map to the points in their field of view. Within a few brief moments, other users are initiated in the artist's coordinate system, allowing them to view the mural as it was intended.

Accomplishments that we're proud of

We are proud of our willingness to take the risk of attempting to fulfill a complex vision. Before we began, we grappled with ideas ranging from hours to days, but concluded that a 24 hour hackathon demanded risk-taking. Despite our inexperience with mobile development and AR development, we believed the project would be a push outside what is expected, and for this, we are proud that we managed to stay organized, optimistic, and create a product that we believe is now worth more than just vision.

What we learned

Each of us learned a number of hard skills: all of us learned mobile development and using the iOS development kits; those of us worked on the backend learned to implement Websockets and integrate them into a Swift project; another half of us worked on the AR functionality, and gained thorough insight into Apple's ARKit, the fundamentals behind AR tech, and the challenges that still face this emerging industry. Most importantly, we learned to grapple with the challenges of a time-constrained environment. Scoping was one of our team's key focuses, especially for a project this big, and it made us decide what our product needed, as opposed to what we wanted.

What's next for Wallscapes

In the future, we hope to add the additional features that we were unable to implement during the hackathon due to time restrictions. For example, we wanted to flesh out the AR user interaction by allowing for the section of a wall on which they wanted to paint their mural. Furthermore, we originally planned for even more community by allowing other users to comment and like an artist's mural, or even for multiple artists to draw murals that interacted with each other on the same wall.

Built With

Share this project:

Updates