Spatial Jam | Devpost

Jam Session Present
Virtual Instrument: Drums
Virtual Instrument: Piano
Jamming!
Selecting Virtual Instrument

Inspiration

Some of the greatest hits we know today started with everyday sounds, such as a beat someone stomped to or a melody hummed in a moment of inspiration. We wanted to capture that magic. With Spatial Jam, we’re turning everyday spaces into shared musical moments where anyone can leave a piece of audio for others to build on. It’s all about making music creation simple, playful, and truly collaborative because great music can come from anyone, anywhere.

What it does

Spatial Jam allows users to see the real world blended with virtual instruments and music loops through AR. When inspiration hits, whether it’s a beat, melody, or spontaneous sound, users can record and leave audio in the environment, fostering a collaborative jam session. Passersby can listen to what’s been left and add their own layer. Whether using virtual instruments, their voice, or anything nearby, users can contribute a musical phrase to the growing loop. Over time, each location becomes a unique, community-created chorus where anyone can leave their mark and join the music.

How we built it

We began by brainstorming in FigJam, exploring different ideas before narrowing in on Melody Mash. Once we aligned on the concept, we set key milestones and defined our MVP to guide progress and pacing. We then broke the system into core components: AR interaction, spatial audio, loop recording, and spatial persistence. Each team member took the lead on different modules while collaborating closely throughout the process. Our programmers focused on API implementation and Typescript development, while our designers used Figma for UI design and Blender for 3D assets. We developed and tested each module in parallel before bringing everything together into a cohesive AR music experience using Lens Studio and the Spectacles.

Challenges we ran into

Our project was built in Lens Studio using Typescript, which was new to all team members. There was a steep learning curve at the start as we worked to get familiar with the workspace, and even with our Unity experience, it still took some time to get up to speed. Integrating the different modules into a cohesive experience required multiple iterations, since each component had to work seamlessly with the others. Additionally, some plugins, like Figma, are not compatible with Spectacles, so we had to carefully navigate what tools and features were native to Spectacles versus Snapchat.

Accomplishments that we're proud of

We’re proud to have built a working MVP within the limited hackathon timeframe, integrating AR interactions, spatial audio, and loop recording into a cohesive experience, which is no small feat for a timed project. The team learned how to use Lens Studio for the first time and developed a project using Snap’s latest technology, while designing with the intention of fostering social interactions through everyday objects and spaces. On top of that, many of us were participating in a hackathon for the first time and still managed to contribute meaningfully to an ambitious project.

What we learned

We learned the importance of quickly defining a minimum viable product (MVP) at the start, which gave us a clear goal and helped guide our design and implementation choices before diving into development. In addition, we learned to use Lens Studio for the first time, explored new developments in the AR and MR community, and got hands-on experience working with Spectacles for our project. This experience gave us a glimpse into the future of the XR community and the exciting possibilities that lie ahead.

What's next for Spatial Jam

We plan to explore AI-generated music as an option for users to add to their audio tracks. In addition, we would consider future improvements, including supporting more simultaneous loops, enhancing music mixing, and expanding the variety of virtual instruments to make the collaborative experience even richer. We also want to integrate waypoints for jam sessions, with the goal of placing them in locations that encourage people to get out, support small businesses, and interact with fellow jammers.