CultureScape

• 

  A "capture bubble" that can be popped to reveal the elements inside

• 

  A sample from the Puerto Rican capsule, featuring poet Luis Lloréns Torres

• 

  A sample from the United States capsule, featuring the Statue of Liberty

• 

  A sample from the India capsule, featuring a life-size temple

Inspiration

In high school, many of us found world history classes to be distant and text-heavy, lacking the vibrancy of real cultures and events. Recognizing our diverse backgrounds, we set out to design an educational experience that immerses students in history, bridging the gap between traditional textbooks and the dynamic, interconnected world they represent.

What it does

CultureScape is a mixed-reality (MR) experience that enables its users to interact with a virtual globe that sits on top of their desk. The desk features many different "bubbles" with flags of the culture they represent. When the user taps on a bubble, it raises and enlarges so that they can inspect the flag in greater detail. If the user chooses to "pop" it, the bubble presents the cultural content inside.

How we built it

Given our diverse cultural backgrounds, we combined our knowledge to create three separate "culture capsules;" one for the United States, one for Puerto Rico, and one for India. We then utilized Meta's Building Blocks and interaction tools in their all-in-one SDK to quickly build and iterate our prototype. We used ShaderGraph to create the "bubble" effect (so that it displays a flag and wobbles like a bubble), and modified a poke intractable so that user can "pop" the bubbles. We then gathered content from free assets and samples from platforms like Sketchfab, the Unity Asset Store, and Freesound.org.

Challenges we ran into

We wanted to have a LLM-powered A.I. character serve as an intractable guide so that the user could ask questions about the culture. However, we realized that getting an LLM to run on-device was tricky since we never had tried it before, and we had to pivot to other areas of the project due to time. However, we learned a valuable lesson of time management and project scope and expectations.

Accomplishments that we're proud of

We are particularly proud of the teamwork that lead to the creation of this app. We feel that our different backgrounds allowed us to approach the design of the application in different ways. And of course, having different backgrounds of culture led us to create a more meaningful and hands-on application that each team member contributed towards.

What we learned

We helped each other to teach Unity, Git, some basic Blender, and more. Although we weren't able to integrate the A.I. assistant into our application, we learned a bit more about the packages that exist to integrate LLMs and other A.I. tools. We also got more practice with animations in Unity.

What's next for CultureScape

We believe that future iterations of this application could incorporate an A.I. character assistant to serve as an interactive guide for the culture. We would also like to add more cultures, and perhaps use more of Meta's mixed-reality technologies to further integrate the culture into the user's space; for example, we could replace the art on one's walls with art from the culture using the Scene Understanding tools.

Built With

• c#
• oculusinteractionsdk
• photogrammetry
• polycam
• shadergraph
• unity