A "Fire" Training App

Inspiration

Fire safety training is not done that frequently or only with younger children. In the event of a disaster, many people can forget the safest steps to take if they even learned them at all. Our goal is to help save lives by making this training more memorable and accessible to anyone with a headset.
We came in knowing we wanted to do something with newly released Meta Quest 3 SDK features like Inside-Out Body Tracking (IOBT), or the Depth API that uses the depth sensor in the headset to cut out and display real-world objects on top of virtual objects naturally if they're in front of them.
We originally had ideas for some sort of echolocation or radar navigation in your real space with the Meta Haptics Studio as a sort of hi-tech walking stick, but we didn't feel there was a problem being solved or a clear use-case. We also realized that there is a limitation with the Depth API, which is that it only returns whether a object is in front or behind reality, not a full depth buffer image.

It's an educational fire escape safety training app that lets you practice escaping your own home with helpful tips and fire and smoke that is procedurally placed throughout your space.
After the user sets up their room and marks out doors with Meta's scene creation tool, they're presented with their room covered in smoke and procedurally placed fire to navigate through. On their wrist, they can see their "survivability score" that goes down from 100% based on how long it takes for them to escape through the mixed-reality-marked exit sign. The wrist UI also shows safety tip notifications based on what they're doing in the scene, like "Look for the exit!" If they make it out before their survivability reaches 0%, the experience ends and they're presented with a winning panel.
It uses a never-before-seen mixed reality smoke effect that limits visibility even a couple meters away, while letting you see your hands and nearby objects in front of you.
This new type of real-world accessible immersive safety training has the potential to save many lives in the future.

It took a a good portion of our time troubleshooting the behavior of the Scene API to make it work for our needs.
Creating the shaders to be compatible with the Depth API and have everything render in the correct order.

This was many of our first times making a mixed reality app and making custom shaders for a Unity project.
We were able thoroughly understand the difference between the capabilities of the Scene API and Depth API and how to most effectively use both in tandem.

We learned a lot about how to make apps effectively with many Meta Interaction SDK features we haven't worked with before, like Building Blocks, Depth API, Scene API, mixed reality design, and how to work with multiple transparent objects at once.

We'd like to add more fire escape safety tips and features:

A "home checkup" that helps you identify and remember all of the smoke alarms in your home and reminds you to replace their batteries.
More fire safety tips like "Stop, drop, and roll," "Get out, stay out, and call for help."
More features and interactive elements like having closed virtual doors with fire underneath that remind the user to not touch door handles in case they could be burning hot.
Adding randomly placed debris that could block paths in addition to fire.
Tips to improving your survivability if you stay lower to the ground to avoid inhaling smoke.

I was the team lead and was responsible for some of the brainstorming and vision of the project. I also set up the project with Meta's Depth, Scene, and hand tracking API's and did the majority of reading documentation to understand, troubleshoot, and communicate to the team how to work with them and their limitations. With all that in consideration, DJ was still the MVP and everyone on the team contributed a significant amount, so I am incredibly grateful for that.

Jack English
Chase Johnson
D.J. Figueroa
Ian Morrill
Samuel Wiggins