ORA

• 

  User interface with an opened chatbox and the vital chart at the top left corner

Inspiration

Amidst the escalating wildfire crisis, firefighters confront intense stress and critical decision-making challenges. In the United States alone, an average of 2,770 civilian fire fatalities occur annually, with almost half of firefighter injuries happening on the fireground.

Key challenges hindering efficiency include the necessity for swift life-and-death decisions under extreme stress, vision impairment due to smoke, and elevated risks of high blood pressure and cardiac events.

In response, we've developed ORA: an innovative AR goggle designed specifically for firefighters. ORA addresses these challenges by enhancing decision-making clarity, improving visibility in smoke-filled environments, and monitoring vital signs to mitigate health risks. This groundbreaking tool is poised to revolutionize firefighting, enhancing both safety and effectiveness in battling blazes.

What it does

ORA revolutionizes firefighting through its advanced infrared light mapping technology, enabling firefighters to clearly map their surroundings. This feature cuts through the visual chaos of fire and smoke, providing distinct outlines of the environment. This clarity accelerates scene analysis and reduces decision-making time significantly.

Additionally, ORA integrates a sophisticated hand gesture detection and recognition system. In the chaotic and often noisy environment of a fireground, clear communication is crucial. ORA's innovative system allows firefighters to communicate effortlessly through designated hand signals, opening and closing a chatbot on command, ensuring effective communication.

Crucially, ORA prioritizes firefighter health with its constant health-monitoring display, strategically positioned in the user's peripheral vision. This real-time health chart keeps track of vital signs, alerting firefighters to any concerning changes. This monitoring is pivotal in preventing long-term health impacts such as heart attacks, ensuring that firefighters remain acutely aware of their physical state during intense rescue operations.

How we built it

ORA's development journey is a fusion of cutting-edge technology and innovative design. We harnessed the power of Oculus for its superior AR capabilities, ensuring an responsive user experience. This provided the robust foundation necessary for the intricate simulation and visualization demands of firefighting scenarios.

Further augmenting ORA's functionality, we integrated UltraLeap's advanced hand tracking technology. This inclusion was pivotal, enabling us to create a highly intuitive and natural interface for gesture-based communication and control. UltraLeap's precision in hand gesture recognition allowed us to design a system that is both user-friendly and reliable under the extreme conditions of firefighting.

Combining Oculus's AR prowess with UltraLeap's gesture tracking expertise, we crafted ORA into a tool that is not only technologically advanced but also attuned to the practical needs and challenges of firefighting.

Challenges we ran into

Our initial hurdle was navigating Unity software, a platform unfamiliar to our team, and posed a significant challenge in bringing our ideas to fruition. To tackle this, we strategically divided our team into specialized groups, each focusing on different aspects of this new technological scope. While learning curve was steep, we accelerated our progress through various resources: delving into YouTube tutorials, seeking guidance from hackathon sponsors, and drawing insights from mentors. These efforts substantially enhanced our understanding and application skills.

Another major challenge we faced was developing a dynamic wall mesh that adapts to real-time user movements. We overcame this by leveraging a key feature in the MetaQuest 3 regarding room scanning capabilities. This feature allowed us to pre-scan the environment, mirroring our idea of real-life infrared scanning. By doing so, we were able to generate precise colliders along environmental surfaces, ensuring accurate and responsive interaction within the AR space.

Accomplishments that we're proud of

At the hackathon, our excitement for the potential of AR was high, despite our limited experience with the technology. Embracing the challenge, we quickly learned Oculus and UltraLeap applications within limited time to create ORA. We are especially proud of and excited to have ORA offering a potential solution to reduce casualties in a local area like California, where wildfires have almost become an annual experience around the Bay Area. This hackathon helps us to harness the power of technology in solving real-world problems while adapting in unfamiliar spaces.

What we learned

Participating in this XR hackathon is an enriching experience that offers a myriad of learning opportunities. First, this hackathon creates an environment that fosters teamwork. During this short two days journey, our team created an excellent chemistry as we collaborate closely, pooling our diverse skills and perspectives to create innovative solutions. It is a hands-on immersion in cutting-edge technology, allowing participants to explore and experiment with the latest advancements in VR and AR. Additionally, we were able to utilize such technologies and learn how to edit, code, and create our own projects that fulfills a dream we all strive to fulfill (in our case, it is helping first responders like firefighters).

What's next for Ora

In our future roadmap, ORA will integrate enhanced navigation and infrared detection, further improving spatial awareness and maneuvering in high-risk areas. We are also expanding hand gesture communication for more intuitive interactions. A significant upcoming feature is a multiplayer mode, enabling real-time location sharing and communication among firefighters through a bird's eye view mini-map, enhancing team coordination and operational efficiency in critical scenarios.

Built With

• c#
• oculus
• quest3
• ultraleap
• unity