Inspiration
The MSU active shooting incident left a deep impact on our team, prompting us to re-evaluate the effectiveness of our current systems for handling such devastating events. We recognized that existing systems often result in misinformation, distrust, and inefficiency, which can further exacerbate the impact of such events, particularly on people with disabilities. To address these challenges, our team sought to develop a new system that makes indoor localization and evacuation more accessible for individuals with limited sight or mobility.
What it does
One of the main features of Quick Action is the ability to localize the user in a room, and create a custom AR map to the nearest exit of the building. This system was designed to be extremely accessible to all users, especially the vision impaired. Our app guides the user through the environment through detailed spoken instructions and visual cues based on the user's current position.
Our team also implemented a fully-integrated person-counting system for lockdown situations. In this system, users are able to report specific locations that they are currently sheltering in place in which can be used as extra information by law enforcement. Users are able to "panic" which signals the 911 dispatcher that they are in distress. Additionally, a backend was fully implemented that allows the 911 dispatcher to sort and signal groups of people with important rescue information.
How we built it
Our team has always dreamed of creating a full-stack Augmented Reality application. However, to make that dream come true over the past 36 hours, our team members have been having significantly fewer dreams than usual.
We built our system using a combination of powerful tools and technologies to create a seamless user experience. For the user side of our system, we used Swift and Xcode to build a mobile app with a beautiful user interface that makes it easy to navigate and use. We also used Firestore, a cloud-based NoSQL database from Google, to store and retrieve data in real-time. This allows our app to update and sync with the latest information, ensuring accurate and up-to-date data. On the operator side, we used React and Express to build a web-based platform that enables emergency operators to receive updates and communicate with users in distress. By combining these technologies, we were able to create a powerful system that is both robust and reliable.
We also integrated Twilio, a cloud communications platform, to enable communication between the users and the emergency operators. This feature ensures that users in distress can quickly and easily communicate with the operators, who can then provide them with instructions and assistance.
In addition, we utilized ARKit, Apple's augmented reality (AR) framework, to enable users to navigate in the real world. By leveraging AR technology, our app provides users with real-directional cues and landmarks to guide them to safety. This feature is particularly important for users with limited visibility, as it makes it easier for them to navigate their surroundings and find the nearest exit.
Our team embarked on an exciting journey with the latest technology, ARKit from Apple, to build our innovative project. Despite being a new venture for our team, we eagerly dived in and quickly became experts in AR, pushing the limits of what we could do.
Challenges we ran into
There were plenty of challenges that our team faced during this project. Our team typically found that when integrating our changes together, small issues would present themselves that would consume a large amount of time (merge conflicts, tricky bugs, etc). On a few occasions, we got to play detective. trying to figure out what specific event caused a bug to be exposed in our code. This process was very enlightening and fun! (more so when you find the solution.)
Our team found a significant challenge in adapting to the ARKit framework that we did not have much experience in. The documentation for the software was decent, however, the amount of examples that our team could find online made it more of a challenge.
Accomplishments that we're proud of
One of our biggest accomplishments in this project is the focus and success of our product for accessibility in indoor navigation. Through extensive testing and development, our team honed in on a ARKit solution with incredibly high fidelity and accuracy at navigating users through indoor environments by visual cues and speech guidance. This component of making technology more accessible for everyone (especially in evacuation/disaster situations) is one that is largely overlooked and we believe that technology can be leveraged in this area to make people's lives easier.
Another accomplishment that we are really proud of is our method of transferring important data from the users (through reports) to the 911 dispatch operators. Through the use of Firebase, our team was able to login, make requests, store user information, and instantly integrate that data into a live dashboard that the dispatcher can take advantage of. This systematic and reliable process gives us hope that a system like ours can be used to prevent some level of danger in future disastrous scenarios.
What we learned
Our team had no shortage of interesting frameworks to learn or problems to tackle. In fact, some of our members were introduced into IOS development for the first time this weekend! Additionally, our team made strides in git version control, and a plethora of new frameworks needed to create a full-stack application.
What's next for Quick Action
Despite our achievements in just 36 hours, we are committed to continuously enhancing and expanding the capabilities of our Quick Action app. Here are some of our exciting ideas for future developments:
More robust evacuation plans: We aim to enable the app to be used for various types of emergencies, including fires, earthquakes, and active shooter situations.
Hazard reporting: With our app, users will be able to report hazards and safety concerns in real time, providing a quick and efficient way to alert emergency responders and other users to potential risks. This could help prevent accidents before they occur.
Including more accessibility features: Our goal is to make the app more accessible to all, with multilingual support and other features that cater to individuals with disabilities. This will ensure that everyone can use our app, regardless of their abilities.
Making indoor tracking more robust: While vision-based tracking has served us well, we plan to explore additional technologies, such as WiFi-based positioning, to enhance our app's indoor tracking capabilities. This will provide reliable location information even when vision-based tracking is not sufficient.
Privacy and security features: We recognize that user privacy and data security are top priorities. Our app will include robust privacy and security features to protect user data and ensure that sensitive information is only accessible to authorized personnel.
Expanding indoor mapping: We can further help students, faculty and visitors navigate a building easily by mapping out more spaces.
We are excited to work on these new developments, and we believe they will take Quick Action to the next level, providing an even better and safer experience for our users.
The business plan moving forward
Our vision is to empower and provide safety to individuals with vision impairments and the wider community during emergency situations such as fires or shooter threats. Our team is passionate about the potential of this app to make a significant positive impact on society thereby we will move forward as a sustainable non-profit organization. we are committed to ensuring that our emergency navigation and notification app is accessible and affordable for everyone.
To achieve this goal, we plan to explore various funding opportunities, including grants and donations from government agencies, private foundations, and individual donors who share our mission of providing safety and inclusivity for all. Additionally, we will seek out corporate sponsorships to help us reduce backend costs and maximize the impact of our app, and we believe that with the right funding and support, we can bring it to life. Together, we can create a safer and more inclusive community, where everyone feels empowered to navigate emergency situations with confidence and ease.
#SpartanStrong 💚 🤍
Log in or sign up for Devpost to join the conversation.