Raidar

Raidar is an augmented reality (AR) platform developed for cyclists. It is an integrated bicycle helmet that provides enhanced awareness of user surroundings and biometric data in real time.

Comment

Inspiration

Raidar was developed as a tool to help cyclists navigate the 21st century. We blended android powered XReal glasses, Orbec monocular camera, and ONNX-yolov8 object detection into a bike helmet to help cyclists tackle 3 distinct problems: 🔦Poor Visibility: a "second set of eyes" to see objects, know what, and how close they are (in daytime or nighttime). 🌏Confusion: project map route and directions to ease navigation burden. 💕 Engage: an immersive display of key biometrics such as heart rate/EKG. We envision Raidar becoming a fully integrated helmet with lightweight XReal glasses that can serve as the ✨ gold standard of cyclist style and performance.

What it does

Raidar is an augmented reality (AR) platform that integrates XReal glasses and Orbec monocular camera mounted on a bicycle helmet. We embedded ONNX-Yolov8-object-detection to track and identify objects in the scene. Our algorithm is capable of classifying over 200 distinct objects in real-time. We anchored EKG/heart rate, time, speed, metrics onto our AR display. We used IMU sensor data from the Android cell phone to estimate speed of the user in real-time.

How we built it

The hardware for RAIDAR consists of a helmet with Orbec camera that was attached with a drilled mount. XReal glasses and android cell phone were connected in the ecosystem with a wired connection. We optimized the user experience by working on UI/UX concurrent with user feedback. We tested Pico4, Lenovo A3, and XReal glasses for portability and function. We worked with different system configurations while creating our demo.

Challenges we ran into

Making a real-time compliant detection pipeline was challenging at first. Another issue we faced was recording our live demo screen output on XReal glasses. This issue was resolved using XReal's RGB camera sample code (C#). We also spent time optimizing the hardware set-up to reduce weight and maintain the balance of the helmet.

Accomplishments that we're proud of

We are proud of our achievement bringing RAIDAR to life and the fact we built a user-friendly AR experience for cyclists. We enjoyed working on a diverse team and found it fulfilling to learn about each member's expertise. This was the first hackathon for at least one member of the team and also the only time other members tackled the hardware track.

What we learned

During the prototype phase of our project, we gained valuable experience working on integration with multiple hardware components, image processing pipelines, and UI/UX. We demonstrated how to apply Orbec monocular camera, android powered XReal glasses, and object identification algorithm(s) in cyclist applications for the first time. Using ONNX-Yolov8 for real-time object detection was a challenge, but we successfully implemented it in the Raidar helmet.

What's next for RAIDAR

In the future, we plan to build on the features we developed in this early-stage prototype. In particular, we would like to integrate real sensors into our ecosystem including thin wearables for heart rate/respiration to fully characterize biometric data. We would like to incorporate a proximity sensor to alert the user to unseen objects to reduce safety hazards (3D mapping and localization with SLAM a bonus). We would also like to miniaturize the hardware and fully embed our system into existing high-quality bicycle helmets such as ScorpionExo or Bontrager through brand partnership.

👪 We are also exploring adding LLM (Chat GPT) for navigation query by the user with whisper.ai and overlay Google Map.

Built With

Share this project:

Updates