Veyesor

Drive your vehicle, unobstructed.

Comment

Inspiration

On the way to McHacks, we travelled by car and encountered quite a bit of traffic. Not to mention, the front and rear windshields were extremely snowy/icy at first, and our car was full of passengers, travelling supplies, luggage, and other miscellaneous items that obscured Alex’s (our driver’s) view of his surroundings. This made for a bit of an unsafe trip, and inspired us to create Veyesor—a VR-powered application that enables drivers to obtain a free, unobstructed view of their vehicle’s surroundings, as if they weren’t even inside the car at all.

What it does

You put on the VR headset, and you are shown a real-time, 180-degree view of your car’s surroundings—powered by 5 iPhone cameras on the outside of your vehicle.

How we built it

  • Python + OpenCV for backend, stitching together our different camera views
  • WebRTC for streaming the stitched video
  • NextJS + Typescript for the web application
  • Vercel for hosting the web app
  • Oculus Quest and Unity for rendering in VR

Challenges we ran into

  • Searching for an app to hook up a phone camera to the laptop to connect the camera feed
  • Connecting the video feed of two phones together
  • Finding a suitable way to stitch the videos together
  • Deciphering the underlying linear algebra used to connect the camera feeds together
  • Grueling over Unity development for 4+ hours
  • Figuring out how to pay a parking ticket in Montreal

Accomplishments that we're proud of

  • We were able to develop and configure an OpenCV project for live panoramic video stitching, which is generally regarded as a difficult project. Many available online resources are research papers on making this work, but there were very few code implementations of them.
  • The project encompases a plethora of software development topics. Including networking, computer vision, pipelines to connect the front-end to back-end to utilize the computer vision model, and accessing it on the web through the VR headset.
  • Winning the Super Smash Bros tournament!

What we learned

  • Don’t visit Queens and McGill in the same weekend...
  • Stitching together videos into a 3-D live stream is hard
  • VR development is fun!

What's next for Veyesor

  • Creating an fully immersive 360-degree camera view, and a more integrated camera build with the car we attach it to, creating a full build.
  • Developing the VR app to have a dashboard with the local speed limit, traffic signs, current speed, time, temperature, etc.
  • Map view: shows you the route that you should take to get to your destination, in AR, on the road in front of you!
Share this project:

Updates