OneShot Vision | Devpost

Inspiration

As anxious coders deprived of typical social interaction, networking events are our worst nightmare. We wanted to hack a way for us to instantly learn about the people we're meeting and break the ice as fast as possible.

What it does

OneShot Vision uses our own one-shot face detection machine-learning algorithm, and our custom-built raspberry pi glasses to recognize and match a face to our database in real-time. We then pull up their LinkedIn information to appear on a small OLED screen in front of your eyes.

How we built it

We built a headset that consists of a raspberry pi, a pair of glasses, an OLED screen, and a camera to process a feed of live image data. Using hard cascade image detection we’re able to isolate the face, and send it into our own ML model for one-shot facial detection.

We utilized siamese neural networks to make the one-shot face detection algorithm.

Once a contact’s face is recognized, we query our DB for the contact’s information and return it back to our raspberry pi glasses, which gets displayed on the wearer’s small non-obtrusive OLED screen. And since there’s only limited space on the screen, we’re also sending the user a text message using Twilio’s API to provide a full profile overview of the person you’re meeting.

Challenges we ran into

One unique constraint for this problem, is that we needed our facial recognition to work with a single sample per person. This is because for networking events, you will typically have access to only the profile pictures of attendees.

This added a lot of difficulty since most facial identification models rely on thousands of images. So this led us to implement a siamese neural network for one-shot facial recognition.

Accomplishments that we're proud of

Overcoming the single image sample size per person is an extremely challenging Machine Learning problem, so we are impressed with ourselves that we've built a feasible implementation of One-Shot facial detection in the short time frame of 36 hours. Also, we think our Hardware Glasses are very impressive (and stylish).