getBounced

Facial recognition with a sharp twist — ensuring your event's exclusivity with high-tech deterrence.

Comment

Inspiration

We have all had our fair share of hosting gatherings with unwanted guests. We are tired of birthday parties and wedding crashers, causing unnecessary clean-up and drama! We are also broke college students who cannot afford to hire a bouncer and not everybody has a 6’7 friend.

Say hello to getBounced - the automated bouncer bot to suit your security needs! Our product mounts easily to any entranceway and uses facial recognition to decide if a guest is allowed through or if they’ll receive a disciplinary smack!

What it does

Before the event, the host will input event information such as the name, location, and start and end time. This information will generate a link to a form the guests will use to register. In addition to providing their contact information, the guests will be prompted to take a 10-second clip of their faces. These clips will be fed to train our computer vision model.

On the day of the event, our smart camera will be set up at the entrance of the event and it will determine if the person is a homie or a nobody. If the latter, then they’ll getBounced! A swift and demure sweep of the blade (trust us, it’s safe) and the alerts that the smart camera will fire will, hopefully, deter the uninvited away.

How we built it

The webapp was built with typescript and sandboxed with expo. We also made a quick backend in Flask, which helps us deal with the controller logic to preprocess image data, while enabling us to gather analytics on the behaviours of our party attendants, like when they arrive, leave, how long they've stayed at the party, and who they've interacted with. We leveraged several machine learning algorithms to segment and predict the identities of those that show up to our front door. Using mtcnn, we were able to isolate the faces from other noise in images, while using FaceNet (by Google) to generate embedding vectors of said faces. Finally, we trained an SVM and Logistic Regression model (choosing the latter at the end due to increased performance) to classify the vectors into users. We were able to use data preprocessing and image handling algorithms to reduce latency in facial analysis and recognition!

For the hardware component, we used a Jetson Nano to run our machine learning and computer vision algorithms, getting live guest video data from a Tapo Wifi Camera. We then connected the Jetson to an Arduino, which is used to control the servo motor upon which our deterrent weapon is mounted. The Jetson sends a signal through serial communication to the Arduino whenever an intruder is detected.

Challenges we ran into

One challenge we ran into was our lack of initial hardware materials such as power supplies and mounting equipment. This led to many wild goose chases around campus and visits to the hackathon hardware desk begging them to “check if they have any extras in the back”. Eventually though, after a lot of phone calls and a lot of kind souls willing to lend us parts from their collection, we were able to source everything we needed for our amazing bot.

Another challenge we ran into was not being able to control the stepper and servo motors properly using the Jetson Nano. This is because we were trying to use GPIO pins to do PWM control, which they are not meant to do. This led to a lot of wacky behaviour from our motors. To solve this problem, we decided to use an Arduino to control our servo motor, as there are many servo libraries built for Arduino.

Accomplishments that we're proud of

The biggest thing we are proud of is how we were able to integrate many different fields and skills in engineering together to create a cool and useful product! We were able to integrate computer vision with machine learning and IoT embedded systems motors and app development and even hot glue and cardboard!

What we learned

On the hardware side, we learned that it is best practice to use specially made microcontrollers to control electronics, especially power ones like motors. This way, our main system on a chip can be used primarily for algorithm running and communication.

What's next for getBounced

One next step would be to create “hot-swappable” punishments for unwanted guests. Currently, we only have a sword-smacking mechanism but in the future, we would elevate our product to include other options such as water gun spraying, trap door, ice bucket dump, and many more devious things.

Another next step would be to connect our product to a mobile app which can notify the host in real-time whenever any unwanted guests arrive and capture video evidence of their embarrassing downfall!

Built With

Share this project:

Updates