Contouring You | Devpost

model created using dlib
model created using mediapipe
contour filter suited for a diamond face shape

Inspiration

While numerous online face filters claim to assist individuals in applying makeup, our team has experienced that these filters either never suit your face shape at all, or alter your face shape to better align with the filter, rather than adapting the filter to enhance your unique features. That is why we decided to create Contouring You, our AI makeup assistant, designed to craft the ideal contour makeup look tailored to your specific face shape!

What it does

Contouring You begins by conducting a comprehensive analysis of your facial structure, comparing it to a diverse selection of pre-categorized face profiles from the internet. It then classifies your face into one of four distinct categories: round, diamond, heart, or rectangle. Next, a tailored contour filter, designed to complement your specific face shape, will be applied.

How we built it

To create our Contouring You, we used both computer vision technology and open-source models. Firstly, we utilized OpenCV for computer vision capabilities, allowing us to access to the camera to accurately capture the user's face. For maximum precision, we incorporated two open-source models made using mediapipe and dlib, to extract detailed facial coordinates, which we then plotted onto a graph.

We then used training data to enhance the models. This training data comprised a wide range of facial shapes, each with their measurements obtained through obtaining the coordinates relative to the image.

Next, we categorize the user's face based on one of four distinct categories: round, diamond, heart, or rectangle. This categorization was achieved by measuring the Euclidean distances between the user's facial coordinates and those obtained through the training data.

Finally, once we had determined the user's face shape, we applied a specially crafted filter, tailored to their specific facial features. This filter represents the ideal makeup look personalized to suit their unique face shape.

Challenges we ran into

As newbies to the realm of machine learning and computer vision, we spent a large amount of time understanding the various models we incorporated and getting acquainted with the software used in computer vision.

Our very first significant challenge while coding revolved around accuracy, and we addressed it by adopting a two-model approach.

Another notable challenge we encountered was isolating faces from the photos and resizing them consistently to match a set of reference points. To tackle this, we calculated the dimensions of both the face's background and the image itself before scaling each image to achieve the desired uniformity.

Accomplishments that we're proud of

As a team that was unsure we were going to have a finished product just only 2 hours ago, we are proud to have something we are excited to submit and present to the judges. We also all had a lot of fun learning the ropes in machine learning and AI!

What we learned

We learned how to use a lot of python libraries such as pandas, NumPy, and matplotlib. We also touched upon OpenCV, which we hope to use more in the future.

What's next for Contouring You

In our future plans for Contouring You, we hope to expand our training dataset, allowing for a broader spectrum of face shape categorizations. Additionally, we aim to incorporate measurements for eye and eyebrow shape, allowing us to create the perfect eye makeup look tailored for the user's unique features as well.