-
-
Magic Mirror (Webcam covered-up)
-
Magic Mirror waiting for me to put my shoulder line inside the box so it can start dressing me up
-
My shoulders inside the box, Magic Mirror gives me a dress
-
Changing clothes by raising my arm
Inspiration
Dress-up rooms in clothing shops, cardboard cutouts photo props
What it does
Allows you to check how different clothes would potentially look on you. Users use a webcam, and on the live video output, they can change clothes by gesturing with their left arm. To turn it on, users have to put their shoulder line inside a designated box area that is seen on the video output. Users can choose to blur out their faces if they want.
How we built it
Detectron2 for keypoint detection to allow detection of certain gestures and the detection of the shoulder line of the user. GoogleColab environment to take advantage of their free GPU. Python for the language. Python facial detection library to detect a face, cut it out of the video frame, blur it and then overlay it over the output frame. OpenCV to overlay images over each other.
Challenges we ran into
Overlaying images makes them change color. Adjusting the images that are being overlayed (the clothes) is time-consuming, cutting out their background, and making them smaller so that if one scales the original image to the video frame size the output will go over the person and not somewhere to the side. Overlaying one image over another requires them to be of the same dimension, thus each image has to be rescaled. Installing detectron2 is also a challenge as it requires the right Torch and CUDA versions which took time to find. Webcam video frames had 4 channels while image files had 3 channels, thus to overlay one over the other had to figure out how to add the alpha channel to the image files.
Accomplishments that we're proud of
Gesture control using the keypoints. Figuring out how to overlay the clothes and the blurred-out face.
What we learned
Working with OpenCV to modify live videos is very hard. Overlaying images is not an obvious process.
What's next for Magic Mirror
Edit the clothes so that they actually go over the user, leaving the face open. Play with the overlaying mechanism, figure out how to preserve original color of the clothes. Make gesture detection less sensitive, sometimes it detect gestures that didn't happen. Get a bigger collection of clothe images.
Log in or sign up for Devpost to join the conversation.