Motivation
Have you ever experienced the hassle of scrolling through the long list of emojis trying to find the one you are looking for? We have. Wouldn't it be nice if there was a tool that could find the perfect emoji instantly? We present to you Instamoji: a computer vision based algorithm that matches your expressions and hand gestures with emojis effortlessly.
Explanation
Since there isn’t any public emoji imitation dataset, we took about 100 photos of ourselves mimicking 10 emojis. Our first approach was to build a small convolutional neural network, but it was too shallow to capture the features. Then we tried deeper ones such as ResNet and Xception, but the validation and test accuracy were also too low. Given such a small dataset, we realized that using neural network is probably not the best approach, so we moved on to a more traditional method: image comparison. We measured the similarities between images by an algorithm called SSIM and displayed the 3 most similar images for our user to choose. In consideration of user experience, we designed a user interface using python package PyQt5 that displays the video stream along with the top 3 most likely emojis in real time.
Future Work
Integration into mobile messaging apps; higher accuracy and speed; more emojis to choose from!
Log in or sign up for Devpost to join the conversation.