Superheroes save lives and have superpowers. In our game, groups of friends enjoy competing for the best superhero pose and then get inspired to become a superhero in real life. They gain their superpowers by performing small acts of kindness that go a long way. Currently, we support gifting a superhero costume or an educational book to kids that need help.
SuperPose analyzes user submitted pictures and determines the closest match to the real superhero pose. It supports gift donations to kids in need using eBay.
The prototype for SuperPose is a web application built with Flask. We used technologies such as the TensorFlow port OpenPose, the eBay Find API, Beautiful Soup, and the cosine similarity method for matching sets of body keypoints.
The biggest challenge we ran into (besides learning new API's) was how to define a match using the data generated by OpenPose. Since we didn't have time to train a model, we chose to use cosine similarity for prototyping purposes. In order to perform analysis on the keypoint values, we had to make customizations to tf-openpose to extract the x, y coordinate values for all body and face parts.
When we first started reading the docs for OpenPose, we thought it might be too hard for us to work with due to our experience level. But we pushed through and produced a working product! This felt like a major accomplishment because instead of using a third party API to do the heavy lifting, we researched and implemented the methods of analysis ourselves.
We gained more experience with Git, TensorFlow, OpenPose, the eBay API, Beautiful Soup, methods for measuring similarity between images, but most of all that working on projects outside of your comfort zone can be the best way to grow.
Due to time constraints of the 36 hour hackathon, hardware limitations, and our beginner machine learning experience, we were not able to implement the ideal solution for matching poses which would have been to train our own model. In the future, we plan to improve the accuracy of the matching by crowdsourcing a dataset of people doing superhero poses. We would also like to bring our game to mobile platforms and make the leap to processing realtime video.
