Poseify

Online Education Simplified ~ Content Creation Automated

Comment

Inspiration 💡✨

Asynchronous online education, especially in the context of motion-related activities, can be a hassle. We often need to make multiple video attempts until we generate one that we feel comfortable sharing. Some reasons that may make this process burdensome and dissuade individuals from creating content:

  1. messy background,
  2. self-conscious about personal appearance
  3. poor camera technology
  4. time consuming to edit video
  5. confusion over right-left sides in video

We wanted to automate the task of sending videos and make it a fun and easy experience for the senders. Therefore, we made Poseify, a one-stop solution for converting your video to simplified animations, easy for students to follow along. The simple animation steps along with color-coded right and left sides is ideal for generating: 1) Yoga sequences 2) Martial arts drills 3) Physical therapy exercises 4) Dance steps 5) Fitness Class

What it does 🚩✨

  • Asks the user to record or upload a video
  • The ML model converts the video to a series of simplified drawings of stick figures
  • The user can easily select frames they want to keep
  • Final GIF sequences are available for user download or upload through the web application
  • Users are sent a text message through Twilio, welcoming them to Poseify once they log in

How we built it 🧱✨

  • Figma: UI Design
  • HTML/CSS: For frontend
  • Python/Django: For backend
  • ML: We used the ml5.js library for doing most of our machine learning work. After much researching, we identified the PoseNet model, which was trained on a large labeled dataset of 17 keypoints representing landmarks on the human body (ex: eye, nose, elbow, shoulder, wrist hip, etc...). We sent each frame in the camera to the PoseNet model and received an output of keypoints. We used p5js for interactively drawing the keypoints on the user video feed and then to generate the simplified animations.
  • Javascript: All of the canvas drawings on the website were dynamically generated using p5js library.
  • Twilio: for SMS notifications

Best Domain from domain.com

https://poseify.tech

Challenges we ran into 🧗✨

  • Figuring out how to interact with the ML model and use the outputs to generate new art
  • Manipulating the HTML canvas element to generate art
  • Connecting different components (frontend - backend - ML + user interface)
  • Communicating across multiple time zones
  • Coming up with a feasible idea that solves content creation problem
  • Generating GIF from the individual png
  • Some members did not have much experience, so they had to learn along the way

Accomplishments that we're proud of 🏆✨

  • Utilizing a ML model which could accurately analyze poses
  • Dynamically generating art to represent movement
  • Creating an aesthetically beautiful website that is easy to use
  • Completing the project within the given time frame

What we learned 🏫✨

  • Collaboration with other developers
  • Efficient time management
  • How to implement Django and machine learning
  • Updating user interface elements

What's next for Poseify 🔮✨

  • Improving GIF generation through the web application
  • Adding more features (sharing on social media, liking, commenting, etc.)
  • Options to generate different art styles
  • Gamifying the user interface
  • Building a mobile app
  • Improving UI
Share this project:

Updates