FORMulator | Devpost

FORMulator: Perfecting Body Movements

Inspiration

The idea for FORMulator stemmed from a desire to help athletes, dancers, and fitness enthusiasts improve their form and technique in a more intuitive way. We noticed that while traditional training methods rely heavily on external feedback from coaches or mirrors, these approaches often lack precision and real-time guidance. This inspired us to create a tool that allows users to perfect any type of body movement on their own, by leveraging computer vision technology to provide instant feedback.

What We Learned

Throughout the development of FORMulator, we deepened our understanding of computer vision, specifically through the use of PoseNet, a real-time human pose estimation model. We learned how to process video input to detect key points on the body, such as elbows, wrists, hips, shoulders, knees, and ankles. Additionally, we gained experience integrating machine learning models with live video feeds and understanding the challenges of real-time performance analysis.

How We Built It

FORMulator begins by taking in a video input of a single subject performing an action. This input is processed through the PoseNet model, which tracks the positions of key points on the body. These PoseNet features are then overlaid onto a live webcam feed, allowing users to see their positions in real time. The system compares the user's current movements with a pre-recorded "ideal" movement, highlighting any discrepancies.

The entire project was built using Python and TensorFlow for the PoseNet model, along with OpenCV for video processing and rendering. This combination of tools enabled us to create a seamless user experience that emphasizes real-time feedback.

Challenges Faced

One of the biggest challenges was ensuring the accuracy of the pose detection in various lighting conditions and environments. PoseNet's performance can be affected by factors like background clutter, occlusion, and variations in body type. We had to experiment with different pre-processing techniques to improve the robustness of the system.

Another challenge was optimizing the performance to maintain real-time feedback without compromising accuracy. Processing video frames at a high enough frame rate to ensure a smooth user experience required careful tuning of the model and the video processing pipeline.