POSTURA | Devpost

Landing Page
SetUp
Live Feedback
Feedback Summary

Inspiration

Our team consists of individuals with diverse backgrounds. Anshuman is a Biomedical Sciences major, Alex previously interned at Limbitless Solutions (a prosthetic arm company), and Rohan and I have been personally impacted by physical therapy in our personal lives. We are deeply inspired by our experiences and wanted to create a solution to help make physical therapy easier at home, in conjunction with a physical therapist. We all share the same idea that physical therapy is beneficial when feedback is provided at the time one is exercising. Our application is capable of providing that while also giving a video summary with pointers on how to do better. With our application, we hope to provide a modern solution to the medical sector so that it is easier to supplement treatment plans with exercise at home.

What it does

POSTURA is a coaching software designed to provide live feedback when doing exercises. When someone is doing a movement, such as a squat, POSTURA is able to detect when they are not bending their knees enough or when they are going too low. Such critiques are important as having improper form not only leads to injury, but can worsen a pre-existing injury. During the exercise, POSTURA provides two types of feedback: summarized and detailed. Summarized feedback is given as one is doing the exercise and is meant to promote quick adjustments. Detailed feedback is provided after the exercise is completed and gives a more in-depth analysis of form, allowing for a comprehensive understanding of where one may improve over time.

How we built it

React: Was used to develop our front-end and connect Flask with OpenCV/MediaPipe

Flask: This was used to connect React and OpenCV/MediaPipe to leverage their capabilities for our project

OpenCV: Used to interface the camera (Computer Vision)

MediaPipe: An AI and Computer Vision model that maps a skeleton over a person that has key markers that can be used to derive valuable metrics for our project

Claude.AI: Claude helped us design our app logo

Challenges we ran into

We ran into a few challenges during POSTURA's development, chiefly in how joint angles are determined and what level of tolerance to include to factor slight deviations in form between people. Rohan, using MediaPipe, found significant joint angles using the relative positions of the arms and legs. The most important angles analyzed are those of the wrists, knees, and ankles. The latter specifically proved challenging as the angle was originally determined using the toes. This provided unreliable results and so the point on the foot was changed to the heel, yielding much more accurate results.

Accomplishments that we're proud of

Provide live feedback while exercising
Create a visually appealing website with a challenging color palette
Cover a variety of exercises that test the limits of our application

What we learned

This project carried many opportunities to learn new skills and apply existing ones. We each had a great opportunity to learn about the technologies used. Alex learned how to use React, integrate frontend and backend development, and transfer data between React and Flask. Rohan learned how to better identify objects and angles, analyze video, streaming input, and make changes to his code accordingly, as well as to better apply calculus and trigonometry principles in real-world settings. Anshuman learned how to work with React, acquired a greater understanding of frontend development, and gained more knowledge of how computer science and healthcare are intertwined. Nessa also learned a lot about integrating all the former technologies and has had personal experience with physical therapy for many years. As the project went on, she gained a significant understanding of how to cross the "language barrier" between people and computers.

What's next for POSTURA

This project has the capability to expand and bring real change to people currently in physical therapy but also those who are interested in working out at home. This application has the potential to reach many audiences due to how big exercising is in our society. Overall, we would do the following--