Snapose | Devpost

Snapose

Inspiration

We felt that physical therapists like our daughter Ruchi and their patients can greatly benefit from a gait analysis. A gait analysis is an evaluation of how the patient walks. An appropriate deep-learning AI based tool can broaden the availability and simplify the process of gait analysis.

What it does

A one-line code of OpenPose can detect key points of the body in a video of a person walking.
Our Python code can analyze x and y coordinates of the key points in different frames.

How we built it

After comparing several possible options for developing an app, we have selected Flutter with Dart, because it is a user-friendly approach with excellent documentation, and it can produce apps for Windows, Linux, iOS, or Android. We have designed the GUI of our Flutter-Dart-based app, and have written some of the necessary codes. However, we have so far not completed and finalized all codes for our app.
Qualcomm has a few deep-learning models that can detect some key points of the body. However, Qualcomm’s models so far cannot suit for analyzing the pattern of walking. We found Carnegie Melon University's OpenPose algorithm to be most suitable for our objective. We used one of the laptops remotely provided by Qualcomm for applying OpenPose on videos of people walking. The Qualcomm laptop’s Snapdragon chip architecture and sizable RAM could process videos much faster than on our own Intel chip based computers.
OpenPose is available as a Windows executable .exe file. A simple one-line command-line code can process a video, detect key points of bodies of multiple people, and produce .json files, one file for each frame of the video. We used an example video of multiple people walking.
To avoid possible mix-up of coordinates of key points, we created and analyzed our own video of Laxmi walking, with the camera facing on her front.
To make it possible to analyze movements of arms with respect to feet, we created and analyzed our own video of Laxmi walking, with the camera facing on her side.
We wrote a Python code to analyze the .json files from the multiple people video. We found that the selected person raises feet more than appropriate. Using Matplotlib package, we created an animation of key points of the selected person.
We tweaked the Python code to analyze the .json files from the video of Laxmi facing the camera. We found that Laxmi raises feet correctly.
We used the same Python code to analyze the .json files from the video with the camera on a side of Laxmi. Here also we found that Laxmi raises feet correctly.