Inspiration
After an injury or surgery, most patients are asked to continue physical therapy exercises at home. But once they leave the clinic, therapists lose visibility and patients often struggle to stay consistent or perform exercises correctly. Research shows that adherence to home rehabilitation can drop to 35–50% within just a few weeks. We wanted to build a system that gives therapists the ability to continue supporting patients remotely, while giving patients confidence that they are exercising safely and correctly. PhysioSync was inspired by a simple but common scenario: an older patient falls, begins rehabilitation, but struggles at home without real-time guidance or feedback. We believed AI could bridge that gap between clinic visits and home recovery.
What it does
PhysioSync is an AI-powered physical therapy platform that helps therapists remotely monitor patients and helps patients complete their exercises correctly at home. Using only a phone or laptop camera, the system tracks body movement in real time, analyzes joint angles, counts repetitions, and gives live voice feedback. Therapists can assign exercises, set goals, monitor adherence, and receive alerts when a patient may be struggling or at risk. The platform also includes a multi-agent AI system that can detect complications, suggest treatment adjustments, and generate personalized encouragement for patients. A unique feature is that therapists can teach the system entirely new exercises simply by uploading a short demonstration video. PhysioSync automatically learns the movement and creates a reusable exercise template.
How we built it
We built PhysioSync with: React for the frontend FastAPI and Python for the backend MediaPipe Pose for body tracking Gemini API for intelligent goal suggestions and clinical reasoning SQLAlchemy and SQLite/MySQL for storing patients, sessions, and exercises The browser processes the patient’s movement locally using MediaPipe, so no video is uploaded. Only lightweight movement data such as angles and repetitions is sent to the backend. We then layered AI agents on top of this data to provide therapist insights, treatment suggestions, and personalized patient feedback. We also built a custom exercise-learning pipeline: upload a YouTube or recorded exercise video → detect body keypoints → identify which joints move → generate a complete exercise configuration automatically.
Challenges we ran into
One of the biggest challenges was balancing privacy with functionality. We wanted accurate motion tracking, but we did not want to send patient videos to a server. Solving that required running pose estimation entirely in the browser while still keeping the system fast and accurate. Another challenge was making the AI clinically useful rather than just “smart.” Therapists need trustworthy suggestions, so we combined rule-based logic with AI-generated insights. If the AI is unavailable, the system still works using traditional calculations and thresholds. We also had to handle many different recovery scenarios, ages, injuries, and precautions, which made designing the treatment adaptation and complication detection agents much more complex.
Accomplishments that we're proud of
Built a full end-to-end rehabilitation platform in a short time Created a working exercise-learning system that can learn new exercises from a video Designed a multi-agent clinical intelligence system with specialized AI agents for triage, treatment adaptation, complication detection, and patient motivation Kept all video processing private by running it directly in the browser Made the platform usable with only a phone camera—no wearables or special hardware required We are especially proud that PhysioSync does not replace therapists—it amplifies them. Every recommendation still stays under therapist control.
What we learned
We learned that building healthcare AI is not only about building a model—it is about building trust. Therapists need explanations, patients need encouragement, and both need a system that is reliable even when AI fails. We also learned that small details matter: voice feedback feels more human when it references a patient’s specific progress, and therapists are far more likely to trust AI when it explains why it made a recommendation instead of only showing a score. Finally, we learned that privacy-first design is possible. By processing movement locally in the browser, we can deliver meaningful healthcare support without requiring sensitive video uploads.
What's next for PhysioSync
Next, we want to: Pilot PhysioSync with real physical therapists and patients Add support for more conditions such as stroke recovery, back pain, and neurological rehabilitation Integrate with wearable devices and electronic health records Improve the AI agents using larger clinical datasets and evidence-based benchmarks Launch SMS and email reminders for patients Expand the exercise-learning system so therapists can create full rehabilitation programs from a few demonstration videos Long term, our goal is to make high-quality physical therapy more accessible, personalized, and effective for everyone—especially patients who cannot regularly return to a clinic.
Log in or sign up for Devpost to join the conversation.