RehabEdge

RehabEdge: The Privacy-First AI Rehabilitation Assistant Inspiration

The inspiration for RehabEdge stems from a critical gap in the healthcare industry: the "Compliance Crisis." Studies show that up to 65% of physical therapy patients fail to complete their home exercise programs, often due to a lack of guidance and the fear of performing movements incorrectly.

While Telehealth solutions exist, they suffer from two major flaws: Latency (making real-time correction impossible) and Privacy Intrusion (streaming video of patients in their homes to the cloud).

I realized that to solve this, the AI needed to live with the user, not on a server. I wanted to build a solution that acts as a "Digital Mirror"—giving users professional-grade feedback on their form instantly, without a single frame of video ever leaving their device. This project is my attempt to democratize access to physical recovery using the power of Arm-based mobile computing.

How We Built It

RehabEdge is a native Android application built with Kotlin and Jetpack Compose. The core architecture relies on a "Sense-Process-Act" loop optimized for mobile hardware.

1. The Vision Pipeline (Arm Optimized)

We utilized MediaPipe Pose (running on TensorFlow Lite) to extract 33 distinct 3D landmarks of the human body. To ensure smooth performance (30+ FPS) without draining the battery, we utilized the Arm Mali GPU Delegate. This offloads the heavy matrix operations of the neural network to the GPU, leaving the CPU free to handle the geometric logic and UI updates.

1. The Geometric Logic (The Math)

Once we extract the landmarks (e.g., Shoulder, Elbow, Wrist), we need to determine if the user's form is correct. We calculate the angle θ at a pivot joint (like the elbow) using vector mathematics.

Given three keypoints—A (Shoulder), B (Elbow), and C (Wrist)—we treat the limb segments as vectors
BA

and
BC

. The angle θ is calculated using the dot product formula:

θ=arccos( ∣ BA

∣∣ BC

∣ BA

⋅ BC

​ ) Where the dot product
BA

⋅ BC

is:

(x A ​ −x B ​ )(x C ​ −x B ​ )+(y A ​ −y B ​ )(y C ​ −y B ​ ) And the magnitude ∣ v

∣ is calculated using the Euclidean distance:

(x end ​ −x start ​ ) 2 +(y end ​ −y start ​ ) 2

​

We implemented this calculation using optimized Kotlin math functions to run efficiently on the Arm Cortex CPU.

1. Real-Time Feedback Loop

A state machine monitors these angles. If the angle θ deviates from the "Safe Zone" (e.g., θ<160 ∘ during a squat extension), the app triggers an immediate Text-to-Speech alert utilizing Android's TTS engine.

Challenges Faced

1. The "Jitter" Problem Raw output from pose estimation models can be noisy, especially in low light. Keypoints would flicker, causing the angle θ to jump erratically (e.g., 140 ∘ →145 ∘ →138 ∘ ) in milliseconds.

Solution: We implemented a One-Euro Filter (a low-pass filter optimized for human motion) to smooth the signal. This removed the jitter without introducing the "lag" associated with standard moving averages.

1. Thermal Throttling Running camera preview + Neural Network inference + UI rendering initially caused the device to heat up after 5 minutes.

Solution: We optimized the inference loop by bypassing the CPU for image processing. We used the Android Camera2 API to pass the hardware buffer directly to the GPU delegate, minimizing memory copying between the CPU and GPU. This reduced thermal load significantly.

What I Learned

Building RehabEdge taught me that Edge AI is not just about performance; it is about trust. By keeping the processing on the Arm processor, we can offer a medical-grade tool that respects user privacy implicitly.

I also learned the intricacies of Heterogeneous Computing—balancing loads between the Cortex CPU (logic), Mali GPU (inference), and DSP (audio) is the secret to building mobile apps that feel "magical" rather than sluggish. RehabEdge is proof that the modern smartphone is powerful enough to replace specialized medical hardware.

Built With

• android
• android-studio
• arm
• jetpack
• kotlin
• mediapipe
• python
• tensorflow