Learning new exercises without feedback can increase uncertainty, frustration, and fear of incorrect form, which negatively impacts both physical safety and mental wellbeing.

Oops All Motion is a hardware-powered fitness coaching system that attempts to solve this by:

• Tracks exercise movement using an Arduino UNO Q connected to a webcam and running an Edge AI classification model
• Analyzes exercise form through webcam feed locally (no cloud, no latency)
• Sends real-time feedback and video feed through a websocket to a Samsung Galaxy XR headset

Displays a digital twin that:

• Identifies what’s wrong with your form
• Visually demonstrates the correct movement
• Helps you fix mistakes while you’re mid-rep

Armie is our hardware kit powered by the Arduino UNO Q that parses the webcam feed to the Unity project.

Make sure to install python and venv onto the Arduino device. We recommend connecting to the Arduino via SSH https://docs.arduino.cc/tutorials/uno-q/ssh/.

We used the following public dataset on Edge Impulse https://studio.edgeimpulse.com/public/834421/live. Clone this project to add more to the dataset or download a model for your device. We have provided the eim model from this project at dumbbell-standard-exercise-recognition-linux-aarch64-v5.eim which is built specifically for the Arduino UNO Q.

python3 -m venv venv

source venv/bin/activate

pip install opencv-python tensorflow websockets bson asyncio

python3 stream_pipeline.py

Open the project with Unity v6000.1.17f, build and run.

To run Edge Impulse Runner with our model run the following command edge-impulse-linux-runner --model-file dumbbell-standard-exercise-recognition-linux-aarch64-v5.eim --gst-launch-args "shmsrc socket-path=/tmp/opencv.sock is-live=true do-timestamp=true ! video/x-raw,format=BGR,width=640,height=480,framerate=30/1 ! videoconvert ! jpegenc" --verbose

If webcam stream can't be found, verify your webcam device is found with v4l2-ctl --list-devices.

Jaiswal, Abhishek, et al. “Using Learnable Physics for Real-Time Exercise Form Recommendations.” ArXiv (Cornell University), 14 Sept. 2023, pp. 688–695, https://arxiv.org/abs/2310.07221.

Cornick, Jessica E, and Jim Blascovich. “Consequences of Objective Self-Awareness during Exercise.” Consequences of Objective Self-Awareness during Exercise - PMC, U.S. National Library of Medicine, 11 Aug. 2015, https://pmc.ncbi.nlm.nih.gov/articles/PMC5193311/.

Berke, Jordan. “OpenBarbell.” Hackaday.io, 2023, https://hackaday.io/project/3706-openbarbell.
Accessed 25 Jan. 2026.

Previous research seems mixed on the topic of digital twins to improve health outcomes. For novices it seems being self aware of one’s own mistakes leads to users feeling more stressful and less eager to exercise. With experienced users, the external view can help them improve their form when they already have some confidence in working out. Where our study expands upon this research is adding a classification model to update the user on what specifically the user is doing incorrectly. This may help them improve their exercise outcomes without focusing on the negative feeling of getting the exercise wrong.