A physical-therapy app that turns a phone — plus an optional IMU — into a closed-loop progress tracker, giving patients real-time form feedback and clinicians richer movement data than a self-reported scale.
Doctors configure a patient's exercises; patients record themselves and get real-time form feedback plus long-term recovery insights. 3D keypoints from RTMPose3D fuse with an optional ESP32 IMU over BLE through an Extended Kalman Filter that weights each camera update by detection confidence. Won the Healthcare track at Nexhacks @ CMU and went live on TestFlight in Feb 2026.
Patients doing at-home PT have no good way to see whether their joint flexion is actually improving session-to-session, and clinicians want richer movement data than a self-reported number. SmartPT closes that loop with hardware the patient already owns.
100 Hz IMU reads on the ESP32 batch into 30 Hz BLE notifies; a round-trip-time calibration (app sends an ack, RTT/2 added to each timestamp) aligns the IMU stream to the video.
State = [joint angle, IMU bias]. The IMU drives the angle at high rate; each CV estimate is a corrective update weighted by detection confidence — high-confidence frames pin the bias.
Frames stream to the Overshoot API for live activity classification while simultaneously recording locally; on session end the full clip runs through RTMPose3D for the EKF.
The IMU sharpens accuracy but isn't required — the same EKF can fall back to a constant-velocity model so the app still works phone-only.