-
-
The cane
-
The audio set up interface
-
Object Detection camera
Inspiration
Helping people navigate the busy streets of Toronto and get help if they need it.
What it does
Detects if the person walking has fallen over and sends alerts, and describes the surroundings.
How we built it
The hardware takes in sensor data and feeds it into a custom TensorFlow model to detect whether the cane has fallen, and it can alert people. An HTML website uses voice recognition to help set up an account and contact information. A second mobile website uses camera data and sends it to ChatGPT to detect what obstacles are around.
Challenges we ran into
Trying to get the sensor data streaming over WiFi so the data could be collected for training, as we couldn't leave it plugged in as we were walking around and dropping the stick. Trying to get an Arduino camera working on RP2040 (we got it working on an Arduino Uno, but couldn't get it to work so we switched to a mobile phone), and trying to get the TensorFlow model running on the RP2040!!
Sleeping too much :(
Accomplishments that we're proud of
Get the sensor data trained on a custom model running on a small Arduino!! 💪
What we learned
ChatGPT API, Python Backend, TensorFlow Lite for Arduino
What's next for Caneine
Give a better and faster description of surroundings
Log in or sign up for Devpost to join the conversation.