SpotAssist | Devpost

Boston Dynamics' Spot
System Arch Diagram

Inspiration

In response to the challenges faced by the elderly and disabled who rely on traditional, costly service dogs, our team drew inspiration to create SpotAssist. Traditional service dogs demand substantial resources, extensive training, and are limited to one user at a time. SpotAssist leverages Boston Dynamics’ Spot to overcome these challenges, offering advanced functionalities through technologies like machine learning, computer vision, and natural language processing. This innovative robotic companion not only surpasses traditional service dogs but also holds the potential for mass deployment in settings like senior care homes, addressing financial and logistical constraints. SpotAssist embodies our commitment to making a significant impact by combining technology, compassion, and scalability to enhance the lives of those in need.

What it does / How we built it

SpotAssist is an assistive robot meant to achieve better functionality than a regular service dog. First, Spot will take an image of its surroundings and generate a list of objects detected along with their corresponding bounding boxes (using YOLO). Then, Spot will wait for the user to make a verbal query including a target landmark to approach; an example of such a command would be “Take me to the nearest chair.” Finally, Spot will gently guide the user to their desired destination through noting the difference between the center x-coordinate of the bounding box and the actual center of the image. Spot will utilize this quantity in order to align its heading angle to be in line with the desired object and subsequently move forward, stopping at a reasonable distance from the object in question. Furthermore, we built health-first integrations from the ground-up for Spot. Using our custom ML and analytical methods and Terra API integrations, we are able to use Spot as an alert for a medical event, such as abnormal heart rhythms and/or heart rate.

Challenges we ran into

There were a number of technical challenges we encountered throughout the development process. Firstly, a major hurdle was the restrictions placed on the compute module attached to Spot; we were not allowed to perform any commands directly to Spot itself (we instead had to communicate through the module). Furthermore, we were not allowed to SSH into the module (nor were we given elevated privileges), so we had to build our own version of a remote communication system between our remote servers and Spot. This led to many creative avenues being explored for development.

Accomplishments that we're proud of

We’re especially proud of building upon a system/environment that was entirely new to the whole team, especially without any documentation on the compute module we built upon. Furthermore, none of the team knew each other prior to the hackathon, so we’re proud of the teamwork skills we were able to utilize to accomplish our success this weekend.