C.A.R.E. — Companion Autonomous Robotic Entity

Tagline: The AI-powered companion that sees, moves, and helps.

Inspiration

Elderly and mobility-limited individuals often face challenges in performing simple daily tasks or moving independently. We wanted to create something that empowers them rather than replaces their independence.

C.A.R.E. was designed as an AI-driven, AR-integrated robotic companion that can see what you see, move where you can’t, and assist naturally through intuitive human-robot interaction.

What It Does

C.A.R.E. bridges the Snap AR Spectacles and the Booster K1 robot to create a seamless collaboration between human and machine.

AR Control Interface: Through the Snap Spectacles, users can view a live video feed from the robot’s perspective, complete with a joystick and minimap-style HUD overlay.
Autonomous and Manual Modes: The robot can patrol autonomously or be manually controlled using simple head movements or joystick gestures through the AR interface.
Human Tracking and AI Detection: Using Gemini and OpenCV, the robot detects and tracks people, following them safely for potential interaction or monitoring.
Real-Time Communication: Through ROS2, WebSockets, and an ngrok tunnel, the robot and the AR interface remain connected, streaming live data and video between devices.
Head Tracking Control: C.A.R.E. maps the user’s head orientation (for example, 0° forward, ± degrees for turns) to the robot’s movement, enabling natural, intuitive direction control.

The result is a personal, responsive AI companion that combines physical assistance with digital intelligence.

How We Built It

Hardware: Booster K1 Robot, Snap AR Spectacles
Software:

ROS2 for robot control and sensor data
Python + OpenCV for object and human detection
Gemini for AI acceleration and reasoning
Ngrok + WebSockets for live video streaming and control
Snap Lens Studio for AR interface design and head tracking integration

Workflow:

The robot streams video and sensor data via ROS2.
That data is sent to a local server, then tunneled using ngrok to the Snap Spectacles, where users can view it in real time.
Movement commands (via joystick or head rotation) are sent back through WebSockets, controlling the robot with minimal latency.

Challenges

Latency: Streaming live video through multiple layers (ROS2 → ngrok → Lens Studio) introduced lag. We mitigated this using two WebSocket channels—one for control (low bandwidth, high priority) and one for video (higher bandwidth, lower priority).
Real-Time Object Tracking: Integrating Gemini’s detection pipeline with ROS2 to enable smooth following behavior.
AR Interface Design: Building a minimal and intuitive HUD inside Lens Studio.
Head Rotation Mapping: Calibrating rotation degrees between AR camera orientation and robot motor angles for accurate control.

Use Cases

Accessibility support for elderly or mobility-limited individuals
Security and patrol applications
Search and rescue operations
Construction and inspection scenarios

AI Component

C.A.R.E. uses computer vision and AI reasoning to:

Detect and track people in real time (Gemini + OpenCV)
Interpret visual context through AR integration
Enable future multimodal AI understanding using visual-language models

What Makes It Unique

A working prototype, not just a concept
Human detection and AR control integrated end-to-end
Seamless bridge between robotics and augmented reality
Real-world demonstration of AI-driven assistance and spatial interaction

What’s Next

Integrate proactive AI behaviors to anticipate user needs
Add voice interaction and multimodal reasoning (VLA/VLM)
Optimize streaming with adaptive bitrate and on-device compression
Extend to outdoor navigation and multi-robot coordination

Team & Mission

We are a small team exploring how AI, AR, and robotics can make tangible real-world impact—starting with accessibility but scalable to healthcare, security, and other fields.

C.A.R.E. represents a step toward empathetic robotics: technology that doesn’t just respond, but truly helps.