Realbot | Devpost

Inspiration

As Waterloo students, we constantly relocate to different cities for internships. Every term comes with the same process of looking for housing, which naturally develops into a key problem: how can you confidently sublet a unit you've never physically seen? You have two options: drive for hours or spend hundreds of dollars on a flight to visit the unit for 10 minutes, or rely on a curated set of images and video walkthroughs online. You can't look inside a cabinet, check the ceiling for water damage, or see if the overhead light flickers.

We created Realbot to bridge the gap between the convenience of virtual viewing and the interactability of an open house or unit tour.

What it does

Realbot allows landlords and realtors to easily set up autonomous virtual tours for prospective tenants and buyers. For viewers, Realbot provides the freedom of a walkthrough from anywhere.

For the landlord/realtor, a one time set-up allows for unlimited tours, no filming required. Simply bring the robot to the unit and let it to map out the space with SLAM. Next, walk through the unit and point at what a tenant would want to try. Our system defines interactable objects such as cabinets, doors, and light switches using hand gestures and voice (thumbs up to enter definition mode, point to select, speak to describe). To open up for viewing, log on to our web app, select a time range for prospective tenants to control the robot, and share the generated link.

As the viewer: Use the link or code to enter the video feed on the web app during an open time slot. Use WASD to move around. Click on the screen to interact with the doors and cabinets defined by the landlord/realtor.

How we built it

We combined a browser-based property touring platform with a mobile robot physically located within the unit. We built the web app with React, TypeScript, and Vite, using Tailwind CSS and Framer Motion for the interface. Three.js turns the robot’s scanned floor plans into interactive 2D and 3D views, allowing realtors to manage spaces and visitors to understand where the robot is during a tour.

LiveKit connects the browser to the robot over the internet. It carries the live camera feed, robot status updates, and driving commands with low delay. Visitors can move the robot using keyboard controls, while built-in safety measures stop movement if keys are released, the browser loses focus, or the connection becomes unreliable.

On the robot, Python services connect LiveKit to Bracket Bot’s existing bbOS hardware and mapping systems. These services manage camera streaming, movement, and map data by leveraging the robot’s built-in controls.

Supabase provides realtor sign-in and organization membership, while visitors can join tours without creating an account. We also built local simulators and automated browser tests so the full experience can be developed and verified without always requiring access to the physical robot.