Copy Cat

An always-on AI listener for equipment inspections. Technicians just talk and CopyCat transcribes, detects issues, fills the checklist, logs smells and other notes, and builds fleet memory.

Comment

Inspiration 💡 After talking to the Cat Digital team at HackIllinois, we learned technicians inspect 50-ton machines using tools rated 2.6 stars on the app store. The Cat engineer told us inspections rely on four senses (see, hear, smell, feel) but zero sensory data gets captured digitally. We thought: what if the app just listened?

What it does 🎙️ CopyCat runs continuous speech-to-text while a technician walks around a CAT 982 Wheel Loader, auto-matching spoken observations like "transmission is grinding" to checklist item 1.7 and suggesting FAIL status instantly. It captures smell data from voice using a 10-type smell knowledge graph, snaps photos for visual defect analysis, and stores everything in fleet memory so the AI flags recurring issues across machines. Technicians can also use our AI chatbot to diagnose problems on the spot.

How we built it ⚙️ React frontend with Web Speech API for live transcription, OpenAI Whisper for audio processing, and SuperMemory API for persistent inspection memory across sessions. RoboFlow YOLO model trained on CAT equipment for visual parts identification. FastAPI backend with JWT auth, bcrypt hashing, and a custom Cat Wordbank with 111 industry terms for smarter AI matching.

Challenges we ran into 🧩 Juggling SuperMemory, Whisper, and RoboFlow APIs into one cohesive pipeline was tricky since each returns data in different formats. Mapping unstructured speech like "that sounds sketchy" to structured checklist items required building domain-specific keyword logic. Git merge conflicts at 3am were not fun but we survived.

Accomplishments that we're proud of 🏆 The full voice-to-report pipeline works end to end with zero typing required. Smell analysis as a first-class inspection input is something no existing tool does, and the Cat team at the hackathon told us they'd actually want to use it. We built 5 complete screens (Voice, Report, Smell, Fleet, Wordbank) plus a login system as a 2-person team.

What we learned 📚 This was our first real hackathon and we learned how powerful Claude Code is for rapid prototyping. Experienced technicians rely on smell and sound way more than sight, but no digital tool captures that today. Building for gloved hands in harsh field conditions changes every UX assumption you have.

What's next for CopyCat 🚀 Integrate with Cat Inspect's API so reports push directly into their existing system. Add on-device engine sound classification (normal vs abnormal) using open-source models from Hugging Face. Build the feedback loop where technician corrections retrain the AI so it gets smarter with every inspection.

Built With

Share this project:

Updates