CardMate

Real-time playing card recognition that makes every deck accessible

Comment

💡 Inspiration

Playing card games is a universal way to connect with friends and family, but they often exclude people with visual impairments unless special Braille decks are purchased. Those decks are harder to find, more expensive, and separate the visually impaired from the shared experience of using standard cards. We wanted to create a solution that makes any deck accessible without requiring special equipment.

🃏 What it does

CardMate uses real-time computer vision to recognize playing cards through a smartphone or laptop camera and announces them instantly using text-to-speech. Players simply point their cards at a camera, and CardMate tells them the rank and suit. It even works for an entire hand of cards, so users can retrieve each card in their hand in the proper order from left to right. We mainly built this to support the game Go Fish although games such as Poker can easily be implemented. Players can easily play the game with a regular deck of cards even if they are visually impaired.

🛠️ How we built it

  • Utilized a Roboflow model and workflow to detect playing cards in various lighting conditions and orientations.
  • Created a Python FastAPI backend for API endpoints with websockets to allow for quick support with our HTML/CSS frontend application.
  • Integrated a multithreaded text-to-speech so detected cards are read aloud instantly and speech-to-text to allow the system to detect which action a user is performing.
  • Supports mobile devices as cameras or just a laptop webcam

🚧 Challenges we ran into

  • Getting the model to recognize cards reliably when partially visible, under poor lighting or at odd angles.
  • Working with threads and websockets as none of us have been familiar with those before
  • Designing an interface that’s accessible and intuitive for visually impaired users.

🔮 What's next for CardMate

  • Expanding to games like Poker and Uno and collectible card game.
  • Adding haptic feedback + multi-language support for inclusivity worldwide.
  • Packaging as a polished offline mobile app.
  • Exploring wearable integrations (smart glasses or compact cameras) for better user experience.
  • Adding an additional roboflow workflow to detect if a player is holding up the back of a card.

Built With

Share this project:

Updates