Poker Face | Devpost

Check out our pitch for a complete understanding of our technology:

Our Breadboard: https://breadboard-ai.web.app/?tab1=https://breadboard-community.wl.r.appspot.com/boards/@AdorableElephant/poker-face-copy.bgl.json

Inspiration

Playing poker is easy. Playing poker well is hard.

At PokerFace, we want to make playing poker well easy for everyone by democratizing optimal decision making strategies used by top players across the world and bringing them to your live poker games. Our platform views and identifies your cards, chip stacks, pot sizes, table and bet sizes, using Computer Vision to provide you GTO optimal advice and strategic explanations to make you the best poker player possible regardless of your background. We use computer vision, statistics, probability, and Generative AI to precisely and accurately provide real time insights and explanations on your real life Poker games, allowing you to maintain composure in critical situations. Computed mathematical analysis to compute pot odds, bet sizes, risk strategies, equities, expected values, allow you to build strategic intuition for the beautiful game of poker.

Using a combination of AI-driven insights and mathematical analysis, PokerFace delivers real-time data on pot odds, bet sizing, equities, and risk strategies. Whether you're new to the game or an experienced player, PokerFace enables you to maintain composure and execute optimal moves during critical moments by providing detailed strategic guidance based on calculated odds, EV (Expected Value), and other crucial poker metrics.

What it Does

Dashboard and Computer Vision

Our live-updating dashboard is powered by real-time computer vision that accurately captures and processes in-game information like bet sizes, stack sizes, your cards, community cards, and pot sizes. Based on this data, the system generates real-time recommendations, such as optimal bet sizes, calculated pot odds, risk-adjusted strategies, and expected values, guiding you toward the most profitable moves. Additionally, we don’t just give you the numbers—we also provide strategic explanations for each decision so you can learn from every hand, improving your overall poker skills.

Gameplay Tracking

PokerFace doesn't just optimize your play in real-time; it also stores and analyzes past gameplay data to give you a comprehensive view of your habits and tendencies over time. By understanding your biases, patterns, and heuristics, our system offers personalized insights that help you become a better poker player. Whether you're prone to risk-averse behavior or overly aggressive bluffs, PokerFace’s AI provides tailored advice that addresses your unique playstyle, making you more adaptable in future games.

AI Agent / Gemini

Our AI agent, dubbed “Lady Gaga,” leverages prompt engineering, recursive contextual models, and advanced machine learning techniques to offer expert-level insights into poker strategies. By processing data gathered from gameplay and image streams, the AI agent can analyze your current situation and suggest optimal moves while teaching poker at a high level, regardless of your current skill. Integrating Google Breadboard and computer vision from a live video feed allows the AI to continuously update itself with the most accurate and relevant data.

How We Built It

PokerFace was developed as a mobile application using React Native (Expo) for the frontend and FastAPI for backend operations and statistical analysis. We used Roboflow to build our computer vision model, which processes poker chips, cards, and other gameplay elements. For the AI-driven insights, we integrated Google AI Breadboard, enabling our LLM (Language Learning Model) assistant to provide expert feedback, gameplay highlights, and strategic recommendations.

From a mathematical standpoint, we delved into poker theory, combinatorics, and game theory to design accurate statistical models. Monte Carlo simulations were employed to calculate Expected Values (EV), pot odds, equities, and other vital poker metrics based on real-world gameplay data. We also factored in player behavior tendencies—such as risk-aversion or risk-seeking—to ensure our AI’s decision-making mirrors real-life strategic dilemmas.

Challenges We Faced

We encountered multiple technical challenges throughout development. Establishing a continuous WebSocket connection for the video stream was complex, particularly given the transition from React.js to React Native for mobile development. Additionally, learning new technologies like Google Breadboard and Roboflow was initially difficult due to unfamiliarity, but ultimately they became crucial to improving the overall quality of PokerFace.

Another major challenge involved structuring the input pipeline for Google Breadboard. Initially, the context mapping was handled incorrectly, causing the model to infer the wrong information. By reconfiguring the system to handle different input modes (e.g., image streams and text data), we eventually resolved this issue. Moreover, we struggled with SSE (Server-Sent Events) versus JSON formats, which required a more specialized approach to handle real-time data streams.

Accomplishments We’re Proud Of

We successfully integrated a custom AI agent using Google Breadboard that could process player history, contextual data, and game information to deliver expert poker advice. This recursive feedback system allows the agent to learn from each player’s behavior and improve decision-making recommendations over time.

Furthermore, we developed a full-stack mobile application with a dynamic user interface, a robust backend powered by FastAPI, and a computer vision model that recognizes poker chips and cards in real-time. Our use of advanced mathematical models and algorithms to guide optimal decision-making is another accomplishment we're particularly proud of.

What We Learned

Throughout this project, we gained extensive experience in integrating AI agents with custom machine learning models. We also learned how to use FastAPI to create a real-time interface for our computer vision model. Additionally, working with Google Breadboard gave us hands-on experience in integrating multi-modal inputs and building a custom LLM tailored to poker strategy.

What’s Next for PokerFace

Looking ahead, we plan to enhance PokerFace by integrating wearable cameras (such as those embedded in glasses) to improve the user experience and provide a more seamless way to capture gameplay data. We’re also excited to explore sentiment analysis as a way to better understand player emotions, helping users maintain an effective "Poker Face."

In the future, we intend to build a replay engine that allows players to review their previous games and learn from their mistakes. This would allow for deeper insights into gameplay performance, creating a powerful tool for learning and growth.