STATVISION

Inspiration

After watching numerous baseball games, we always wondered what happens when a batter swings—will it be a home run? We also thought about the ball's trajectory. I have a cousin who is blind, and he mentioned how cool it would be if AI could explain things like the ball's speed, the probability of a home run, and the exit velocity, so he could "visualize" the game and enjoy it too.

What it does

Our platform generates AI-powered commentary and stats such as Exit Velocity, Hit Distance, Launch Angle, Pitch Type, Performance Score, and Ball Trajectory. These stats are displayed in a video with exciting commentary.

How we built it

We used MLB data provided by Google to create a model using a random forest classifier to predict outcomes. We analyzed video frames with Vertex AI to extract stats, then used the Gemini 2.0 model to generate commentary. We added realistic native voice audio and used tools like OpenCV and MoviePy to overlay images with stats onto the video.

Challenges

We faced challenges deciding which stats to extract from the video, as some are hard to track if the camera moves. It was tricky to create a model to predict home runs and to integrate overlay images into the video. Debugging that part took couple of days.

Accomplishments

We're proud of creating a video with AI commentary and stats that help bring the game to life.

What we learned

We learned how to use Vertex AI, Google Cloud, Virtual Machines, and how to deploy FastAPI. We also explored machine learning, video/audio editing, and code-based image manipulation.

What's next for STATCAST

Next, we plan to use ImagenAI 3 to predict the outcomes of upcoming MLB games and visually show how two teams might play based on their historical performance.

Built With

• fastapi
• gcp
• gemini
• google-cloud
• mern
• pandas
• scikit-learn
• vertex-ai