⚽ TacticoAI - Your AI Tactical Analyst

Automated game analysis for college and high school sports teams

💡 Inspiration

Every week, sports coaches spend 10-15 hours manually reviewing game footage, pausing, rewinding, and taking notes to understand what went right and wrong. For professional teams, this is manageable—they have dedicated analyst teams. But for high school and college programs? Coaches do it all themselves, often late into the night after already coaching practice, teaching classes, and managing a team.

We built TacticoAI because every coach deserves access to professional-grade tactical analysis, regardless of their budget. We wanted to use AI not just to save time, but to provide deeper insights that would actually make teams better.

🎯 What It Does

TacticoAI is an AI-powered tactical analyst that watches your game footage and tells you what happened—automatically.

Upload Your Match

Drop in your game video (soccer or basketball), and our system gets to work. Whether it's a full 90-minute match or a 5-minute scrimmage clip, TacticoAI handles it.

AI Watches Every Frame

Our computer vision system tracks every player, follows the ball, identifies teams by jersey color, and measures real-world speeds and distances—all automatically.

Get Tactical Insights

Within an hour, you receive:

Annotated video showing player movements, ball possession, and team positioning
Tactical summary explaining what your team did well and where to improve
Performance metrics like possession percentage, press intensity, formation analysis
Player statistics including speed, distance covered, and involvement in key plays
Interactive visualizations of formations and tactical patterns

Ask Your AI Coach

Have follow-up questions? Our voice-enabled AI coach has watched your entire match and can discuss tactics conversationally: "How did our press perform in the second half?" or "Which players covered the most ground?"

🚀 How We Built It

The Vision Pipeline

We use YOLOv8 (state-of-the-art object detection) combined with ByteTrack (multi-object tracking) to identify and follow every player and the ball throughout the match. This isn't simple detection—we maintain consistent player IDs even when they overlap, leave the frame, or are partially hidden.

To distinguish teams, we implemented K-means clustering on jersey colors. The system automatically figures out "this team wears blue, that team wears red" without any manual input.

For accurate measurements, we built a perspective transformation system that converts pixel coordinates to real-world meters, accounting for camera angle and position. Combined with optical flow analysis for camera movement compensation, we can accurately measure player speeds and distances even with moving cameras.

The Brain

All that tracking data feeds into Letta AI (powered by Claude Sonnet 4.5), which analyzes tactical patterns and generates human-readable insights. It's not just pattern matching—the AI understands soccer and basketball tactics and can explain formations, pressing patterns, and strategic strengths/weaknesses.

We integrated VAPI for voice conversations, letting coaches ask follow-up questions naturally and get context-aware responses grounded in the actual match data.

The Platform

Frontend: Built with React + TypeScript for a responsive, modern interface that works on phones and tablets
Backend: FastAPI (Python) handles video uploads, job processing, and API endpoints
Database: Supabase provides PostgreSQL storage for all tracking data, match analyses, and team statistics
Processing: Background job queue processes videos asynchronously with real-time status updates
AI Services: Letta AI for tactical reasoning, VAPI for voice coaching

The Architecture

Video Upload → Supabase Storage
     ↓
ML Analysis (YOLO + ByteTrack + OpenCV)
     ↓
Tracking Data → Database
     ↓
Letta AI → Tactical Summary
     ↓
Frontend Visualization + Voice Coach

🏔️ Challenges We Ran Into

"The Ball is Really Small"

Detecting a soccer ball in 1080p video when it's 50 meters away? Turns out that's really hard. Ball tracking was our most frustrating challenge. We solved it with interpolation—when we lose the ball for a few frames, we use pandas to intelligently fill in the gaps based on trajectory.

"What Color is That Jersey?"

Lighting changes, shadows, and mud make jersey color classification tricky. Two teams with navy blue vs. black jerseys almost broke our K-means clustering. We ended up sampling multiple regions of each player's torso and using HSV color space instead of RGB for better consistency.

"Perspective is Everything"

Converting pixel movements to real-world distances requires knowing the camera's perspective relative to the field. We built an automatic field detection system that finds the field boundaries, but unusual camera angles still cause issues. Our fallback uses proportional positioning when automatic detection fails.

"90 Minutes is a LOT of Video"

Processing 90 minutes of 1080p footage frame-by-frame with deep learning models is computationally intense. Even with GPU acceleration, it takes 45-90 minutes. We implemented stub caching (saving detection results for reprocessing) and optimized batch processing, but there's no magic bullet—computer vision is just expensive.

"Context is King for AI Coaching"

Generic LLMs don't know soccer tactics. We had to carefully structure the match data we feed to Letta AI—not just raw numbers, but contextual information about formations, possession zones, and event sequences. Getting the AI to generate actually useful tactical insights (not generic commentary) required extensive prompt engineering and context building.

🏆 Accomplishments That We're Proud Of

It Actually Works

We built a real, working product. You can upload a match video right now and get back professional-quality analysis. That's not a demo, not a mockup—it's functional end-to-end.

Professional-Grade Computer Vision

Our tracking accuracy rivals broadcast systems that cost millions. We're detecting players at 95%+ accuracy, maintaining consistent IDs across the entire match, and measuring real-world speeds within margin of error of professional systems.

AI That Understands Tactics

The tactical summaries aren't generic—they're specific, insightful, and actionable. Coaches could actually use these insights to adjust their game plan. That's because we went beyond basic LLM usage and built sophisticated context pipelines that give the AI real understanding.

Beautiful, Responsive Interface

Our frontend isn't just functional—it's polished. Dynamic theming by university, smooth animations, real-time job status updates, interactive visualizations. It feels like a professional product, not a hackathon project.

Voice-Enabled Coaching

The VAPI integration for conversational AI coaching works seamlessly. You can literally talk to your AI coach about the match, and it responds with relevant, data-grounded answers. That's a genuinely novel interaction model for sports analysis.

📚 What We Learned

Computer Vision is Humbling

We thought "just run YOLO" would be 80% of the solution. Turns out that's 20%. The real work is tracking consistency, handling occlusions, compensating for camera movement, transforming coordinates, and dealing with edge cases. We have so much more respect for the engineers behind broadcast tracking systems now.

Context is Everything for LLMs

Throwing raw data at an LLM doesn't produce good results. We learned that how you structure the context is as important as the model itself. Our best results came from building rich, hierarchical context structures that mirror how coaches actually think about tactics.

Background Jobs Are Non-Negotiable

For long-running processes, you must have background job processing with real-time status updates. Users need feedback that things are happening. We built a proper job queue system with progress tracking, and the UX difference is night and day.

The AI Stack is Evolving Fast

Integrating Letta AI, VAPI, Supabase, and computer vision libraries taught us that the AI tooling ecosystem is maturing rapidly. A project like this would have taken months two years ago. Now it's feasible in a hackathon because the infrastructure exists.

Sports Coaches Are Our Heroes

Talking to coaches during development gave us perspective on how hard they work. They're not just teaching sports—they're educators, mentors, and strategists doing three jobs at once. Building something that actually helps them is incredibly rewarding.

🔮 What's Next for TacticoAI

Short Term (Next Month)

Real-time processing optimization: Get processing time under 30 minutes for 90-minute matches
Basketball-specific models: Train dedicated detection models for basketball
Mobile app: Native iOS/Android apps for on-the-go analysis
User testing: Get the platform into the hands of 10 college coaches for feedback

Medium Term (3-6 Months)

Live match analysis: Real-time tactical insights during games with <10 second latency
Advanced tactical metrics: Heat maps, pass networks, defensive line analysis, zone coverage
Team collaboration: Multi-user access with coach/analyst/player roles
Video editing tools: Automatic highlight reel generation for recruitment and teaching

Long Term Vision

Opponent scouting: Automated analysis of opponent footage for game preparation
Player development tracking: Multi-season player improvement analytics
Predictive analytics: ML models that predict opponent strategies based on formation and positioning
B2B SaaS platform: Subscription service for high schools and colleges nationwide
Integration ecosystem: Connect with coaching platforms, recruiting services, and video editing tools

The Big Dream

Make TacticoAI the standard tool for every high school and college coach in America. If we can save coaches 10 hours per week and make their teams 5% better, we'll have impacted millions of student-athletes. That's worth building.

🛠️ Built With

Frontend: React • TypeScript • Tailwind CSS • shadcn/ui • Vite
Backend: Python • FastAPI • Uvicorn
AI & CV: YOLOv8 • ByteTrack • OpenCV • PyTorch • scikit-learn • Letta AI • VAPI
Database: Supabase • PostgreSQL
Infrastructure: Docker • Background Job Queue