PitPerfect | Devpost

Inspiration

In Formula 1, milliseconds can decide championships—and millions of dollars. Teams rely on human judgment and fragmented tools to make critical pit stop calls. After watching several races where delayed or premature pit stops changed the outcome, we wondered:

What if AI could combine visual analysis, telemetry, and strategy to make those calls faster and smarter?

That question inspired PitPerfect—an AI-powered race strategy assistant that brings data-driven intelligence to the pit wall.

What It Does

PitPerfect analyzes race footage and telemetry data to detect car damage, assess severity, and recommend pit stop strategies in real time.

Key features include:

AI Damage Detection: Uses Google Gemini 2.5 Pro model to identify damaged parts from multi-frame race footage with severity and confidence scores.
3D Visualization: Displays an interactive Formula 1 car model, highlighting damaged areas with color-coded severity (green → red).
Telemetry Analyzation: Generates realistic analysis based off real F1 data patterns (speed, temperature, throttle data, etc.) across laps to show performance impact.
Decision Engine: Combines AI reasoning and budget constraints to recommend whether to “Pit Now,” “Monitor,” or “Stay Out.”

How We Built It

We built PitPerfect using a Next.js 15 frontend powered by React 19, TypeScript, and Tailwind CSS for UI.

3D Modeling: Implemented with Three.js and React Three Fiber, using our custom F1 car model for real-time damage visualization.
Backend: Built on FastAPI (Python) with endpoints for AI inference, video processing (via OpenCV), and telemetry visualization (NumPy).
AI Integration: Leveraged Google Gemini 2.5 Pro for visual reasoning and natural language explanations.
Data Visualization: Used Recharts for telemetry graphs and Framer Motion for smooth dashboard animations.

We also implemented a simple mathematical model to quantify pit strategy decisions—analyzing the difference in time per lap if we made the pit stops and whether that was a smart repair to make based on our budget for the race and the season.

Challenges We Ran Into

Model Mapping: Aligning GLB part names with real car components required deep debugging and manual part inspection.
Performance Bottlenecks: Processing multi-frame video data while maintaining frontend interactivity was challenging.
AI Reasoning Consistency: Balancing the model’s natural language explanations with concise, actionable recommendations took several iterations.
Telemetry Data: Analyzing telemetry data from real F1 patterns demanded mathematical tuning and smoothing.

Accomplishments That We’re Proud Of

Created a fully functional AI-driven pit strategy assistant in under 36 hours.
Built a 3D interactive dashboard that visualizes real-time car damage and telemetry.
Integrated Gemini 2.5 Pro for multi-modal reasoning—a first in our hackathon experience.
Developed a complete data pipeline from video input, damage analysis, and pit recommendation with both visual and textual outputs.

What We Learned

How to integrate multi-modal AI models (text + vision) with real-world data.
The power of 3D interfaces for making complex data more interpretable.
The importance of designing AI explanations that are transparent and human-readable.
How to work efficiently as a team under time pressure while merging technical and creative workflows.

What’s Next for PitPerfect

Live Stream Integration: Analyze damage in real-time during races with video and telemetry data.
Multi-Car Comparison: Compare damage and performance across teams.
Weather & Track Modeling: Incorporate conditions into strategy predictions.
Historical Analysis: Train custom ML models on past F1 crash datasets for predictive insights.
Beyond Racing: Expand to NASCAR, Formula E, and aerospace maintenance analytics.

PitPerfect is just the beginning of a new way to fuse AI, strategy, and performance—where technology helps make the fastest calls in motorsports.