MacroLOL | Devpost

Inspiration

Elon posted on X about Grok 5 beating the best human League of Legends team by 2026, only seeing the monitor through a camera, reacting no faster than a human could. That got us curious about what's possible today.

What it does

A macOS app that lets Grok play League of Legends autonomously. It runs a closed loop - capture screenshot, send to Grok for analysis, execute the action Grok decides on, capture the result, repeat. The system prompt uses scenario-based instructions with hardcoded coordinates covering positioning, ability usage, chasing, retreating, and farming.

How we built it

Built with Electron, using a transparent overlay window that stays on top of the fullscreen game. The agent loop captures a screenshot (resized to 1600px and converted to 80% JPEG to keep payloads small), sends it to grok-4-1-fast with a League-specific system prompt, parses tool calls from the streaming response, and executes actions using nut.js for mouse/keyboard control. Gaming mode uses grok-4-1-fast-non-reasoning for faster inference.

Challenges we ran into

Model speed vs reasoning quality. League requires making complex decisions quickly: where to move, when to attack, which ability to use. We use grok-4-1-fast-non-reasoning to reduce response time, but that trades off action quality. Building up gameplay history (screenshots after each action) helps the model understand what's happening, but the growing context adds latency.