-
-
Live Splunk Enterprise: 218,456 DNS events in the BOTSv3 index — the real data the agent investigates, no mocks.
-
Architecture: analyst briefing to agent loop, tool dispatch through the Splunk MCP server, MITRE-mapped verdict, sealed audit log.
-
Completed triage: confirmed DNS-tunneling C2 verdict with confidence, MITRE techniques, and an entity graph linking hosts to the C2 endpoint
-
The instrumented agent loop: structured-output parsing, MCP tool dispatch chokepoint, and the verdict-harvest path that feeds the dashboard.
-
Per-iteration audit trail: every SPL call logged with outcome, latency, tokens, and cost — sealed into a tamper-evident log.
Inspiration
SOC analysts drown in alerts. Triaging a single incident means pivoting across dozens of searches, remembering field schemas, and manually mapping behavior to MITRE ATT&CK — under time pressure, repeatedly. I wanted to see how far an autonomous agent could go on a real SIEM: not a scripted demo over canned data, but an agent that forms its own hypotheses, writes its own SPL, queries a live Splunk instance, and reasons to a defensible verdict — with every step auditable.
What it does
Splunkology is an autonomous SOC triage agent. You give it an incident briefing and a Splunk data source; it runs a multi-iteration investigation against live Splunk Enterprise and returns a structured verdict.
In a single run it:
- Forms an initial hypothesis from the briefing
- Writes and executes its own SPL searches against live Splunk (via a Splunk MCP server)
- Reads results, discards false leads (NetBIOS noise, forwarder traffic, legitimate CDN lookups), and re-pivots
- Confirms the threat, identifies compromised hosts, and maps the behavior to MITRE ATT&CK techniques
- Produces a verdict with a calibrated confidence score, supporting SPL evidence, and an entity graph linking hosts, the C2 endpoint, and techniques
- Seals every tool call in a tamper-evident audit log
Run against the BOTSv3 dataset (218,456 DNS events), it independently uncovers DNS-tunneling command-and-control — confirming the beacon, the compromised endpoints, and the ATT&CK mapping without human guidance.
How we built it
The core is an instrumented agent loop running on Claude (Sonnet). Each iteration parses a structured output schema, accumulates findings, and decides whether to search again or conclude. Tool dispatch routes through a Splunk MCP server behind a single chokepoint, so every orchestrator hits Splunk the same way.
The stack: Python 3, FastAPI + a streaming SSE dashboard, Pydantic schemas for structured agent output, a per-call MCP stdio client, and a SQLite tamper-evident audit log. Splunk Enterprise runs in Docker against the BOTSv3 dataset. Orchestration is pluggable — Native Claude loop, LangGraph, OpenAI function-calling, and Gemini — all behind one interface routing through the same MCP server, so the orchestration paradigm is the only variable.
Challenges we ran into
The hardest problems were on the live async path, not the agent logic. The streaming dashboard surfaced a class of bugs that offline tests never caught because they only appeared when events crossed a thread boundary under a real server: an MCP stdio client that deadlocked when a Splunk error filled an undrained stderr pipe; an error-path that raised before a response variable was assigned; a verdict-synthesis fallback that looped instead of concluding; and a frontend watchdog that killed legitimately slow synthesis turns. Each was diagnosed from real tracebacks, fixed with an offline test proving the fix first, then confirmed on a single live run — keeping paid agent runs to a minimum.
Accomplishments that we're proud of
A fully autonomous investigation against real Splunk data that reaches the correct verdict and maps it to MITRE ATT&CK — with a tamper-evident audit trail and a live streaming view of the agent's reasoning. The MCP integration covers every orchestrator through one injection point, and the whole thing is offline-tested with CI coverage.
What we learned
Offline-green is not live-green: path-dependent async bugs only appear on the real wire, so the discipline of proving a fix offline and confirming it live matters. Calibrated uncertainty also beats false confidence — an agent that reports a justified 70–93% with evidence is more trustworthy than one that always claims certainty.
What's next for Splunkology
Hardening the alternate orchestrators to full parity with the Native loop, expanding beyond DNS to multi-sourcetype attack chains, and adding analyst-in-the-loop feedback so verdicts can be confirmed or corrected and fed back into the investigation.
Log in or sign up for Devpost to join the conversation.