MemoryHound | Devpost

Inspiration

Inspired by Daniel Miessler's PAI pattern: the .claude/ directory plus the custom MCP server give the AI driver the skills, subagents, hooks, and forensic primitives to operate as a domain specialist, without modifying the underlying model. The hackathon prompt was the spark: AI threats strike in minutes, build the defender that responds in seconds. Real incident response is slow because a human analyst drives a dozen forensic tools by hand. We asked a simple question. What if the AI drove the tools, and a second AI checked its work?

What it does

MemoryHound turns any major AI engine into an autonomous DFIR analyst. You drop evidence into a folder, run one command, and get back a structured forensic report whose every claim is pinned to a specific tool call against a specific artifact.

The custom MCP server's 47 forensic tools are the substrate. The AI driver is swappable: Claude Code, Anthropic API, OpenAI, or Ollama running locally on a laptop. Same orchestrator, same findings.json, same audit trail, regardless of engine. A judge with a Claude subscription, a shop on OpenAI, and an air-gapped DFIR lab on Ollama all run the same ./bin/mh run command.

It covers the whole forensic spread: Windows (registry, EVTX, Prefetch, MFT, Amcache via EZ Tools), macOS, Linux, memory (Volatility 3), disk (Sleuth Kit), timeline (Plaso), and network (tshark, Zeek). Every run maps to NIST CSF 2.0, ISO 27035, SANS PICERL, MITRE ATT&CK, and D3FEND.

How we built it

We started on April 25 with one idea borrowed from PAI. Don't build a new model. Wrap a general one in the right scaffolding. Our first instinct was skills. Claude Code makes it easy to write a skill for everything, so we did. A skill per artifact type, per analysis step, per OS. It worked, but it burned tokens fast. Every skill loads its instructions into the context window, and a full triage run was dragging dozens of them along.

So we leaned on MCP tools instead. An MCP tool runs as code and only returns the result, which is far cheaper than a skill that reasons in-context. Parsing a Windows registry hive as a skill means the AI reads pages of instructions and works through the hive itself, spending tokens the whole way. As an MCP tool, it's one typed call that runs the forensic parser under the hood and hands back clean structured data. Same answer, a fraction of the cost.

But we couldn't build an MCP tool for everything. Some work is genuinely judgment. Routing a case, weighing conflicting evidence, writing the narrative. That belongs in a skill, not a hard-coded function. So we settled on a hybrid. Deterministic MCP tools for the mechanical forensic work (47 typed tools across 13 modules, covering Volatility 3, Sleuth Kit, Plaso, EZ Tools, Zeek), and a smaller set of skills for the reasoning the AI is actually good at.

We kept the brain swappable on purpose, and that wasn't planned from day one. Early on we relied entirely on Claude, and one day Claude was down. We couldn't test, couldn't run a single case, completely stuck. That was the wake-up call. A forensic tool can't have a single point of failure. So we built the driver to be provider-agnostic, switchable with a one-line config change (MH_PROVIDER). On top of that we shipped two modes: mh run for exploratory work, and mh orchestrate, a deterministic 20-node LangGraph state machine for repeatable, framework-aligned IR.

Challenges we ran into

Skills were too expensive. Our skill-everything first draft worked but burned tokens on every run. Moving the mechanical work into MCP tools and keeping skills only for judgment was the fix, and finding that line took real iteration.
"How do we know it's right?" After a few full runs, we hit the real question. The report looked confident, but how do we trust it? That's when we added an independent Verifier subagent, a second AI that reviews the findings and can dissent. We then had to make dissent actually re-run the analysis, not just log a complaint.
Proving every claim. We made the schema reject any finding not pinned to a specific tool call on a specific artifact, and chained the audit log with SHA256 so it can't be quietly edited.
Subagents that hang. Long forensic jobs stalled on large prompts, so we built a liveness monitor that watches CPU and output and group-kills stuck processes. We also made the pipeline fail open on timeout. Degrade and disclose the gap, never silently drop a finding.
One report from four engines. Claude, OpenAI, and Ollama all behave differently, so getting them to produce the identical pinned findings.json took a native-plus-text-fallback tool-calling layer.

Accomplishments that we're proud of

- It works on a real challenge image with known ground truth. On a ~19 GB Windows memory dump, it produced 16 pinned findings, confirmed the RDP intrusion across 9 findings each backed by two or more independent in-memory artifacts, and openly declared five named gaps (full report below). - Engine independence that works. A judge on Claude, a shop on OpenAI, and an air-gapped lab on local Ollama all run the identical command and get the identical contract.

Every claim is provable. Un-pinned findings can't exist in the output. Add a SHA256 manifest at ingest and a mh verify chain-of-custody re-check, and the evidence handling holds up.
A real engineering bar. Roughly 12K lines of Python, 60+ test files, 435 tests, CI green.

What we learned

The model isn't the bottleneck anymore. The scaffolding is. But scaffolding has a cost, and we learned the hard way that more isn't better. Skills are powerful and easy to write, which is exactly the trap. Every one you add quietly eats your context budget. The real skill was knowing what to make a cheap deterministic tool versus what to leave to the AI's judgment.

Never depend on one model. In the early phase we ran everything through Claude, and the day it went down, our whole project went down with it. That one bad day taught us the most important architectural lesson of the build: no single point of failure. We rebuilt the driver so any model can run the same job, with only a one-line config change to swap engines.

We also learned that in forensics, trust beats cleverness. Adding the Verifier wasn't a feature we planned. It came straight out of staring at a confident report and asking "but is it true?" Most of our hardest work went into removing room for the AI to be wrong. Rejecting unpinned claims, forcing the verifier to re-analyze, refusing silent fallbacks.

What's next for MemoryHound

More evidence types and faster runs on bigger images.
Tighter framework reporting so an analyst gets a complete package, not just findings.
Broader local-model support so the air-gapped path keeps pace as open models improve.
A simple dashboard so non-experts can read the findings, not just specialists.