Repository navigation
Supporting interoperable probabilistic defense structures through shared attack eval datasets #106
Description
As noted in various discussions (most recently in #97), it's unlikely that security challenges around WebMCP can be addressed purely through deterministic specification rules. Browser and/or agent developers will have to deploy custom probablistic defense architectures, such as improved model understanding, "critic" models that prevent misaligned actions, or sophisticated tool security architectures like CaMeL.
This risks being a departure from the traditional Web specification security workflow: New implementers of the specification would not be able to use some of the lessons learned by early adopters and the community in delivering a secure feature.
One thing we can (and should) do is write very good and precise security considerations. However, there are limits to that, particularly in that it is hard to quantify what e.g. "should be resistant to prompt injections" means in practice.
As such, I think this group should embrace the default approach by agent makers to quantify the performance of an agentic system: evaluations. A robust dataset of prompt injection attacks that any implementer of WebMCP MUST (or SHOULD?) prevent with >99% recall would be a good and pragmatic step forward in interoperably enabling security for WebMCP.
Obviously there are a lot of open questions here around both maintenance and architecture for running these evals.
Activity
victorhuangwq commentedon Feb 25, 2026
thanks for raising this. I agree that deterministic spec rules and well documented considerations might never be fully sufficient. evals does (at this moment in time) seem like a pragmatic path forward.
a few thoughts here:
I think a publicly available "eval set" would be useful, both as a security baseline, and also as a way to build cross-vendor developer confidence in webmcp. kind of akin to wpt tests almost.
I would frame the eval as the following three possible layers:
1 and 2 feels close to necessary for this to be meaningful. 3 is worth discussing as well - a fair evaluator can help ensure consistency, but it could encode implicit model preferences at odds of being not vendor neutral (maybe more of a BYOM system?).
on broader framing, I would strong encourage this to be perceived as part of a swiss cheese model - one layer of defence amongst many. Passing the evals shouldn't be conflated with being secure.
some other open questions worth considering:
johannhof commentedon Mar 2, 2026
The nice thing about security-related evaluations is that rating can often be done deterministically - there's a bad type of information to leak or a bad tool call to make which can often just be validated via checking a regex of proposed actions or having an event listener trigger some test failure function. Maybe this is what we should go for in the initial iteration.
Yes, I agree, one question is how do we give other ecosystem implementers the confidence that this is a safe technology for them to implement without building up their own swiss cheese architecture for years to pass these evals. Ideally we could define a deterministic / HITL security model for the spec that would always pass these evals, and then have increased autonomy for agent providers that are willing to ensure that their agents have enough security mechanisms to still pass.
Yes this is a hard question we'll have to figure out - I would assume that the spec evals are generally "lagging" real world findings by quite a bit of time.
npdoty commentedon Jul 13, 2026
Does protecting against a known attack 99% of the time seem like an effective mitigation? How many users per day should have their browsing history and pregnancy status emailed to an attacker when they encounter an already-documented-and-protected-against malicious prompt?
jpagnucco commentedon Jul 14, 2026
You are highlighting why we should not rely on evals as a standalone defense against over-parameterization threats. In addition to evals as a benchmark, we recommend 1) pre-execution guardrails (like a critic) and 2) deterministic boundaries (like
exposedTo).realArcherL commentedon Sep 17, 2026
Following the layering in @victorhuangwq's comment, I have three small pages that might fit layer 1. Also, shared these as part of the W3C meeting CC: @jpagnucco 🙏
https://realarcherl.github.io/WebMCP-Security-Testing/
They came out of a security and privacy review of the draft. Each one is the smallest page I could write that reproduces a single behaviour, and each is self contained with a trimmed copy of the reference polyfill, so they run without the flag:
search-webtool from the prompt injection section verbatim, and showsregisterTool()resolving.<head>takesadd_numbersfirst, and the site's own registration then fails.On @johannhof's point about deterministic rating, the first two can carry a pass or fail signal without a critic. The first fails if a decoy send tool is ever invoked. The second fails if the second tool is called when the task only asked for the first. Both are a flag set inside
execute.The third one is probably not an eval. Its outcome does not depend on the model, so it may belong in web platform tests instead.
Happy to reshape these into whatever format the group prefers. Two questions: where should they live, and is
registerTool()level behaviour in scope for this corpus, or only agent behaviour?