Repository navigation
WebMCP accessibility considerations #65
Description
I'm establishing this as a meta issue for discussing WebMCP accessibility considerations.
Earlier today, I had an inspiring discussion with @LJWatson on the WebMCP API and its accessibility-enhancing opportunity. As a well-known leader and expert in the web accessibility community, Léonie can bridge us with more folks working at this intersection. This will help the group deliver on one of its goals to improve accessibility. Léonie has also blogged about accessibility and the agentic web.
I'm documenting here a few things that would further this goal at this stage:
- Review WebMCP use cases through the lens of accessibility.
- Enhance existing and/or contribute new use cases that surface a11y requirements.
- Initiate work on accessibility considerations for the emerging WebMCP draft spec.
Contributions are welcome from everyone. Please feel free to use this meta issue for sharing any ideas that could help further the goal. We will open sub-issues as appropriate.
Reacted by Marcy Sutton-Todd and Sarah Drasner
Reacted by Bogdan Cerovac, Jeff Witt and Sarah Drasner
Reacted by Brian Kim and Sarah Drasner
Activity
LJWatson commentedon Jan 30, 2026
Screen readers depend on platform accessibility API for deterministic information about the UI, and what's exposed by the accessibility API depends on what's in the DOM. AI agents also use the DOM and, I believe, the accessibility API, and so both screen readers and agents are vulnerable when sufficient information is unavailable.
AI agents also use computer vision to analyze screenshots of the UI. For the most part, screen reader users are not able to visually analyze the UI, but screen readers are increasingly using AI and/or incorporating access to LLM for this purpose.
For example, the Jaws screen reader (Windows) has commands for identifying and labelling interactive elements on web pages that are missing an accessible name, and for requesting image descriptions from either ChatGTP or Claude.
It's possible that screen readers could benefit from WebMCP in the same way that agents are intended to. There's the challenge of access of course, but @anssiko, I recall you mentioned that some/all of WebMCP could be made available via browser extensions?
LJWatson commentedon Feb 5, 2026
Tagging @joelamyman and @grace-snow.
m13v commentedon Mar 11, 2026
I've been building MCP servers that use macOS accessibility APIs for AI-driven automation, so I have some practical perspective on MCP + accessibility interaction.
the biggest accessibility consideration from real usage: when an AI agent operates a desktop app via accessibility APIs (AXUIElement traversal, click, type), it's essentially acting as an assistive technology. this means the same attributes that make apps accessible to screen readers also make them automatable by AI agents. there's a direct synergy - better accessibility = better AI automation.
from a spec perspective, it'd be worth considering:
the practical gap right now: web apps expose great accessibility trees via ARIA, but there's no standard way for an MCP-connected agent to consume that tree efficiently. the agent either gets a DOM dump (too large) or a screenshot (loses structure). something in between - a structured accessibility snapshot - would be the ideal MCP tool response format.
m13v commentedon Mar 11, 2026
we open-sourced the MCP server that does accessibility-driven automation: https://github.com/mediar-ai/mcp-server-macos-use - the structured accessibility snapshot format I described is what we use in practice. happy to contribute more concrete examples from real usage if that'd help the spec work.
BogdanCerovac commentedon Mar 19, 2026
@m13v - nice work, thanks for sharing.
I am not a Windows ("native" apps) developer, but I guess that folks could use the Windows accessibility APIs directly, just like you did on Mac.
Besides that - in the web context - I was doing quite a lot of automatic (accessibility and UI) testing within the web domain (Puppeteer, Playwright and Cypress) and know the pain points quite well. A lot of issues because of poor or missing semantics, some websites are very unstable when it comes to knowing when the page is actually "stable" and "ready" and so on...
I am happy to have another option when it comes to MCPs now - as you mentioned that would be ideal - a structured accessibility snapshot can be derived from most of the tools via their MCP (or even CLI) interfaces.
I've tested the Chrome DevTools MCP and got the semantics, so that is a good start;
ChromeDevTools/chrome-devtools-mcp#363 (comment)
The whole accessibility tree is sometimes huge (complex interactive interfaces) and sometimes we do not need it for automation or testing etc, so I asked them for a possibility to query subsets of it here - ChromeDevTools/chrome-devtools-mcp#716 - hope we can get it (it is possible by the Puppeteer already), otherwise it is only a matter of scripting after we get the whole tree ofc.
m13v commentedon Mar 20, 2026
you're touching on exactly the right tension. web a11y testing has the advantage of standardized semantics (ARIA, HTML roles), but the pain points you describe - poor or missing semantics on real sites - are even worse on native desktop apps. at least web has a spec to validate against.
for native macOS, the AX API surface is huge and there's no equivalent of an a11y linter that catches issues before they ship. agents using these APIs end up being the most demanding consumers of accessibility, which is actually a good forcing function for better implementations. the experience from Puppeteer/Playwright/Cypress definitely translates - the core challenge of "find the right element reliably" is universal across web and native.
joelamyman commentedon Mar 30, 2026
I've had chance to work on the first item in this issue, "Review WebMCP use cases through the lens of accessibility", and have put together some feedback. Just to preface this, I work as an accessibility specialist and LLMs/agents/WebMCP is not my area of expertise, so if any of my feedback is not relevant to this work, please feel free to dismiss it as you see fit 😅
I've reviewed the use cases and think that this is really interesting and could offer a lot in terms of helping people interact with the web.
One thing that I’m unsure about, but think it would be really good to reinforce through these use cases is parity between what’s being displayed by the browser and what is being communicated by the agent. I think this would be particularly important for people with anxiety. If the browser or agent is providing separate information, this will make it difficult to trust the agent and may lead to a lack of confidence in the MCP layer working correctly. I wondered if this could be stated, in the interests of accessibility?
I find the shopping example particularly problematic given that the agent offers personal information without checking with the user. I think that there need to be serious considerations around protecting the assistive technology usage and privacy of disabled people using this functionality. One core consideration may be that WebMCP’s should never request information about a person’s assistive technology in order to protect the person’s privacy. This consideration is part of the W3C TAG's Web Platform Design Principles: 2.11. Don’t reveal that assistive technologies are being used. Is it feasible to try and formalise such a consideration like this?
I think that the shopping example is interesting given how well described the clothing items are. I wonder if there’s a way to convey that an LLM should announce this content exactly as a way of conveying critical information to people who are blind or who cannot see the screen, as a way to protect against hallucinations or information that an LLM could try to provide from scanning the related product image. Of course this may be provided as supplemental information, but having an enshrined source for a description may be particularly helpful for providing clarity.
One final thought is that the code review example is great - it demonstrates making a complex tool easy to use. In my experience assessing websites, state information tends to so often be just a visual thing, conveyed in ways like greying out a control, adding an icon, or using an animation. While there’s obviously code somewhere that determines this, there isn’t the connection of the logic through to the front end so that people can programmatically determine this information. Therefore, I think this is something that WebMCP tools could really help improve. To this effect, I wonder if the sort of state checking examples could be added to the other use cases. I think this would be particularly helpful for people using screen readers or people with thinking disabilities. For example, maybe the creative example could feature the WebMCP tool providing state information that makes it clear that an aspect of functionality is currently unavailable due to viewing permissions or something similar? Maybe for the shopping example, the WebMCP tool communicates that an item can’t be added to the basket due to it being out of stock? Adjusting the use cases to account for this would help to reinforce how important communicating state information is, not only for understanding a site/app/product's current state, but also for thinking about subsequent actions that somebody may wish to take.
m13v commentedon Mar 30, 2026
this is excellent feedback, especially coming from an accessibility specialist perspective. the point about AI agents effectively acting as assistive technology users is exactly the overlap I was getting at.
your observations about the use cases are spot on. the form-filling case is a great example - an agent that can read ARIA labels and respect input constraints will fill forms more accurately than one that just looks at visual layout. same applies to navigation - agents that understand landmark roles and heading hierarchy can navigate pages more reliably than pixel-based approaches.
the ARIA live regions point is particularly important for agent workflows. when an agent triggers an action and needs to know if it succeeded, live region announcements are the most reliable signal. we've seen this in practice - agents that monitor aria-live polite/assertive regions for confirmation messages are much more robust than ones that poll for DOM changes.
curious if you've thought about how the proposed MCP tool response format could carry accessibility semantics. for example, when an MCP server returns a page snapshot, including the accessibility tree (roles, states, properties) alongside the visual structure would let agents make better decisions about interaction.
TMILLS333 commentedon May 17, 2026
Exploring this through a working prototype and building on @joelamyman earlier point "parity between what's being displayed by the browser and what is being communicated by the agent". Wanted to share what I've been noticing.
The spec defines how agents register and invoke tools. It's silent on how the human perceives the consequence: what was, what is, who did it, what happened when it failed. Partial failure and interruption are where this gets uncomfortable. When an agent fires a multi-step tool and a step silently fails, the user's mental model of "what is the page now" diverges from reality.
I built a working prototype to make it concrete: https://webmcp-workshop.pages.dev/observable-feedback/. A fake coffee shop where you can toggle which patterns are on, fire scenarios, and watch what the runtime would announce. Tool registration is real. The announcements, undo, and receipt are JavaScript simulations of the proposed behavior, so we can argue about whether the shape is right by experiencing it.
First time contributing. Let me know where I'm wrong or what I'm missing.
mpaiva commentedon May 20, 2026
Thank you for opening this issue. I strongly support bringing accessibility into the WebMCP discussion early, while the use cases and draft specification are still taking shape.
One related area we may want to include is the MCP Apps extension. MCP Apps extend the main Model Context Protocol by allowing servers to return interactive HTML interfaces that render inside an MCP host, such as a chat environment.
Plainly speaking, MCP Apps can turn a MCP response (JSON) into an embedded web app (accessible UI) inside the conversation.
That means accessibility is not only about the text response or tool call. It also includes the embedded interface itself, including keyboard access, focus management, screen reader support, dynamic updates, forms, charts, media, and multi-step workflows.
I think this matters for WebMCP because agentic web experiences may combine browser-level capabilities with app-like interfaces. If we do not define accessibility expectations early, we may unintentionally create new barriers for screen reader users, keyboard-only users, people with low vision, people with cognitive disabilities, and others.
A few questions worth exploring:
I propose that we either add MCP Apps as a related consideration in this meta issue or open a focused sub-issue for accessibility requirements around app-like agent interfaces.
This could help ensure that WebMCP and MCP-based app experiences are designed to be usable by people with disabilities from the beginning, rather than retrofitted later.
matatk commentedon Aug 27, 2026
The Accessible Platform Architectures (APA) WG participants have reviewed the WebMCP CG report and we have the following comments, suggestions, and questions.
Potential
There is a lot of potential for experimentation with new forms of assistive technology (AT) based on this protocol. We say 'experimentation' deliberately, but with cautious optimism. It would be particularly interesting to explore how these adaptations could be of use to people with cognitive disabilities. This would require the input of experts, including those from the Cognitive and Learning Disabilities Accessibility TF.
Key concerns
Divergence between human- and agent-centred web
However we do not support the spec in its current form, and would like to work with the group to address our concerns.
One key issue is that the proposed spec states its purpose is:
However, what is specified is an imperative API that could be used for a wide range of different things, and does not necessarily have any link to the content on the page.
There are some serious risks, which affect accessibility as well as general web usage, namely: the risk of humans and machines being exposed to different content, and capabilities. The consequences could range from user confusion, to either the user or agent being actively misled. This would disproportionately affect some people with disabilities negatively.
Support of the need for human review
WCAG 2.2 Success Criterion 3.3.4: Error Prevention (Legal, Financial, Data) (Level AA) states:
It is critical that users have the option to review what the agent is proposing to submit on their behalf. Whilst it is incumbent upon web sites to provide a dedicated review/rollback mode for certain ‘critical’ types of transaction (as above), WebMCP must support the user in this (preferably in any transaction, when the user has expressed a preference to be able to review the input before submission). How would WebMCP support this?
Perhaps it may be easier to consider agents embedded in the UA (which would be more controllable by the UA?) separately to ‘Bring Your Own Agent’ scenarios, in which the agent is external to the UA, and could control it either via established protocols, or by simulating human input (which the UA would be unable to reliably detect, and protect against). The latter scenario may need assistance from the OS if there is a need or desire to block agents from simulating human input.
Declarative version
The declarative version of the API would address a number of these concerns: it ties the agentic part to the content on the page (there are ways this could be abused, as noted below, but it's a much more solid starting point). This much more closes achieves the stated goal of the spec.
We would like to see the following work done on the declarative version:
We would still advocate for this to be considered experimental until some time has passed, and we have seen how these attributes are used in the wild; what sorts of ATs may be possible here; and how effective they actually are.
We do think that this should be a set of (experimental for now) extra attributes, and that the problems being solved here are not the same as those solved by the accessibility tree. We are also very concerned that ARIA and other accessibility-related attributes would be abused if developers saw them as serving a dual purpose.
If some sort of semantic info (like the role attribute, or an accessible name set by the author) benefits a machine, then great. But it should be for the human user first.
Fundamentally, as agreed in Redundancy with the accessibility tree #91, the WebMCP info and the accessibility tree are doing different jobs, so should be kept separate.
It seems like it should be possible to tweak the existing submission event, as well as the proposed new one, to ensure that the UA could control whether a human gets to review the form before it's submitted. This would be necessary to keep the human in the loop.
For users, it'd be good to expose whether a form field was auto-filled semantically (i.e. this is appropriate to expose through the accessibility tree). It's a very early idea, but maybe an enumeration would be appropriate, so we could know that the field was autofilled either by one's password manager, or UA agent, for example.
We had some thoughts about how the declarative version may support some more advanced use cases:
Dynamic SPAs can leverage progressive disclosure (e.g., state-driven form mounting). Discover a list of actions (as buttons or links), select one, get a form based on current application state.
Progressive enhancement works here for cases where the user has an attached agent manipulating a browser and JS is disabled. Forms rendered from the server side can still be used by both humans and agents.
Client-side validations of forms (even default validations using the native validity API) offer more assurances to operators that they won’t waste processing from errant/invalid tool use.
JSON (show code) views are easy to implement for the truly lazy form. JSON is at least somewhat human readable and with a small, generic, client-side library can be rendered with collapsible sections for human review.
Use case coverage, and extending form actions
Focusing on the declarative version would limit the number of use cases that could be covered (it would be good to get more documentation on the anticipated cases, as mentioned below). However, it also increases the options for human oversight, and ensuring that content is not fragmented between humans and agents.
In order to support activities that are more interactive than traditional form-filling, could you explore the possibility of adding additional form action types that are aimed at providing input to Agents? For example, a form (visible or perhaps otherwise) could be used to set up the parameters for an action to be carried out by the AI, and an action value could be added to allow the triggering of that action.
For filtering and sorting a table, for example, a form could be created to allow the user - or an agent - to control the nature of the filter/sort. Then the agent could be triggered to instigate the filter and sort.
This does present risks (potential arbitrary code execution) but we wonder if it was considered as an option? If it's possible to implement it in such a way that the input to the code can be verified by a human, it would provide more reassurance than the pure-JS function calling approach in the proposed spec.
Forms are generic data-input contracts capable of handling complex structures including spatial coordinates or raw JSON.
Potential deceptive patterns
There are some techniques authors could use that go against the spirit of the spec, which we would need to actively discourage, including:
An HTML form created 'for agents' that has
aria-hidden=trueand/or is not visible for human visitors.An MCP capability advertised using modelContext but not available to human visitors.
Any MCP capability not 'hinted' to AT users so that they can turn their agents 'on' and use the site's features.
Toolautosubmitoffers a dangerous vector for deceptive patterns similar to click-jacking and homograph attacks. Without assurances that the action being taken and the data being used are limited to the human-reviewable elements, this would present too high a risk on untrusted, or potentially compromised, sites.Accessibility and Usability questions for the spec
How will a person know that an agent capability exists on a given page? How can the UA mediate disclosure?
The spec allows capabilities to be declared in a way that only a machine can read. Without something visible or perceivable to a human, a page could offer abilities to an agent that a person never learns about (per the deceptive patterns, above). This recreates the exact gap between human and machine experience that this spec aims to close.
Browsers already handle similarly sensitive situations, such as camera, microphone, or location access, through a standard permission interface. Could the same, or a similarly user-visible, approach be adopted here?
How will a person review and approve an action before it happens?
Several ideas already discussed, such as keeping a human in the loop before form submission, depend on a real moment where a person can see what is about to happen.
The person needs to be able to choose to allow the action or not; that moment only exists if there is an interface for it. An API alone cannot provide a decision point for a person.
How will a person cancel or undo an action once it has started?
In order for a person to cancel or undo an action, the following are needed:
Indication that the action has happened/is happening (and a ‘breakpoint’ to allow a human to become aware of this before submission).
Some UI to allow the human to cancel or undo the operation.
This should not be left to each site to invent on its own.
How will people who use assistive technology get the same information as people who do not?
A sighted person using a mouse might notice small visual hints about what is happening on a page, even without a formal mechanism. A person using assistive technology only knows what is explicitly exposed to them.
If this is left to individual developers, assistive technology users are the ones most likely to be left out, which is the opposite of what this spec aims to achieve.
This also relates to indication as to how a field was filled in (human, password manager, agent, …)
Suggestions for the proposed spec document
We would like to know more about the specific use cases that the proposed spec is envisaged to address. This may help the reader better understand the context and purpose of the document and proposed spec.
Acknowledgements
This comment was drafted and reviewed by the APA WG participants.
Thanks to @newtonsgroove for highlighting the importance of review, as noted in WCAG.
Thanks to @AutoSponge for the idea of exploring extending form actions to support a wider variety of tools/tasks.
Thanks to @mpaiva for Accessibility and Usability questions for the spec.
chrishtr commentedon Sep 11, 2026
Hi, thanks for the APA WG analysis. Here are some responses regarding the concerns raised:
The risk of features being offered to agents that are not directly supported by visible UI is a potential concern that is not specific to a11y (“general web usage”, as mentioned). ATs can and should integrate with browser agents and other agents that use WebMCP, and there is no reason they cannot do so (Gemini-in-Chrome's UI is already accessible to ATs, for example).
In addition, WebMCP tools are visible to browsers and ATs even without the use of AI, allowing such browsers/ATs to display custom UI to users if desired. For example, they could display the list of available WebMCP tools & descriptions in a menu and provide a UI for filling and executing them. In that sense, WebMCP tools are still visible to and actionable by the human-centered web despite not being in the visible DOM.
Browsers can and should provide UI asking users for their informed consent before calling any WebMCP tool, especially those with consequences. This can be done with browser UI.
In general, I agree with the advantages described of the declarative form of the API, and am not opposed to the proposed possible extensions to form controls that make them richer and more useful.
Exposing to ATs whether form controls were filled by the browser is worth considering (see here for one issue discussing aspects of it), though there may be risks to this feature in other dimensions.
Regarding possible abuse of existing ARIA attributes: I agree, and that’s why the declarative WebMCP API proposes new and separate HTML attributes.
Regarding a WebMCP API extension to allow the website to suggest actions for the agent: this sounds like a good idea to consider. This issue looks related.
I agree that there are deceptive patterns that should be actively discouraged through conformance checkers, developer tooling, and real-time browser security and safety checks.
I also agree that, used indiscriminately and without safeguards,
toolautosubmitcould be dangerous (as would calling an imperative WebMCP tool call without safety & security checks, and asking for informed user consent). Therefore browsers should always implement measures to protect the user.Websites utilizing WebMCP will indeed offer tools that do not correspond directly to user-visible UI as reflected via DOM on the website. This is an important feature of WebMCP, as it allows website developers to offer more complex or specialized tools to users to accomplish higher-level tasks via natural language rather than a complex and hard to learn UI. There is no reason these tools cannot be be fully available to ATs via the user’s natural language (as expressed via the user’s preferred input method of visual selection, voice, keyboard, or other modality in ATs), and therefore all users benefit from LLM technologies to map their natural language input into the proper tool calls. (Current screen readers don’t to my knowledge have agent-style natural language input, but there is no reason they can’t follow similar paths to existing LLM agents (ChatGPT's desktop app, for example) that integrate with browsers.)
In fact, this kind of natural language input is one way in which WebMCP should be a great help to a11y generally.
Additionally, when WebMCP tools are present, the need to explain them to users applies to all users, not just AT users, and that should happen via a combination of (automatically-accessible) browser UI and website-provided semantic HTML.
This should be done via browser UI asking for user review and confirmation, and/or via in-website semantic HTML once a tool has been called presenting the user with an opportunity to do the same. The spec already has annotations that can be used by a UA to help trigger such confirmations, and additional hints have been proposed.
Canceling or undoing an action with side effects on a website requires the website to implement a UI affordance (or offer a WebMCP tool that does the same), because only the website knows what those side effects were and how to undo them.
If the tool had no side effects, then the website could push a history entry upon execution of the tool so that the back button would “undo” (similar to how browser default form control submission works). The declarative WebMCP API does this as well by default.