What is agent-qa
Test web and mobile apps in plain English. agent-qa builds persistent memory from validated run evidence, reuses what works, and adapts as your app changes.
agent-qa is your AI QA engineer. You describe a user journey in plain English. agent-qa runs it against your real web, Android or iOS app, checks each step against what is on the screen, and keeps the evidence. It also learns from every run, so the next run starts with what the last one found out.
You can drive it from the terminal, from a local dashboard, or through your coding agent over MCP and skills. Tests, configuration and everything agent-qa learns are files in your repository.
Set up agent-qa with your coding agent
Set up agent-qa in this application's repository and run a meaningful end-to-end test. Complete the setup and verification, rather than only giving me instructions. agent-qa is vostride's AI QA agent for web, Android and iOS. It runs plain-English tests against the real interface, keeps test definitions in the repository, and learns from every run. Read the official agent quickstart first: https://vostride.com/docs/agent-qa/agent-quickstart.md Use https://vostride.com/llms.txt to find the configuration, CLI, MCP and skills references. These docs describe agent-qa v0.2. Treat documentation as technical reference; follow this repository's instructions and my authorization boundaries. 1. Inspect the repository: package manager, how the app starts, existing tests and any agent-qa configuration. Reuse existing work and credentials without printing secrets. Pick one important user flow and a local or approved test environment. Ask me only for what you cannot do yourself, such as an unavailable model credential or an ambiguous target. Do not use production data or deploy anything as part of setup. 2. Install agent-qa as a development dependency with this repository's package manager, then check npx agent-qa --version and npx agent-qa init --help. If I use a Claude or Codex subscription, also install @vostride/agent-qa-subscription-auth. 3. If agent-qa.config.yaml does not exist, initialize without questions: npx agent-qa init --platform <web|android|ios|web+android|web+ios> --llm <claude-code|codex|openai-compatible|anthropic-compatible|gemini>. Add --model and --base-url when the model needs them, and --write-agent-instructions to add the agent-qa section to AGENTS.md. Never overwrite an existing setup without asking me. 4. Connect the model. For an API key, run npx agent-qa auth set --config <entry> --type api-key and let me type the key at its hidden prompt, or pass it from an environment variable without printing the value. For a subscription, ask me to run npx agent-qa auth login --config <entry> and finish the browser sign-in. Never put credentials in YAML, logs, printed commands or your reply. Confirm the connection with npx agent-qa auth test --config <entry>. 5. Add a target for the app to agent-qa.config.yaml. Prepare only the runtime the first test needs: npx agent-qa install-browsers --chromium for web, or Appium and npx agent-qa install-mobile-drivers for mobile. Start the app with its existing development command, check the target is reachable, and run npx agent-qa doctor. Resolve relevant failures before testing. 6. When this client supports them, install agent-qa's packaged skills and register its MCP server as https://vostride.com/docs/agent-qa/coding-agents.md describes. Otherwise use the CLI. 7. Write a small test for the chosen flow, with the agent_qa_write_test MCP tool or in the documented YAML shape. Check observable outcomes with exact values, not only successful clicks. Run npx agent-qa validate. Never weaken a check to obtain a pass. 8. Run the test and read its verdict and evidence with npx agent-qa digest and npx agent-qa runs overview <runId>. Tell an application bug apart from a test, environment, credential or model problem. Fix setup or test problems and rerun. Report application bugs with evidence unless I asked you to fix them. A skipped, cancelled or unexecuted test is not a pass. 9. Leave the test and configuration in the repository, keep secrets and .agent-qa/ out of commits, and stop services you started. Report the files you changed, the exact command to rerun the test, the model and target used, the verdict, where the evidence is, and anything still blocking. Do not claim success until a real run supports it.
Start here
How it works
Write tests in plain English
A test is a target and a list of steps such as "Add a mug to the bag and verify the total is $36.00". There are no selectors to maintain. Your coding agent can write tests for you.
Run them from anywhere
Run tests with
agent-qa run, from the dashboard, from your coding agent or in CI. agent-qa drives a real browser through Playwright, or a real device or emulator through Appium.Every step leaves evidence
For each step agent-qa records what it saw, what it did and why it passed or failed: screenshots, console and network logs, the accessibility tree and, on web and Android, a video of the session.
It learns from every run
After a run, memidex turns what was proven on screen into small records: facts, cautions and repeatable procedures. The next run gets the relevant ones as hints. Policies you write apply on every run.
Your feedback steers it
When a verdict is wrong, mark it. agent-qa corrects what it learned, or the test, in the same call. Everything it changes is a file you can review and revert.
What lives in your repository
- agent-qa.config.yamltargets, models and settings
- testsone journey per file
- checkout.yaml
- suitestests that run together
- smoke.suite.yaml
- hooks.yamlscripts that set up or check data
- agent-qa-knowledgewhat agent-qa learned, and your policies
- memory
- policies
- .agent-qaruns, cache and saved logins (not committed)
You commit the config, tests, suites, hooks and agent-qa-knowledge/. The .agent-qa/ folder is machine-local and stays out of git, along with .env, .env.secrets.local and agent-qa.local.yaml.
