E2E Test Commands

Norrix runs E2E tests against completed virtual-device builds on Norrix cloud runners, with either runner:

• Maestro : YAML flows.
• TesterArmy e2e : TypeScript tests that mix plain-English agent steps (agent.act, agent.assert) with exact locator checks, driven on the simulator or emulator through @e2e-dev/mobile.

E2E tests are available to Enterprise organizations.

norrix test

Run E2E tests against a completed build.

Synopsis

norrix test [platform] [options]

Prerequisites

E2E tests need an artifact that can install on a virtual device:

• iOS: build with norrix build ios --simulator
• Android: build with norrix build android --emulator or produce an APK

Maestro flows live in a local directory, usually .maestro/ or a project-specific folder such as apps/mobile-e2e/src/flows. A TesterArmy e2e suite lives in its own directory, usually e2e/; see TesterArmy e2e suites.

Choosing a Runner

--runner maestro or --runner e2e picks the runner. Without it, the CLI uses the runner your options belong to (--flows-dir and --flows are Maestro; --suite-dir, --target, --tests, --grep, and --tag are e2e), then a suite it finds (e2e.config.ts in the current directory or ./e2e). It asks when the project has both .maestro/ and a suite, and otherwise runs Maestro.

Arguments

If platform is omitted, the CLI can prompt you to select from available testable builds.

Options

Examples

Interactive Run

norrix test

The CLI lists successful testable builds and runs all flow files in .maestro/.

CI-Gated iOS Run

# 1. Build a simulator artifact.
norrix build ios debug --simulator \
  --project my-app \
  --config stg \
  -V 1.2.3 \
  -B "$BUILD_NUMBER" \
  --org "$NORRIX_ORG_ID" \
  -n
 
# 2. Run selected Maestro flows and publish JUnit from the output directory.
norrix test ios \
  --build "$NORRIX_BUILD_ID" \
  --flows-dir apps/my-app-e2e/src/flows \
  --flows 01-launch.yaml,02-login.yaml \
  --device-model iPhone-16-Pro \
  --device-os iOS-26-2 \
  --wait \
  --timeout-minutes 50 \
  --poll-interval 20 \
  --junit-output ./norrix-e2e-junit \
  --org "$NORRIX_ORG_ID" \
  -n

Android Run

norrix build android debug --emulator --project my-app --config stg -n
 
norrix test android \
  --build "$NORRIX_BUILD_ID" \
  --flows-dir e2e/maestro \
  --wait \
  --junit-output ./norrix-e2e-junit \
  -n

TesterArmy e2e Run

norrix build ios debug --simulator --project my-app --config stg -n
 
norrix test ios \
  --build "$NORRIX_BUILD_ID" \
  --suite-dir e2e \
  --tag smoke \
  --wait \
  --junit-output ./norrix-e2e-junit \
  -n

Device Pinning

--device-model and --device-os are iOS runner hints. If either option is provided and the runner cannot find a matching available simulator, the run fails instead of silently using a different device.

These options are useful when flows depend on stable screen geometry, for example coordinate-based login steps.

Test-Time Environment Variables

The E2E workflow injects Norrix environment variables before the tests run. For TesterArmy e2e suites, see Model credentials and secrets.

For Maestro, Any injected variable whose name starts with MAESTRO_ is passed to maestro test with -e, so flows can reference values such as ${MAESTRO_USERNAME}.

Set org-global values when every project can share the same test account:

norrix env set MAESTRO_USERNAME "$MAESTRO_USERNAME"
norrix env set MAESTRO_PASSWORD "$MAESTRO_PASSWORD"

Set project-scoped values when one app needs different credentials:

norrix env set MAESTRO_USERNAME "$MAESTRO_USERNAME" --project my-app
norrix env set MAESTRO_PASSWORD "$MAESTRO_PASSWORD" --project my-app

Project-scoped values override org-global values with the same name.

Exit Behavior

• Without --wait, norrix test exits after the remote run is scheduled.
• With --wait, the CLI exits non-zero when the run fails, is cancelled, or times out.
• With --junit-output, the CLI downloads JUnit XML from Norrix when reports are available. Your CI system can publish *.xml from that directory.
• For e2e runs, --wait also prints the run’s results: passed, failed, and flaky tests, model tokens, estimated cost, and replay cache hits.

TesterArmy e2e Suites

TesterArmy e2e  is an open source agentic test runner. Norrix runs your suite as-is: the runner installs its dependencies, installs your build on a booted simulator or emulator, and runs e2e run for the targets on the build’s platform.

For a step-by-step setup with an example test, a local run, and a CI workflow, see the E2E Testing guide.

Suite Layout

Keep the suite in its own directory with its own package.json, so Norrix can install it without your app’s dependencies:

e2e/
├── package.json        # e2e, @e2e-dev/mobile, and your model provider (e.g. @ai-sdk/anthropic)
├── package-lock.json   # or pnpm-lock.yaml, yarn.lock, bun.lock
├── e2e.config.ts
└── tests/
    └── login.e2e.ts

norrix test uploads that directory without node_modules, .env files, platforms/, build artifacts, or run output under .e2e/ (the replay cache in .e2e/cache/ is kept).

import type { E2EConfig } from 'e2e';
import { mobile } from '@e2e-dev/mobile';
import { anthropic } from '@ai-sdk/anthropic';
 
export default {
  agents: {
    default: { model: anthropic('claude-sonnet-5-5') },
  },
  targets: [
    {
      name: 'ios',
      engine: mobile({ platform: 'ios' }),
      app: {
        bundleId: 'org.example.myapp',
        appPath: '../platforms/ios/build/Debug-iphonesimulator/myapp.app',
      },
    },
    {
      name: 'android',
      engine: mobile({ platform: 'android' }),
      app: { bundleId: 'org.example.myapp' },
    },
  ],
  workers: 1,
} satisfies E2EConfig;

On Norrix, every target on the build’s platform runs unless you pass --target. Norrix installs the build before the run and points those targets’ app.appPath at it, so device.installApp() in your fixtures installs the Norrix build. Leave device out of mobile() to use the simulator Norrix booted (pick it with --device-model/--device-os); a pinned device must exist on the runner.

Model Credentials and Secrets

Agent steps call the model your config names, with your own key. Store the key as a Norrix environment variable; tests without agent steps need none:

norrix env set ANTHROPIC_API_KEY "$ANTHROPIC_API_KEY"

Every Norrix environment variable reaches the suite’s processes, so E2E_SECRET_<NAME> and E2E_USER_<NAME>_USERNAME/_PASSWORD fill the config’s secrets and credentials, which the runner keeps out of model input. Norrix’s own runner credentials are never passed to the suite.

Replay Cache

Passing agent.act steps are recorded and replay later without a model call. Norrix keeps a replay cache per organization, project, and platform between cloud runs, so repeat runs get cheaper. Entries committed in your suite’s .e2e/cache/ fill gaps in it. agent.assert, agent.extract, and agent.waitFor always call the model.

Results

Each run uploads report.json, junit.xml, the markdown report, screenshots, and a video of every failed attempt. In the dashboard, View Report shows the report and failure videos. The run’s message names failed tests, and a run that never reaches its tests says why (for example a missing model key or an unknown target).

norrix test-status

Check the status of an E2E test run.

Synopsis

norrix test-status <e2eTestId> [options]

Arguments

Options

Example

norrix test-status e2e-test-1751234567890

Status Values

Dashboard

Enterprise organizations can also start and monitor E2E tests from the Norrix dashboard at /test. The dashboard accepts uploaded Maestro flow zips or e2e suite zips, shows recent runs with their results, supports cancellation, and links to streamed logs and, for e2e runs, the report and failure videos.