Skip to content

test: retry Playwright tests in CI - #466

Merged
josephfusco merged 1 commit into
WordPress:mainfrom
muneeb-ashraf:fix/playwright-ci-retries
Sep 4, 2026
Merged

josephfusco merged 1 commit into
WordPress:mainfrom
muneeb-ashraf:fix/playwright-ci-retries

Conversation

@muneeb-ashraf

@muneeb-ashraf muneeb-ashraf commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Add two Playwright retries in CI while preserving zero retries for local runs. This allows transient wordpress.org timeouts to be retried without masking failures during local development.

Fixes #403

Testing

  • Verified both Playwright projects resolve to 0 retries locally and 2 retries in CI.
  • npx playwright test --config tests/e2e/playwright.config.js --list - 44 tests discovered.
  • npm run test:unit -- --runInBand - 21 tests passed.
  • npm run test:scripts - 79 tests passed.
  • git diff --check
Use of AI Tools

AI assistance: Yes
Tool(s): OpenAI Codex
Model(s): GPT-5
Used for: Code and tests.

@github-actions

github-actions Bot commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props muneebashraf, iamchitti, joefusco.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.85%. Comparing base (909c9b5) to head (0d230ec).
⚠️ Report is 58 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #466      +/-   ##
============================================
- Coverage     96.36%   95.85%   -0.52%     
- Complexity      240      255      +15     
============================================
  Files            19       21       +2     
  Lines          2835     3014     +179     
============================================
+ Hits           2732     2889     +157     
- Misses          103      125      +22     
Flag Coverage Δ
multisite 95.85% <ø> (-0.52%) ⬇️
phpunit 71.06% <ø> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@i-am-chitti i-am-chitti left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried it locally: 2 retries in CI, 0 on my machine, all 44 tests still found. Also checked the specs start fresh on a retry. So, no new flakiness.

#372 is already merged. So, no blocker here.

LGTM 🚀

Nit: @muneeb-ashraf Could you fill in the template's Use of AI Tools block (AI assistance: / Tool(s): / Model(s):)? The current PR description doesn't say which model.

@muneeb-ashraf

Copy link
Copy Markdown
Contributor Author

Updated the PR description with the complete disclosure block, including the tool, model, and usage. Thanks for the catch.

@josephfusco
josephfusco merged commit c9a62dd into WordPress:main Sep 4, 2026
22 checks passed
@josephfusco josephfusco added the [Area] Infrastructure Issues for the repository infrastructure label Sep 4, 2026
@josephfusco josephfusco added this to the 0.4.0 milestone Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Area] Infrastructure Issues for the repository infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e specs hit wordpress.org on every WP-CLI call, so one timeout fails the job

3 participants