Skip to content

Conversation

@dkotter
Copy link
Collaborator

@dkotter dkotter commented Dec 23, 2025

What?

Closes #80

Adds in the initial scaffolding to run E2E tests via Playwright and adds our initial tests

Why?

We currently have no E2E tests and it would be ideal to get those in place so any Experiments we add in the future can have proper test coverage.

How?

  • Sets up the initial test configuration using Playwright, following what is done within the Gutenberg repo
  • Runs these tests as part of our test GitHub Action workflow
  • Adds initial tests for the following:
    • Plugin can be deactivated properly
    • Plugin can be activated properly
    • Both the AI Credentials and AI Experiments settings pages are accessible
    • Can properly add credentials on the AI Credentials setting page
    • Can see proper error messages on the AI Experiments page, both when no credentials have been added and when invalid credentials are added
    • Can globally enable Experiments
    • Ensures Title Generation works as expected
  • Adds a request mocking plugin and installs that when the E2E environment is spun up. Right now this plugin will make the OpenAI Models response (other responses to come)

Testing Instructions

Ensure the E2E tests run and pass on this PR

Open WordPress Playground Preview

@dkotter dkotter self-assigned this Dec 23, 2025
@codecov
Copy link

codecov bot commented Dec 23, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.89%. Comparing base (7dbbc44) to head (6478545).
⚠️ Report is 31 commits behind head on develop.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #162   +/-   ##
==========================================
  Coverage      46.89%   46.89%           
  Complexity       208      208           
==========================================
  Files             19       19           
  Lines           1271     1271           
==========================================
  Hits             596      596           
  Misses           675      675           
Flag Coverage Δ
unit 46.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

… tests. Add a mock request for the models endpoint for OpenAI
…where an Experiment is enabled individually but Experiments are globally disabled that the global setting takes precedence
@dkotter dkotter marked this pull request as ready for review December 23, 2025 20:36
@github-actions
Copy link

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <[email protected]>
Co-authored-by: jeffpaul <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@jeffpaul jeffpaul added this to the 0.2.0 milestone Dec 23, 2025
@jeffpaul jeffpaul merged commit 0ac2d0a into WordPress:develop Dec 23, 2025
40 of 42 checks passed
@dkotter dkotter deleted the feature/e2e-tests branch December 23, 2025 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add e2e tests (likely via Playwright)

2 participants