[6.x] Pest time-balanced sharding - #15140
Merged
Merged
Conversation
The provider keyed data sets by URL, and an empty string is one of the internal URLs, producing an empty-string key. PHPUnit rejects this when listing tests, which broke any tooling relying on --list-tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
setThemeColors() writes resource_path('preferences.yaml'), and while
setUp() deleted it, nothing cleaned up after the last test. The final
test's colors survived into unrelated tests that assert on preferences,
making the suite order dependent. Delete the file in tearDown() too,
matching DefaultPreferencesTest and PrecedenceTest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pest runs the existing PHPUnit test suite as-is, with no conversion to Pest syntax. It's added for its sharding support, which PHPUnit has no equivalent for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test run is around 90% of each job's wall clock, so sharding is close to linear. Four shards takes the workflow from ~12.8 to ~4 minutes. Windows needs its own include entry per shard, since an include that overrides a matrix key creates a standalone combination rather than merging, and so wouldn't inherit the shard dimension. shards.json holds recorded timings so shards are balanced by duration rather than test count. Regenerate it with --update-shards. Adds a php-tests-result job that aggregates every shard, so branch protection can require one check instead of one per matrix cell. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Windows tests take roughly twice as long as ubuntu's, so splitting both of them four ways leaves Windows as the critical path. Six brings its slowest job down to about the same as the slowest ubuntu one, which is as far as it's worth going.
The increment and rotate tags keep their counters in static properties that are only cleared when the ClearState listener fires on a response. A test that renders Antlers without going through a full request leaves them populated, and the next test in the same process inherits them. StateTest picked up a counter of 31 from IncrementTest once sharding put the two in the same job.
jasonvarga
marked this pull request as ready for review
August 10, 2026 14:51
duncanmcclean
approved these changes
Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Pest as the CI test runner so we can shard the PHP suite across parallel jobs, using its time-balanced sharding to keep them even.
The suite is ~10,000 tests and the slowest job (Windows) had grown to nearly 13 minutes, which sets the floor for how long a PR sits waiting.
Tests themselves (after the overhead of setting up the jobs) goes from ~11m to ~3m.
A pull request workflow run goes from ~14m to ~4m.
Pest 4 is used because we still support Laravel 12. Luckily, the time-balanced sharding feature actually landed on Pest 4 - not Pest 5 like their marketing implies.
Pest is only the runner
Intentionally not rewriting any tests. Pest is just being used as the runner and for its sharding feature.
You can use
vendor/bin/phpunitorvendor/bin/pestlocally.Ubuntu is split 4 ways, Windows 6
Windows runs the same tests in roughly twice the time, so splitting both evenly leaves Windows as the critical path with Ubuntu jobs sitting idle. Six shards brings Windows' test execution down to roughly the same as Ubuntu's, for two extra jobs.
Test fixes
Three pre-existing bugs surfaced while getting this green. They're follow-ups to #15143 — same class of problem, just not caught at the time — and are included here because sharding can't work without them:
ProvidesExternalUrls::externalUrlProvider()keyed its data sets by URL, and''is one of the internal URLs — an empty-string key, which PHPUnit rejects. This broke--list-testsoutright, which is how shard assignment is discovered.CP\ColorTestleaked state. It wrote theme colour preferences toresources/preferences.yamland never cleaned up, soPreferences\EndpointsTestinherited a straytheme.colorsblock. Now cleaned up intearDown(), matchingDefaultPreferencesTestandPrecedenceTest.incrementandrotatetags keep counters in static properties that are only cleared when theClearStatelistener fires on a response. A test that renders Antlers without a full request leaves them populated.StateTestpicked up a counter of 31 fromIncrementTestonce sharding put them in the same job.TestCase::setUp()now resets view state.Notes
--parallelis deliberately not used. The suite writes into the testbench skeleton, and concurrent processes corrupt it.