Repository navigation
Parallel Stache warming produces inconsistent Stache _indexes #13261
Description
Bug description
After enabling the new parallel Stache warming (introduced in v5.70.0), the generated Stache _indexes become inconsistent between runs and differ from the output produced in the versions before the PR.
How to reproduce
- Enable parallel Stache warming (e.g. parallel_processing = true, concurrency_driver = fork) on 5.70.0.
- Run the Stache warm command and check
storage/framework/cache/data/stache/indexes/**/_indexes
v5.69.0:
Indexes are not changing when running the stache warm command multiple times.
9999999999a:1:{i:0;s:4:"path";}
v5.70.0:
Indexes are changing when running the stache warm command multiple times.
9999999999a:2:{i:0;s:4:"path";i:1;s:2:"id";}
Logs
Environment
5.70.0Installation
Fresh statamic/statamic site via CLI
Additional details
Also it seems parallel processing does not improve stache warm performance (I don't see any significant difference in all of my projects). Because most entries live in collections, and collections are warmed in a single process (one store per process), the parallelization provides little / no real concurrency. The smaller stores finish quickly, leaving most cores idle while the main collection store warms sequentially..
Activity
hastinbe commentedon Dec 5, 2025
Thanks for opening the issue and for documenting the behavior.
Since the performance characteristics were already discussed in Discord prior to this issue being opened, and since they're restated in the additional details here, I'll summarize that explanation again for completeness and for anyone reading this thread later.
The main source of improvement came from optimizing the file traversal itself. Previously, every sequential store->warm() call had to repeat the slower traversal. A greater distribution of entries across stores will naturally see greater improvement from that change. In our case, roughly 50% of ~17k entries were in a single collection, with the rest spread over 15 collections, which is why we saw a more significant speedup. The PR you referenced also includes the benchmarking script that was used to validate these results.
There were no unit tests validating the index shape previously, so this bug wasn't caught.
It may not have been fully clear earlier, but the cache stores are what get warmed concurrently -- not the individual entries within a store. If most entries live inside a single collection, they will still be processed sequentially in one worker. It sounds like your expectation was different, and it may be more accurate to describe this differently.
As also mentioned previously, loading entries concurrently at the file level was tested early on, but it introduces dependency issues that lead to incorrect results, which is the same class of issue we're seeing here. Taxonomies, terms, and collections must be loaded sequentially before other stores can be warmed in parallel to mitigate some of this, but even then the resulting data shape will differ from a fully sequential run.
At this point, I am not confident the sequential index shape can be replicated when stores are warmed concurrently. Once stores are warmed in parallel, dependency ordering is no longer guaranteed, which leads to inconsistent index output as seen here. I do not fully understand yet whether the observed index shape differences are problematic or simply different, and input from someone more familiar with the internal stache indexing model would be helpful here.
If this index shape difference is problematic, I can submit a PR that reverts the parallel warming behavior while keeping the traversal improvements intact, since that optimization is independent of this.
edalzell commentedon Apr 29, 2026
The symptom for us is that structured collections don't show up in the CP (nor the site) as structured...everything is flattened. The raw data (tree files) is correct but they resolve incorrectly (not nested).
This issue is preventing us from deploying our sites on v6, because w/o parallel warming it takes longer than 10 mins and the deploy times out.
We have a site that took <5 mins on v5 and now it takes over 10 so parallel warming is the only solution right now.
edalzell commentedon Apr 29, 2026
@hastinbe I'd love to give this a try if you have it around