[5.x] Updated the Stache Store to not remove stache items that exist

[simonworkhouse]

Fixes #11862

[stanbridge-wcorrea]

I am sorry guys, but this needs to be reverted ASAP, this change is simply not the solution and I will share with you why:

I am currently facing a timeout when I do this: $formData = $event->submission->data()->all();

I have currently 20k form submissions file in my production website which is not that much.

Statamic Performance Issue Analysis: Form Submission Timeout

Executive Summary

A critical performance regression was identified in Statamic CMS that causes timeouts when processing form submissions with large datasets (20,000+ submissions). The issue was introduced in Statamic 5.64.0 and affects all subsequent versions.

Problem Description

• Issue: $event->submission->data()->all() times out during form submission processing
• Affected Versions: Statamic 5.64.0 and later (including 5.65.2)
• Impact: 20,000x performance degradation with large form submission datasets
• Root Cause: O(n²) complexity introduced in Stache file change detection

Technical Analysis

The Performance Regression

Location: /vendor/statamic/cms/src/Stache/Stores/Store.php line 211

Before (Statamic 5.50.0 - 5.63.x):

$deleted = $existing->keys()->diff($files->keys())->values();

After (Statamic 5.64.0+):

$deleted = $existing->keys()->map(fn ($path) => $this->getKeyFromPath($path))->diff($files->keys()->map(fn ($path) => $this->getKeyFromPath($path)))->values();

Performance Impact Analysis

With 20,145 form submissions:

Version	Complexity	Operations	Performance
5.50.0 - 5.63.x	O(n)	~20,145	Fast
5.64.0+	O(n²)	~405,821,025	20,000x slower

The getKeyFromPath() Method

protected function getKeyFromPath($path)
{
    return $this->paths()->flip()->get($path);
}

This method:

1. Loads ALL file paths into memory
2. Flips the entire collection (20,000+ operations)
3. Searches through the flipped collection

Called twice in the new implementation, causing massive performance overhead.

Version Timeline

Version	Date	Status	Notes
5.50.0	March 10, 2025	✅ Working	No performance issues
5.51.0 - 5.63.x	March - August 2025	✅ Working	Should work fine
5.64.0	August 21, 2025	❌ Regression	Issue #11863 introduced
5.65.0 - 5.65.2	August - September 2025	❌ Affected	Contains the regression

Root Cause: Issue #11863

• Issue: "Updated the Stache Store to not remove stache items that exist"
• Author: @simonworkhouse
• Intention: Fix edge case where existing Stache items were accidentally removed
• Unintended Consequence: Introduced severe performance regression

When the Issue Triggers

The handleFileChanges() method is called:

1. Every time a form submission is saved (triggers SubmissionSaved event)
2. During Stache cache updates
3. When Statamic detects file system changes

Immediate Solutions

Option 1: Revert the Change (Recommended)

Edit /vendor/statamic/cms/src/Stache/Stores/Store.php line 211:

// Change this:
$deleted = $existing->keys()->map(fn ($path) => $this->getKeyFromPath($path))->diff($files->keys()->map(fn ($path) => $this->getKeyFromPath($path)))->values();

// Back to this:
$deleted = $existing->keys()->diff($files->keys())->values();

Option 2: Downgrade Statamic

composer require statamic/cms:5.63.0

Option 3: Disable Stache Watcher

Add to .env:

STATAMIC_STACHE_WATCHER=false

Note: This disables file change detection and may require manual cache clearing.

Conclusion

The performance regression in Statamic 5.64.0+ makes the CMS unusable with large form submission datasets. The issue is a bug in Statamic's core and should be fixed by reverting to version 5.63.x or applying the manual fix until Statamic releases an official patch.

---

[jasonvarga]

You should absolutely have the stache watcher disabled on production though.

[simonworkhouse]

@stanbridge-wcorrea Would you mind reviewing this performance regression too? #11116