Read bundled CLI dependencies directly from wp-files#3125

bcotrim · 2026-04-17T11:18:43Z

Related issues

STU-1537

How AI was used in this PR

Claude Code explored the setupServerFiles() call graph, identified which bundled deps safely work read-only from the CLI bundle vs. those that need a writable cache, then applied the refactor incrementally — one dep per commit, each validated on CI before moving on. A previous attempt had tried moving multiple deps at once and hit a wp-admin 500 failure; this time each dep was isolated and the real culprit (language packs) got narrowed down cleanly.

Proposed Changes

setupServerFiles() used to run on every CLI invocation and copy every bundled dependency from the read-only wp-files/ (shipped with the CLI) to the writable ~/.studio/server-files/. Almost none of those copies were needed — only WordPress is genuinely updated at runtime. The copy logic added ~60ms of overhead per CLI invocation.

This PR has consumers read read-only deps directly from bundled wp-files/ and drops the corresponding copy steps.

Kept writable (still copied into ~/.studio/server-files/):

WordPress latest (wordpress-versions/latest) — genuinely updated at runtime via updateLatestWordPressVersion() / downloadWordPress().
Language packs (language-packs/) — see Deferred below.

Now read directly from wp-files/ (no copy):

AI instructions / skills (skills/) — CLI and desktop both repointed
available-site-translations.json
SQLite plugin (sqlite-database-integration/)
WP-CLI phar (wp-cli.phar) — mounted into PHP-wasm VFS
SQLite command (sqlite-command/) — mounted into PHP-wasm VFS
phpMyAdmin (phpmyadmin/) — mounted into PHP-wasm VFS

Consumer changes:

CLI: server-files.ts, site-language.ts, sqlite-integration.ts, dependency-management/setup.ts
Desktop: lib/server-files-paths.ts (AI instructions repointed, new getBundledSqlitePluginPath export), lib/sqlite-versions.ts
Shared: tools/common/lib/sqlite-integration.ts — replaced the misleadingly-named getServerFilesPath() abstract method with a getSqlitePluginSourcePath() override, so subclasses can point anywhere.

Cleanup:

New migration 04-cleanup-obsolete-server-files deletes the now-unused entries from ~/.studio/server-files/ on first CLI run after upgrade: skills/, sqlite-database-integration/, sqlite-command/, phpmyadmin/, wp-cli.phar, and wordpress-versions/latest/available-site-translations.json.
Deleted dead code: apps/studio/src/lib/sqlite-command-versions.ts (nothing imports it).
Removed unused copySourceDirectoryIfNewerOrMissing helper from setup.ts after all its callers were dropped.

Deferred

Language packs were initially in scope but consistently failed E2E (localization.test.ts) in CI when moved to bundle-read — 2 failures, 2 passes on the equivalent commit-1-only state, so the failure is specific to reading language packs from the bundled location. Working theory: the readdir over the ~1,300-entry bundled latest/languages/ dir inside the packaged .app misbehaves in CI under a code path we haven't traced yet. Reverted to preserve the other wins; will revisit in a follow-up once we can grab a Playwright trace from a failing run.

Testing Instructions

Automated

npm run typecheck passes
npx eslint --fix on modified files — clean
npm test — all tests pass
E2E passing on CI for each incremental commit

Manual

Fresh install — delete ~/.studio/server-files/ → run CLI (studio site list) → verify only wordpress-versions/ and language-packs/ get created. No more skills/, sqlite-database-integration/, sqlite-command/, phpmyadmin/, wp-cli.phar, or available-site-translations.json.
Upgrade path — install a pre-refactor build, let it populate ~/.studio/server-files/ with all the old entries, then upgrade to this build. On first CLI run the migration should delete the six obsolete entries; wordpress-versions/ and language-packs/ are untouched.
Site creation — create a new site → verifies SQLite plugin install + SQLite command mount + WP-CLI mount all still work (now read from bundle). Covered by e2e blueprints.test.ts and import.test.ts.
phpMyAdmin — open a running site's phpMyAdmin shortcut → UI loads (now mounted from bundle). Covered by e2e overview-customize-links.test.ts.
AI skills — install skills to a site from the desktop UI; edit STUDIO.md in a site and restart the server — should be refreshed from bundled wp-files/skills/. (No direct E2E coverage — manual smoke.)
Localization — create a Japanese site → verify WPLANG=ja in wp-admin (language packs still go through the copy step, so behaviour should be identical to trunk). Covered by e2e localization.test.ts.
Benchmark — compare time node apps/cli/dist/cli/main.mjs site list before vs after on trunk; expect a measurable drop from dropping six copy steps at startup.

Pre-merge Checklist

Have you checked for TypeScript, React or other console errors?

wpmobilebot · 2026-04-17T11:38:59Z

📊 Performance Test Results

Comparing e6ed07e vs trunk

app-size

Metric	trunk	`e6ed07e`	Diff	Change
App Size (Mac)	1491.74 MB	1491.73 MB	0.01 MB	⚪ 0.0%

site-editor

Metric	trunk	`e6ed07e`	Diff	Change
load	1834 ms	1881 ms	+47 ms	⚪ 0.0%

site-startup

Metric	trunk	`e6ed07e`	Diff	Change
siteCreation	10114 ms	8108 ms	2006 ms	🟢 -19.8%
siteStartup	5928 ms	4952 ms	976 ms	🟢 -16.5%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

fredrikekelund · 2026-04-17T12:25:48Z

Given the changes we made in #3086, could we not read WP-CLI and sqlite-command directly from the wp-files directory, too?

fredrikekelund · 2026-04-17T14:47:38Z

WDYT about adding a migration to clean up the paths in server-files we no longer use?

fredrikekelund · 2026-04-20T09:12:04Z

@bcotrim, did you close this PR intentionally?

fredrikekelund · 2026-04-21T08:20:57Z

Can we remove apps/studio/src/lib/sqlite-command-versions.ts and apps/studio/src/lib/sqlite-command-release.ts in this PR? That's dead code now. It's tangentially related to this change.

…guages/" This reverts commit 79da74f.

…egration/

…te-dependency-handling # Conflicts: # apps/cli/lib/dependency-management/setup.ts

fredrikekelund

Nice and clean change 👍 Tests well. I left a few minor comments

fredrikekelund · 2026-04-22T09:28:43Z

 	}
+
+	protected getSqlitePluginSourcePath(): string {
+		return path.join( 'server-files', SQLITE_DIRNAME );


Suggested change

return path.join( 'server-files', SQLITE_DIRNAME );

return path.join( 'wp-files', SQLITE_DIRNAME );

Might as well swap this to not mislead anyone.

fredrikekelund · 2026-04-22T09:29:56Z

Noting that #3129 removes this file. We shouldn't make any changes here now, but we'll need to reconcile the changes between the PRs, depending on which one we land first.

fredrikekelund · 2026-04-22T09:30:35Z

Let's also remove apps/studio/src/lib/sqlite-command-release.ts

fredrikekelund · 2026-04-22T09:33:45Z

I advise renaming this file. We can call it something like apps/cli/lib/dependency-management/paths.ts

…file

…/paths.ts

## Related issues - Closes STU-1537 (completes the part deferred from #3125) ## How AI was used in this PR Claude Code reconstructed the April context (#3125's deferred section, this PR's original debug instrumentation, and the language-pack copy races later documented in #3492), re-applied the bundle-read change cleanly on top of current trunk. ## Proposed Changes Language packs were the last site dependency still copied from the read-only CLI bundle into the writable `~/.studio/server-files/` cache on every CLI invocation. The copy was pure overhead: language packs are only ever read at site creation, to copy the matching locale's files into the new site's `wp-content/languages/`, and they only change when the app ships a new bundle. This PR has site creation read language packs directly from the bundled `wp-files/latest/languages/`, and drops the per-invocation copy step — shaving the remaining recursive dir-walk (~1,400 files) off CLI startup. Side benefits: - The copy step needed lockfile serialization (#3492) because concurrent CLI invocations raced on the shared cache (transient `ENOENT` warnings). With no copy and no writable cache, that bug class is gone by construction. - Site creation now skips the language-pack directory scans entirely for locales we don't bundle translations for (`WP_LOCALES`). - The existing `04-cleanup-obsolete-server-files` migration also removes the now-unused `~/.studio/server-files/language-packs/` (and its lockfile) on first run after upgrade. **On the April E2E failures that deferred this:** they never reproduced locally — neither at the CLI level (3/3 ja-locale sites healthy, WPLANG=ja, wp-admin 200) nor with the packaged-app Playwright suites that failed in CI back then. The Playground worker-spawn stack has been rewritten since (#3602), and the failure was never root-caused, so CI on this PR is the deciding signal. If mac E2E reds again, this time it fails on a clean branch with inspectable artifacts. ## Testing Instructions 1. `npm run download-language-packs` (plain `npm install` does not populate `wp-files/latest/languages`), then `npm run cli:build`. 2. Set a non-English locale (e.g. `"locale": "ja"` in `~/.studio/shared.json` or via Studio Settings) and create a site. 3. Verify the site's `wp-content/languages/` contains the locale's files (core + `plugins/` + `themes/`), wp-admin loads in that language, and Settings → General shows the locale selected under Site Language. 4. Verify `~/.studio/server-files/language-packs/` is deleted on first CLI run after upgrade and is not recreated. 5. Covered by e2e `localization.test.ts` (locale site creation + WPLANG) and `blueprints.test.ts` (blueprint provisioning). Verified locally on macOS arm64: typecheck, eslint, full unit suite (1963 tests), packaged-app e2e `localization.test.ts` 4/4 (twice) and `blueprints.test.ts` 6/6. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?

bcotrim self-assigned this Apr 17, 2026

bcotrim requested review from a team and fredrikekelund April 17, 2026 13:35

bcotrim force-pushed the stu-1537-refactor-site-dependency-handling branch 3 times, most recently from 9ad5546 to 7334eb1 Compare April 20, 2026 07:02

bcotrim closed this Apr 20, 2026

bcotrim force-pushed the stu-1537-refactor-site-dependency-handling branch from 9849ca2 to ce3716f Compare April 20, 2026 09:05

bcotrim reopened this Apr 20, 2026

bcotrim changed the title ~~Read bundled CLI dependencies directly from wp-files~~ [WIP] Read bundled CLI dependencies directly from wp-files Apr 21, 2026

bcotrim closed this Apr 21, 2026

bcotrim force-pushed the stu-1537-refactor-site-dependency-handling branch from 9315a88 to 77648f0 Compare April 21, 2026 13:56

bcotrim reopened this Apr 21, 2026

Read AI instructions directly from bundled wp-files/skills/

a9a9f94

bcotrim force-pushed the stu-1537-refactor-site-dependency-handling branch from 016f988 to a9a9f94 Compare April 21, 2026 14:56

Clarify AI instructions comment (no-op to re-run CI)

406cbbb

bcotrim force-pushed the stu-1537-refactor-site-dependency-handling branch from 23e990c to 406cbbb Compare April 21, 2026 16:36

bcotrim added 6 commits April 21, 2026 18:17

Read language packs directly from bundled wp-files/latest/languages/

79da74f

Revert "Read language packs directly from bundled wp-files/latest/lan…

5146e27

…guages/" This reverts commit 79da74f.

Read available-site-translations.json directly from bundle

51617f1

Read SQLite plugin directly from bundled wp-files/sqlite-database-int…

c2c8b4a

…egration/

Add migration to clean up obsolete server-files entries

5819382

Remove dead sqlite-command-versions module

accadf0

bcotrim changed the title ~~[WIP] Read bundled CLI dependencies directly from wp-files~~ Read bundled CLI dependencies directly from wp-files Apr 21, 2026

Read WP-CLI phar directly from bundled wp-files/wp-cli/wp-cli.phar

b137f22

bcotrim added 5 commits April 21, 2026 22:05

Drop cleanup-migration test (one-off migration, not worth maintaining)

b7f5ae0

Read SQLite command directly from bundled wp-files/sqlite-command/

a03c7f7

Read phpMyAdmin directly from bundled wp-files/phpmyadmin/

262471f

Remove unused copySourceDirectoryIfNewerOrMissing helper

2d46915

Merge remote-tracking branch 'origin/trunk' into stu-1537-refactor-si…

9fe6b14

…te-dependency-handling # Conflicts: # apps/cli/lib/dependency-management/setup.ts

fredrikekelund approved these changes Apr 22, 2026

View reviewed changes

bcotrim added 4 commits April 22, 2026 11:11

Use wp-files path in SqliteIntegrationProvider test fixture

1da3b76

Inline SQLite command release helper into build script and drop dead …

be46cba

…file

Split cli/lib/server-files.ts into appdata.ts + dependency-management…

8275b4c

…/paths.ts

Fix import-x/order in create.ts after paths module rename

e6ed07e

bcotrim merged commit 609b282 into trunk Apr 22, 2026
10 checks passed

bcotrim deleted the stu-1537-refactor-site-dependency-handling branch April 22, 2026 11:39

bcotrim mentioned this pull request Jun 12, 2026

Read bundled language packs directly from wp-files #3214

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Read bundled CLI dependencies directly from wp-files#3125