Skip to content

fix: install pnpm deps in hoisted mode + declare @babel/types (#24288) (CP:23.7)#24322

Merged
ZheSun88 merged 1 commit into
23.7from
pick-24288-23.7
May 12, 2026
Merged

fix: install pnpm deps in hoisted mode + declare @babel/types (#24288) (CP:23.7)#24322
ZheSun88 merged 1 commit into
23.7from
pick-24288-23.7

Conversation

@ZheSun88
Copy link
Copy Markdown
Contributor

…) (CP: 23.7)

Summary

The Vite dev server intermittently fails (or hangs) on vite-basics and other Vite-using IT modules because transitive npm dependencies are not consistently reachable from the project's root node_modules. Selenium tests then wait for pages that never render, and the matrix shard times out at the 30-minute job cap.

Two distinct symptoms observed:

  1. Cannot find package '@babel/types' — the bundled Vite plugin react-function-location-plugin.js does import * as t from '@babel/types';, but @babel/types was only present transitively via @babel/core and pnpm did not always hoist it to the root.

  2. Failed to resolve import "@lit/reactive-element/...", "cookie", "set-cookie-parser", "@preact/signals-react/runtime" — all transitives of declared deps (lit, react-router, @preact/signals-react-transform) — also not consistently hoisted.

Both are the same underlying problem: pnpm with shamefully-hoist=true does not deterministically expose every transitive at the project root. On 23.7 the same root cause surfaces as Cannot find module 'rollup' when Vite 3.2.11 tries to load vite.generated.js.

Fix

1. Switch the bundled .npmrc template to node-linker=hoisted

Adds node-linker=hoisted to flow-server/src/main/resources/npmrc, the template TaskRunNpmInstall writes to the project. This tells pnpm to install in flat npm-style mode — every package (declared or transitive) at the project root, no .pnpm/ virtual store. Resolutions become deterministic.

shamefully-hoist=true is left in the template alongside, so manual pnpm install invocations from a developer shell still pick up at least the legacy partial-hoist behavior if some user-level config overrides our node-linker.

2. Pass --config.node-linker=hoisted on the pnpm CLI

The .npmrc change alone did not actually flip pnpm's install layout in CI: node_modules/.pnpm/... paths kept appearing in error messages, indicating pnpm was still using its default isolated mode. The pre-existing --shamefully-hoist=true CLI flag added by FrontendTools.getPnpmExecutable() likely took precedence and locked pnpm to the isolated layout (where shamefully-hoist makes sense as a partial-hoist heuristic).

Replace --shamefully-hoist=true with --config.node-linker=hoisted on the CLI. CLI > .npmrc > defaults in pnpm's config precedence, so the install is now unambiguously hoisted regardless of any project- or user-level .npmrc content. FrontendToolsTest updated to assert the new flag.

Backport notes (23.7)

  • Source paths remapped from flow-build-tools/ (post-refactor on newer branches) back to flow-server/ where these classes still live on 23.7.
  • The @babel/types dependency addition and the matching NodeUpdaterTest change from the original commit were omitted: 23.7 uses Vite 3.2.11 and does not ship the React function-location plugin that consumes @babel/types, so declaring it would be dead weight. The load-bearing fixes (pnpm CLI flag + npmrc) are sufficient to resolve the Cannot find module 'rollup' failure observed in the 23.7 nightly build.

Trade-offs

  • Disk/install time: hoisted mode uses slightly more disk and is a few seconds slower than the symlink store for fresh installs. Acceptable for the determinism gain.
  • Loose isolation: hoisted mode allows packages to import any other package, declared or not. Flow's frontend code already relies on this (e.g. plugins importing transitives), so this is the current de-facto behavior — just made deterministic.
  • Behavioral change for downstream users: anyone with a custom .npmrc that lacks the auto-gen marker keeps their config (per existing TaskRunNpmInstall.createNpmRcFile() logic), but the new CLI flag still applies — they'll silently get hoisted mode for Flow-driven installs. Worth flagging in release notes.

(cherry picked from commit d9290f8)

@ZheSun88 ZheSun88 changed the title fix: install pnpm deps in hoisted mode + declare @babel/types (#24288… fix: install pnpm deps in hoisted mode + declare @babel/types (#24288) (CP:23.7) May 12, 2026
@ZheSun88 ZheSun88 marked this pull request as draft May 12, 2026 07:02
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 12, 2026

Test Results

  985 files  ±0    985 suites  ±0   45m 30s ⏱️ + 2m 18s
6 343 tests ±0  6 293 ✅ ±0  50 💤 ±0  0 ❌ ±0 
6 659 runs  +3  6 601 ✅ +3  58 💤 ±0  0 ❌ ±0 

Results for commit 1f78311. ± Comparison against base commit c91d68d.

♻️ This comment has been updated with latest results.

… (CP: 23.7)

## Summary

The Vite dev server intermittently fails (or hangs) on `vite-basics` and
other Vite-using IT modules because transitive npm dependencies are not
consistently reachable from the project's root `node_modules`. Selenium
tests then wait for pages that never render, and the matrix shard times
out at the 30-minute job cap.

Two distinct symptoms observed:

1. **`Cannot find package '@babel/types'`** — the bundled Vite plugin
   `react-function-location-plugin.js` does
   `import * as t from '@babel/types';`, but `@babel/types` was only
   present transitively via `@babel/core` and pnpm did not always hoist
   it to the root.

2. **`Failed to resolve import "@lit/reactive-element/..."`,
   `"cookie"`, `"set-cookie-parser"`, `"@preact/signals-react/runtime"`** —
   all transitives of declared deps (`lit`, `react-router`,
   `@preact/signals-react-transform`) — also not consistently hoisted.

Both are the same underlying problem: pnpm with `shamefully-hoist=true`
does not deterministically expose every transitive at the project root.
On 23.7 the same root cause surfaces as `Cannot find module 'rollup'`
when Vite 3.2.11 tries to load `vite.generated.js`.

## Fix

### 1. Switch the bundled `.npmrc` template to `node-linker=hoisted`

Adds `node-linker=hoisted` to `flow-server/src/main/resources/npmrc`,
the template `TaskRunNpmInstall` writes to the project. This tells
pnpm to install in **flat npm-style mode** — every package (declared
or transitive) at the project root, no `.pnpm/` virtual store.
Resolutions become deterministic.

`shamefully-hoist=true` is left in the template alongside, so manual
`pnpm install` invocations from a developer shell still pick up at
least the legacy partial-hoist behavior if some user-level config
overrides our `node-linker`.

### 2. Pass `--config.node-linker=hoisted` on the pnpm CLI

The `.npmrc` change alone did not actually flip pnpm's install layout
in CI: `node_modules/.pnpm/...` paths kept appearing in error
messages, indicating pnpm was still using its default `isolated`
mode. The pre-existing `--shamefully-hoist=true` CLI flag added by
`FrontendTools.getPnpmExecutable()` likely took precedence and locked
pnpm to the isolated layout (where `shamefully-hoist` makes sense as
a partial-hoist heuristic).

Replace `--shamefully-hoist=true` with `--config.node-linker=hoisted`
on the CLI. CLI > `.npmrc` > defaults in pnpm's config precedence, so
the install is now unambiguously hoisted regardless of any project-
or user-level `.npmrc` content. `FrontendToolsTest` updated to assert
the new flag.

### 3. Update `PnpmUsedIT` to check `pnpm-lock.yaml` instead of the
       legacy symlink layout

`flow-tests/test-frontend/test-pnpm/.../PnpmUsedIT.pnpmIsUsed` used
to assert that `node_modules/lit` was a symlink — the pnpm
isolated-mode signature. With hoisted mode that symlink no longer
exists, so the assertion fails (this was caught by the first PR run).
Replace it with a `pnpm-lock.yaml` existence check, which is the
remaining reliable signal that pnpm (not npm) ran the install. This
matches the equivalent change already on main (commit 1dd7ab3).

## Backport notes (23.7)

* Source paths remapped from `flow-build-tools/` (post-refactor on
  newer branches) back to `flow-server/` where these classes still
  live on 23.7.
* The `@babel/types` dependency addition and the matching
  `NodeUpdaterTest` change from the original commit were **omitted**:
  23.7 uses Vite 3.2.11 and does not ship the React function-location
  plugin that consumes `@babel/types`, so declaring it would be dead
  weight.
* `PnpmUsedIT` is also updated here (inlined `"pnpm-lock.yaml"` rather
  than `Constants.PACKAGE_LOCK_YAML`, which does not exist on 23.7).

## Trade-offs

- **Disk/install time**: hoisted mode uses slightly more disk and is
  a few seconds slower than the symlink store for fresh installs.
  Acceptable for the determinism gain.
- **Loose isolation**: hoisted mode allows packages to import any
  other package, declared or not. Flow's frontend code already relies
  on this (e.g. plugins importing transitives), so this is the
  current de-facto behavior — just made deterministic.
- **Behavioral change for downstream users**: anyone with a custom
  `.npmrc` that lacks the auto-gen marker keeps their config (per
  existing `TaskRunNpmInstall.createNpmRcFile()` logic), but the new
  CLI flag still applies — they'll silently get hoisted mode for
  Flow-driven installs. Worth flagging in release notes.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit d9290f8)
@sonarqubecloud
Copy link
Copy Markdown

@ZheSun88 ZheSun88 marked this pull request as ready for review May 12, 2026 10:44
@ZheSun88 ZheSun88 merged commit b53d9ad into 23.7 May 12, 2026
27 checks passed
@ZheSun88 ZheSun88 deleted the pick-24288-23.7 branch May 12, 2026 10:47
vaadin-bot added a commit that referenced this pull request May 12, 2026
… (CP:23.7) (#24322) (CP: 23.6) (#24323)

This PR cherry-picks changes from the original PR #24322 to branch 23.6.
---
#### Original PR description
> …) (CP: 23.7)
> 
> ## Summary
> 
> The Vite dev server intermittently fails (or hangs) on `vite-basics`
and other Vite-using IT modules because transitive npm dependencies are
not consistently reachable from the project's root `node_modules`.
Selenium tests then wait for pages that never render, and the matrix
shard times out at the 30-minute job cap.
> 
> Two distinct symptoms observed:
> 
> 1. **`Cannot find package '@babel/types'`** — the bundled Vite plugin
`react-function-location-plugin.js` does `import * as t from
'@babel/types';`, but `@babel/types` was only present transitively via
`@babel/core` and pnpm did not always hoist it to the root.
> 
> 2. **`Failed to resolve import "@lit/reactive-element/..."`,
`"cookie"`, `"set-cookie-parser"`, `"@preact/signals-react/runtime"`** —
all transitives of declared deps (`lit`, `react-router`,
`@preact/signals-react-transform`) — also not consistently hoisted.
> 
> Both are the same underlying problem: pnpm with
`shamefully-hoist=true` does not deterministically expose every
transitive at the project root. On 23.7 the same root cause surfaces as
`Cannot find module 'rollup'` when Vite 3.2.11 tries to load
`vite.generated.js`.
> 
> ## Fix
> 
> ### 1. Switch the bundled `.npmrc` template to `node-linker=hoisted`
> 
> Adds `node-linker=hoisted` to `flow-server/src/main/resources/npmrc`,
the template `TaskRunNpmInstall` writes to the project. This tells pnpm
to install in **flat npm-style mode** — every package (declared or
transitive) at the project root, no `.pnpm/` virtual store. Resolutions
become deterministic.
> 
> `shamefully-hoist=true` is left in the template alongside, so manual
`pnpm install` invocations from a developer shell still pick up at least
the legacy partial-hoist behavior if some user-level config overrides
our `node-linker`.
> 
> ### 2. Pass `--config.node-linker=hoisted` on the pnpm CLI
> 
> The `.npmrc` change alone did not actually flip pnpm's install layout
in CI: `node_modules/.pnpm/...` paths kept appearing in error messages,
indicating pnpm was still using its default `isolated` mode. The
pre-existing `--shamefully-hoist=true` CLI flag added by
`FrontendTools.getPnpmExecutable()` likely took precedence and locked
pnpm to the isolated layout (where `shamefully-hoist` makes sense as a
partial-hoist heuristic).
> 
> Replace `--shamefully-hoist=true` with `--config.node-linker=hoisted`
on the CLI. CLI > `.npmrc` > defaults in pnpm's config precedence, so
the install is now unambiguously hoisted regardless of any project- or
user-level `.npmrc` content. `FrontendToolsTest` updated to assert the
new flag.
> 
> ## Backport notes (23.7)
> 
> * Source paths remapped from `flow-build-tools/` (post-refactor on
newer branches) back to `flow-server/` where these classes still live on
23.7.
> * The `@babel/types` dependency addition and the matching
`NodeUpdaterTest` change from the original commit were **omitted**: 23.7
uses Vite 3.2.11 and does not ship the React function-location plugin
that consumes `@babel/types`, so declaring it would be dead weight. The
load-bearing fixes (pnpm CLI flag + npmrc) are sufficient to resolve the
`Cannot find module 'rollup'` failure observed in the 23.7 nightly
build.
> 
> ## Trade-offs
> 
> - **Disk/install time**: hoisted mode uses slightly more disk and is a
few seconds slower than the symlink store for fresh installs. Acceptable
for the determinism gain.
> - **Loose isolation**: hoisted mode allows packages to import any
other package, declared or not. Flow's frontend code already relies on
this (e.g. plugins importing transitives), so this is the current
de-facto behavior — just made deterministic.
> - **Behavioral change for downstream users**: anyone with a custom
`.npmrc` that lacks the auto-gen marker keeps their config (per existing
`TaskRunNpmInstall.createNpmRcFile()` logic), but the new CLI flag still
applies — they'll silently get hoisted mode for Flow-driven installs.
Worth flagging in release notes.
> 
> 
> (cherry picked from commit d9290f8)
>

Co-authored-by: Zhe Sun <31067185+ZheSun88@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants