[Bug?]: `yarn install` does not set executable bit on bin files when folder has been manually removed

[scottbessler]

Self-service

• I'd be willing to implement a fix

Describe the bug

Yarn, when using the node-modules linker, when adding a missing dependency, does not properly set the executable bit for the files defined in the bin directive.

To reproduce

1. add a package like protobufjs-cli to your package.json
2. run yarn install
3. note the file node_modules/protobufj-cli/pbjs is marked executable
4. remove the protobufjs-cli from node_modules (i.e. via rm -rf node_modules/protobufjs-cli)
5. run yarn install
6. note the file node_modules/protobufj-cli/pbjs is NOT marked executable

Environment

System:
    OS: macOS 13.2.1
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 18.15.0 - /private/var/folders/n1/3jwsrn0j1h17qckdb157pdjw0000gn/T/xfs-17b29353/node
    Yarn: 3.5.0 - /private/var/folders/n1/3jwsrn0j1h17qckdb157pdjw0000gn/T/xfs-17b29353/yarn
    npm: 9.5.0 - ~/.volta/tools/image/node/18.15.0/bin/npm

Additional context

➜ cat .yarnrc.yml
nodeLinker: node-modules

yarnPath: .yarn/releases/yarn-3.5.0.cjs

➜ yarn
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed
➤ YN0000: Done in 0s 93ms

➜ ll node_modules/protobufjs-cli/bin
total 16
-rwxr-xr-x  1 username  staff  203 Mar 24 15:21 pbjs
-rwxr-xr-x  1 username  staff  203 Mar 24 15:21 pbts

➜ rm -rf node_modules/protobufjs-cli 

➜ yarn
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed
➤ YN0000: Done in 0s 113ms

➜ ll node_modules/protobufjs-cli/bin
total 16
-rw-r--r--  1 username  staff  203 Mar 24 15:22 pbjs
-rw-r--r--  1 username  staff  203 Mar 24 15:22 pbts

➜ yarn --version
3.5.0

[keepitsimple]

Hm.. I don't thinks it's related only with manually removed folders.

I have similar issues with vite (it cannot find esbuild) and @nestjs/microservices (when I use GRPC) due to the @grpc/proto-loader module cannot be found.

Bug is always reproducible in the pnpm mode.

[dawnmist]

I have a repro of the issue demonstrated with a sentry dependency in the issue initially reported there: getsentry/sentry-cli#1851

Example is an absolute basic npm project with just the 2 dependencies that were required to demonstrate the issue.

I think it's all binary executables that are having issues, not just when the folder has been removed, because I could reproduce it reliably on a fresh install/checkout. I ran into similar ones in esbuild that I initially thought were missed chmod instructions in the build script (which I patched), then ran into the sentry-cli also not being made executable properly.

Environment

• node v20.10.0
• yarn v4.0.2, with pnpm nodeLinker
• "@sentry/remix@npm:7.84.0"
• "@sentry/react@npm:7.84.0"
• "@sentry/node@npm:7.84.0"
• "@sentry/core@npm:7.84.0"
• "@sentry/cli@npm:2.21.3"
• "@sentry/cli-linux-x64@npm:2.22.3"

Minimal example reproduction available from https://github.com/dawnmist/test-sentry-cli-permissions

Steps to Reproduce

• Check out the repo https://github.com/dawnmist/test-sentry-cli-permissions
• yarn install
• ls -l node_modules/.store/@sentry-cli-x64-npm-2.22.3-81f995f8ee/package/bin

Note that the permissions are rw-r--r-- instead of rwxr-xr-x, which means that any attempt to execute the sentry-cli will fail and throw an EACCES error.

Expected/Actual Result

I was using this to create a new sentry release for a remix project (via the createRelease script in @sentry/remix/scripts/createRelease.js), but the lack of execute permission for sentry-cli results in the error:

Failed to create a sentry release Error: spawn /home/username/project/node_modules/.store/@sentry-cli-linux-x64-npm-2.22.3-81f995f8ee/package/bin/sentry-cli EACCES
    at __node_internal_captureLargerStackTrace (node:internal/errors:563:5)
    at __node_internal_errnoException (node:internal/errors:690:12)
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -13,
  code: 'EACCES',
  syscall: 'spawn /home/username/project/node_modules/.store/@sentry-cli-linux-x64-npm-2.22.3-81f995f8ee/package/bin/sentry-cli',
  path: '/home/username/project/node_modules/.store/@sentry-cli-linux-x64-npm-2.22.3-81f995f8ee/package/bin/sentry-cli',
  spawnargs: [ 'releases', 'new', '9de06f6330363c6aae9d0409ef3e9ca6025dec3c' ],
  cmd: '/home/username/project/node_modules/.store/@sentry-cli-linux-x64-npm-2.22.3-81f995f8ee/package/bin/sentry-cli releases new 9de06f6330363c6aae9d0409ef3e9ca6025dec3c'
}

[bradchristensen]

I've noticed this when upgrading a CLI package like Biome (@biomejs/biome). My process is to manually bump the package version in the root-level package.json (e.g. from ~1.4.0 to ~1.4.1) and then re-run yarn, which updates the lockfile and replaces the old version of the package with the new one. The newly installed version does not have the executable bit set.

This persists when other members of my team pull down the latest version of the branch and run yarn to update their local dependencies.

However, this only seems to be a problem if at least some packages are already installed: conversely, if we do a fresh install by manually removing the node_modules folder and then re-running yarn, the package is installed with the executable bit set on the binary as expected.

Our project is using the node-modules linker like the OP, with Yarn 4.0.2 (edit - we're still facing this problem on Yarn 4.2.2 as well).

[issacgerges]

I've just hit this as well. A few things to note

• Only our MacOS developer machines seemed to be impacted by it, not our Linux CI hosts (update: some feedback below makes it seem like this isn't true across the board)
• The file showed the +x, but manually running chmod +x node_modules/.bin/ts-node fixed it

I wonder if this is just some very specific macos bug and could be avoided by having yarn install manually chmod +x of items in .bin on install/upgrade of a module

[bradchristensen]

@issacgerges this occurs for me on Linux too (Ubuntu/WSL2 and inside Debian Docker containers).

[ssbarnea]

So that was what was breaking mocha in my project again and again, as mocha bin ended up missing the execute bit. I had to remove it from the cache to make it work. Apparently the cache copy was missing the bit.

[jdelStrother]

FWIW we ended up adding this to package.json to paper over the issue:

  "scripts": {
    "// postinstall": "yarn often seems to lose the executable bit on node_modules/prettier/bin/prettier.cjs. Try & autofix it after install:",
    "postinstall": "chmod +x node_modules/.bin/*"

[flying-sheep]

I’m trying to use biome’s VS Code extension together with nodeLinker: pnpm. Having @biomejs/biome in my devDependencies doesn’t result in node_modules/.bin/ folder at all.

Setting "biome.lspBin": "./node_modules/@biomejs/biome/bin/biome", also doesn‘t work, as that file doesn’t have the executable bit set.

That extension would even support pnp, but I can‘t use it due to other parts of the project not supporting pnp.