"Use Rosetta": Certain node.js/amd64 workloads cause container to become unresponsive/100% CPU

Description

A few folks on our team have been using Use Rosetta for x86/amd64 emulation on Apple Silicon with our moderately sized docker-compose stack, which is primarily amd64 images of Node.js apps. For the most part, Rosetta is speedier and a net gain, however, we've been noticing that some of our containers will hit 100% CPU and become entirely unresponsive.

I tried many things to debug the 100% CPU, but between the virtualization and the containerization I wasn't able to get perf or any linux debuggers working. I was able to strace but no syscalls were shown, which suggests the app was caught in a tight CPU-bound user loop and not e.g. doing IO or network calls. I was able to use node --inspect with port forwarding, but the debugger stopped responding once it got into a 100% CPU state.

I was able to get an isolated reproduction case however, using an old npm library that performs crypto operations in pure Node. See below.

Reproduce

At a high level, to reproduce, create an amd64 Docker image which uses the npm package keypair, and attempt to create a 2048 bit key.

With Rosetta off, this succeeds after 5-20s.

With Rosetta on, the container immediately hits 100% CPU and "never" returns (I gave up after 14 hours).

Reproduction repo with instructions: https://github.com/jbinto/rosetta-what
Dockerhub: https://hub.docker.com/repository/docker/jbinto/rosetta-what/general

Expected behavior

These types of Node.js amd64 workloads should succeed in Rosetta mode, and not become unresponsive/100% CPU.

docker version

Client:
 Cloud integration: v1.0.35+desktop.4
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.23.0 (120376)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:36 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.4
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0-desktop.1
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.7
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.24.1
    Path:     /Users/jbuchanan/.docker/cli-plugins/docker-scout

Server:
 Containers: 15
  Running: 13
  Paused: 0
  Stopped: 2
 Images: 21
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.3.13-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.666GiB
 Name: docker-desktop
 ID: 84081c6c-7297-46a4-82bb-1039ade535ad
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

3F4DBEC3-6E23-4A45-9421-B2A0CCBF2058/20230926181652

Additional Info

It's unclear whether or not this code is just very, very slow, or if there's some issue in translating the code that causes an incorrect infinite loop. I don't have the skills or tools to drop any lower to see what's actually happening.

But in any case, the outcome is the same: I have to tell my team to turn off Use Rosetta for x86/amd64 emulation on Apple Silicon because our containers will eventually hit 100% CPU and die. And this is not ideal because Rosetta otherwise represents something like a 4-5x speedup over qemu, which we really appreciate.

In my case, disabling experimental features was working.
This might be help for your situation. :)

@fnaoto

Hi, I'm noticing the same hanging effect with any npm command when Use Rosetta for x86/amd64 emulation on Apple Silicon is enabled.

I'm using an Apple M2 Pro (Apple Silicon) running Docker Desktop for Mac v4.23.0 (120376).

This simple command demonstrates the issue for me:

docker run --rm -it --platform=linux/amd64 --entrypoint /bin/sh node:18-alpine
$> node --version
v18.18.0
$> npm --version
9.8.1
# then it hangs, command never ends

Disabling the Rosetta setting in Docker Desktop as suggested by @fnaoto fixes the issue for me.

Thanks

These other posts on Reddit may be people experiencing the same issue:

Disabling experimental features did not work for us (it was already disabled when I submitted this issue).

Our workaround has been to bite the bullet and build arm64 images for everything.

yarn install also fails if Rosetta is enabled. On an M1 Mac with Rosetta:

docker run -it --platform linux/amd64 node:18-alpine yarn install
yarn install v1.22.19

and then there's no further output. I'm using Docker 4.23.0. If I disable Rosetta, the command runs correctly.

Same issue here, in my case on M1 macOS 14.0 Docker Desktop v4.24.0 (122432)

Workaround: switch off Rosetta as shown on screenshot

I updated to Docker 4.25.0 and macOS from 13.3.1 to 13.6.1 and my build issues are fixed, even with Rosetta enabled!

@jbinto

@jbinto

Disabling experimental features did not work for us (it was already disabled when I submitted this issue).

Could be because Rosetta was out of experimental already (way to switch it off had changed).

Hi, Any news about that ?

I've also just bumped into this issue. M1 MBP, MacOS 13.6.2 (latest Ventura), Docker Desktop 4.25.2 (latest) with Rosetta enabled, running nuxt build via npm run build inside a --platform=linux/amd64 node:16-alpine image. The build process just hangs while compiling the server and never completes. Disabling Rosetta fixes the issue for me. A colleague on an M2 MBP on 14.1.1 (latest Sonoma) and same Docker Desktop version has the same issue.

@jjang16

Thank you @jjang16, your issue is indeed related because it's caused by the fix to the original issue. I'll take a look at it

@jjang16

@jjang16 Would you have a simple set of commands that I can use to reproduce your issue?

@jjang16

@jjang16 This will be fixed in Docker Desktop 4.27

@jjang16

@jjang16 This will be fixed in Docker Desktop 4.27

Is there an ETA for this release?

@workspace

Macbook M3 Pro
Node 18.10.0
Docker Desktop 4.26.1

Recibia el mismo error YN0001: │ Error [ERR_WORKER_INVALID_EXEC_ARGV]: <dependency>: Initiated Worker with invalid execArgv flags: --no-opt .

Al tratar de ejecutar un DockerFile donde esta el comando yarn @workspace run build , al parecer se soluciono desactivando el rosetta , gracias @jjang16 .

@JavierAmaya

@JavierAmaya @MattyBalaam 4.27 should be out this week

@JavierAmaya

@JavierAmaya @MattyBalaam Could you test Docker Desktop 4.27 and close this issue if it's fixed?

@dgageot

@dgageot it works for my case 🥳

I am encountering exactly the same problem when running Docker (25.0.2) on a Mac M3 (14.3) in an Ubuntu 22 Virtual Machine supporting compatibility with Rosetta (Parallels). This could be due to Docker on Ubuntu or the emulation of the Ubuntu VM (Parallels)?

@vincent-herlemont

I am encountering exactly the same problem when running Docker (25.0.2) on a Mac M3 (14.3) in an Ubuntu 22 Virtual Machine supporting compatibility with Rosetta (Parallels). This could be due to Docker on Ubuntu or the emulation of the Ubuntu VM (Parallels)?

@vincent-herlemont Are you running Docker for Linux? If so, I think you should create a separate issue in the Docker for Linux repo. You can reference this issue there, but this issue is about Docker for Mac.

@sbleon

@sbleon Yes, I am using Docker for Linux with Ubuntu, and you are right, I have opened an issue here: docker/for-linux#1483.
Thank you for pointing it out.

FWIW I encountered this same issue on 4.34. That is, the dockerfile I was using (which is not mine) builds a bunch of stuff using pnpm. It would freeze partway through the build.

I then disabled the Use Rosetta for x86/amd64 emulation on Apple Silicon setting and the build was successful.

Navigation Menu

Repository navigation

"Use Rosetta": Certain node.js/amd64 workloads cause container to become unresponsive/100% CPU #6998

Description

Reproduce

Expected behavior

docker version

docker info

Diagnostics ID

Additional Info

15 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

"Use Rosetta": Certain node.js/amd64 workloads cause container to become unresponsive/100% CPU #6998

Description

Description

Reproduce

Expected behavior

docker version

docker info

Diagnostics ID

Additional Info

Activity

fnaoto commented on Sep 28, 2023

douglascayers commented on Oct 5, 2023

jbinto commented on Oct 5, 2023

sbleon commented on Oct 5, 2023

corneliusroemer commented on Oct 6, 2023

sbleon commented on Nov 3, 2023

Krienas commented on Nov 9, 2023

enzofrnt commented on Nov 20, 2023

jgillard commented on Nov 22, 2023

15 remaining items

dgageot commented on Jan 2, 2024

dgageot commented on Jan 2, 2024

dgageot commented on Jan 2, 2024

MattyBalaam commented on Jan 22, 2024

JavierAmaya commented on Jan 22, 2024

dgageot commented on Jan 22, 2024

dgageot commented on Jan 26, 2024

MattyBalaam commented on Jan 26, 2024

vincent-herlemont commented on Feb 6, 2024

sbleon commented on Feb 6, 2024

vincent-herlemont commented on Feb 6, 2024

johnpitchko commented on Jan 2, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions