[4.38.0] Docker daemon takes too much time to respond

[idsulik]

Description

docker/compose#12511

4.38.0
curl -w "%{time_total}\n" --unix-socket /var/run/docker.sock http://localhost/v1.47/containers/json
3.004582

4.37.2
curl -w "%{time_total}\n" --unix-socket /var/run/docker.sock http://localhost/v1.47/containers/json
0.003464

Reproduce

1. Install the latest version 4.38.0
2. execute curl -w "%{time_total}\n" --unix-socket /var/run/docker.sock http://localhost/v1.47/containers/json

Expected behavior

Quick response

docker version

Client:
 Version:           27.5.1
 API version:       1.47
 Go version:        go1.22.11
 Git commit:        9f9e405
 Built:             Wed Jan 22 13:37:19 2025
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.38.0 (181591)
 Engine:
  Version:          27.5.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.11
  Git commit:       4c9b3b0
  Built:            Wed Jan 22 13:41:25 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e946
 docker-init:
  Version:          0.19.0

docker info

Client:
 Version:    27.5.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Ask Gordon - Docker Agent (Docker Inc.)
    Version:  v0.7.3
    Path:     /Users/main/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.20.1-desktop.2
    Path:     /Users/main/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4-desktop.1
    Path:     /Users/main/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     /Users/main/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.4
    Path:     /Users/main/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/main/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/main/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/main/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/main/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/main/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.16.1
    Path:     /Users/main/.docker/cli-plugins/docker-scout

Server:
 Containers: 46
  Running: 0
  Paused: 0
  Stopped: 46
 Images: 49
 Server Version: 27.5.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: active
  NodeID: khifaf7qqhm1j2z6kt8yoqt24
  Is Manager: true
  ClusterID: 5k3uecgoqb65bz9nv9j8f95v4
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.65.3
  Manager Addresses:
   192.168.65.3:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.1.12-0-g51d5e946
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.12.5-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.828GiB
 Name: docker-desktop
 ID: eb880ffc-4222-47b2-ac62-561e8b86e1d8
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/main/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

6787D1E1-06B7-48A1-9287-14A2FDB53286/20250201171337

Additional Info

No response

[thaJeztah]

Thanks for reporting; I'm trying to reproduce this issue, but so far wasn't able to. The 3 seconds sounds like an issue I saw at some point though, and which was related to code in Docker Desktop's proxy to try to refresh port-mappings under certain conditions with a timeout of 3 seconds.

ISTR that was a strict requirement for the Docker Desktop GUI, but there was some conditional code to skip that logic for the docker CLI and other clients;

• API will be slow when not "User-Agent" in header moby/moby#47439

If memory serves me well, there was some logic that depended on specific user-agents, and I also recall some conversation on our internal Slack about user-agents being updated, so now I wonder if something regressed there.

To confirm my suspicion, would you be able to try what happens if you set a user-agent that starts with Docker-Client/ ? From that other ticket, this should probably work;

curl -s --unix-socket /var/run/docker.sock -H "User-Agent: Docker-Client/27.5.1" "http://localhost/v1.47/containers/json?all=true"

[thaJeztah]

I did a quick search in the code, and I see there's indeed still some logic related to this, and given that you mentioned compose, I recalled that compose at some point changed to use its own user-agent;
https://github.com/docker/compose/blob/4db5fcd569f29b0c2972635a1b6162e022a070f3/internal/desktop/client.go#L113-L117
https://github.com/docker/compose/blob/4db5fcd569f29b0c2972635a1b6162e022a070f3/internal/desktop/client.go#L36-L37

If that's related, it's possible that docker ps would use the Docker-Client/ user-agent, but compose would use something with compose/ as prefix, and possibly hit a different codepath. So possibly you could check if docker ps indeed does not show the problem (and responds fast)

[idsulik]

@thaJeztah yes, you're right, it works in this way:
curl -s --unix-socket /var/run/docker.sock -H "User-Agent: Docker-Client/27.5.1" "http://localhost/v1.47/containers/json?all=true"

[idsulik]

@thaJeztah, should I fix the UserAgent in the compose repo, or will it be fixed in the daemon?

[thaJeztah]

Thanks for testing! No, docker compose shouldn't update; or at least, shouldn't have to because this looks like a regression in the docker desktop API proxy.

I recall that the fix at the time was fixing the immediate issue, but that there was a tracking ticket to look at ways to remove that conditional code altogether (having conditional code based on user-agent definitely isn't ideal! I recall the alternative was a much more involved change, so as things go, that was left as a backlog item).

I already posted this issue internally, but I'll try to find the right person to look into this after the weekend.

[idsulik]

Thank you for the details! I'll close the PR then.