Docker Desktop truncates the standard output

[almet]

Description

The standard output of a docker run command is truncated if not read() right away (I've included an example of this below).

This seems to happen even on the latest Docker Desktop version, on (at least) macOS and Linux.

Reproduce

Here is a minimal reproducible example of this issue:

import shlex
import subprocess
import sys
import time

# reference.txt can be generated with random bytes (or even zeros)
# head -c 18933764 < /dev/urandom > reference.txt

command = [
    "/usr/local/bin/docker",
    "run",
    "-v",
    "./reference.txt:/tmp/reference.txt",
    "--rm",
    "-i",
    "debian",
    "bash",
    "-c",
    "cat /tmp/reference.txt",
]
proc = subprocess.Popen(command, stdout=subprocess.PIPE)

buffer = b""

counter = 0
while True:
    current = proc.stdout.read(1000)

    # Wait 1s every megabyte (approx.)
    if counter % 1000 == 0:
        time.sleep(1)

    if not current:
        break

    sys.stdout.buffer.write(current)
    counter += 1

When run, it returns a different number of bytes than expected:

$  cat reference.txt | wc -c
 18933764

$ python ./fixme.py | wc -c
11075584

Expected behavior

It should return 18933764 bytes instead of less.

docker version

Client:
 Version:           28.0.1
 API version:       1.48
 Go version:        go1.23.6
 Git commit:        068a01e
 Built:             Wed Feb 26 10:38:16 2025
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.39.0 (184744)
 Engine:
  Version:          28.0.1
  API version:      1.48 (minimum version 1.24)
  Go version:       go1.23.6
  Git commit:       bbd0a17
  Built:            Wed Feb 26 10:40:57 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    28.0.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v0.9.8
    Path:     /Users/alexis/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.1-desktop.2
    Path:     /Users/alexis/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.33.1-desktop.1
    Path:     /Users/alexis/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     /Users/alexis/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.5
    Path:     /Users/alexis/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/alexis/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/alexis/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/alexis/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/alexis/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/alexis/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.16.3
    Path:     /Users/alexis/.docker/cli-plugins/docker-scout

Server:
 Containers: 20
  Running: 0
  Paused: 0
  Stopped: 20
 Images: 6
 Server Version: 28.0.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.654GiB
 Name: docker-desktop
 ID: 155e1359-189a-44fb-a4e2-d5797be0af4c
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 64
  Goroutines: 81
  System Time: 2025-03-19T14:36:53.737763135Z
  EventsListeners: 12
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/alexis/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Diagnostics ID

887FB83C-9093-438B-B224-22ADA5333627/20250319145338

Additional Info

Thanks a lot for looking into this, and don't hesitate to ask if you have any question / need some information.

[almet]

As a follow up, to avoid doing the same work twice:

We have tested this with python 3.13 and with a rust implementation of this script and faced the same error. We also have tested this with Podman Desktop and didn't face an issue there.

Rust code

use std::io::{self, Read, Write};
use std::process::{Command, Stdio};
use std::thread::sleep;
use std::time::Duration;

fn main() -> io::Result<()> {
    // Create the docker command
    let mut cmd = Command::new("/usr/local/bin/docker");
    cmd.args([
        "run",
        "-v",
        "./reference.txt:/tmp/reference.txt",
        "--rm",
        "-i",
        "debian",
        "bash",
        "-c",
        "cat /tmp/reference.txt",
    ]);

    // Set up the process with piped stdout
    cmd.stdout(Stdio::piped());
    
    // Start the process
    let mut process = cmd.spawn()?;
    
    // Unwrap the stdout from the process
    let mut stdout = process.stdout.take().expect("Failed to get stdout");
    
    // Create a buffer to read into
    let mut buffer = [0u8; 1000];
    let mut counter = 0;
    
    // Read in chunks and write to stdout
    loop {
        // Read up to 1000 bytes at a time
        match stdout.read(&mut buffer) {
            Ok(0) => break, // End of file
            Ok(bytes_read) => {
                // Write the bytes to stdout
                io::stdout().write_all(&buffer[0..bytes_read])?;
                
                counter += 1;
                
                // Sleep every 1000 chunks (approximately 1MB)
                if counter % 1000 == 0 {
                    sleep(Duration::from_secs(1));
                }
            }
            Err(e) => return Err(e),
        }
        io::stdout().flush()?;
    }
    
    // Wait for the process to finish
    let status = process.wait()?;
    
    if !status.success() {
        eprintln!("Docker command failed with exit code: {:?}", status.code());
    }
    
    Ok(())
}

[vvoland]

Thanks, I managed to reproduce this on my side on v28.0.2. I didn't reproduce this on v27.5.1 though, so looks like it's a regression in the CLI.

I'm investigating the exact cause. In the meantime, would you like to confirm that it doesn't happen with a v27.5.1 CLI? (you can grab a 27.5.1 binary from https://download.docker.com/mac/static/stable/aarch64/docker-27.5.1.tgz)

[almet]

Hi, and thanks for the quick response.

would you like to confirm that it doesn't happen with a v27.5.1 CLI?

When running the script against the provided binary for v27.5.1, the issue doesn't happen, so it indeed seem to be a regression in the CLI.