c8d: Add `--platform` flag to history, save and load - #5331

vvoland · 2024-08-08T11:22:13Z

Add --platform flag to history, save and load.

- Description for the changelog

containerd image store: `docker load` and `docker save` now support a `--platform` flag allowing to choose a specific single-platform image to load/save if the image is multi-platform
containerd image store: `docker history` now supports a `--platform` flag allowing to choose a specific platform to show the history for.

- A picture of a cute animal (not mandatory but encouraged)

dvdksn · 2024-09-24T13:01:23Z

+If the platform is not specified, the host platform is preferred if it's available, otherwise any available platform is used.
+
+Format: os[/arch[/variant]]
+Example: "linux/amd64"`)


How about including the full cmd?

Suggested change

Example: "linux/amd64"`)

Example: "docker image history --platform linux/amd64"`)

I (somewhat) assume that this was done to allow defining a global const for these flags and for them to be shared (similar to what we did for --format 🤔), but I need to check if they differ in other ways

dvdksn · 2024-09-24T13:04:39Z

+		`Pick a single-platform to be loaded if the image is multi-platform.
+Full multi-platform image will be load if not specified.
+
+Format: os[/arch[/variant]]
+Example: "linux/amd64"`)


Suggested change

`Pick a single-platform to be loaded if the image is multi-platform.

Full multi-platform image will be load if not specified.

Format: os[/arch[/variant]]

Example: "linux/amd64"`)

`Specify a platform from a multi-platform image to load.

If a platform is not specified, and the image is a multi-platform image, all platforms are loaded.

Format: os[/arch[/variant]]

Example: "docker image load --platform linux/amd64"`)

dvdksn · 2024-09-24T13:05:49Z

+		`Pick a single-platform to be saved if the image is multi-platform.
+Full multi-platform image will be saved if not specified.
+
+Format: os[/arch[/variant]]
+Example: "linux/amd64"`)


Suggested change

`Pick a single-platform to be saved if the image is multi-platform.

Full multi-platform image will be saved if not specified.

Format: os[/arch[/variant]]

Example: "linux/amd64"`)

`Specify a platform from a multi-platform image to save.

If a platform is not specified, and the image is a multi-platform image, all platform variants are saved.

Format: os[/arch[/variant]]

Example: "docker image save --platform linux/amd64"`)

codecov-commenter · 2024-09-24T14:34:38Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.67%. Comparing base (6a78e92) to head (d085e24).
Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5331      +/-   ##
==========================================
+ Coverage   60.63%   60.67%   +0.04%     
==========================================
  Files         345      345              
  Lines       23462    23488      +26     
==========================================
+ Hits        14226    14252      +26     
  Misses       8263     8263              
  Partials      973      973

thaJeztah · 2024-09-24T15:23:36Z

+		`Specify a platform from a multi-platform image to show the history for.
+If the platform is not specified, the host platform is preferred if it's available, otherwise any available platform is used.


@dvdksn I applied your suggestions; I'm slightly wondering if we should / could condense the information a bit; for example;

Specify a platform -> Platform from a multi-platform image ...

(I don't think we generally add Specify .... as that's a bit implicit for options?)

The "format" could possibly be included in the first line, something like;

Platform from a multi-platform image to show the history for (format: "os[/arch[/variant]]") By default, the daemon's native platform is preferred, otherwise any available platform is used.

(wondering if the host platform is ambiguous, and if we need to emphasise daemon platform)

i also think we should probably change "any available platform is used" to something more precise.. what does "any" mean in this context? the first available platform in alphabetical order?

@thaJeztah your suggestion sgtm. Coming back to this I realize that multi-line flag descriptions are kinda exotic - we don't use them a lot. Maybe we should instead create sections in the respective .md files for the platform flag, like we usually do, to show examples.

IMO, if there is a specific behavior we've decide is part of the contract "i.e. it's always the first alphabetical one", then we should make that explicit. On the other hand, if we explicitly don't want to make any future guarantees about which platform is used, then any is fine. I honestly kind of would rather just return an error if there's no platform for the given image, but 🤷‍♀️ it's not a big deal either way, especially for history.

laurazard · 2024-10-04T12:11:20Z

I had a thought while looking at this again today – I think it would be really nice if we had a main .md file/doc with a general explanation about multi-platform images – i.e. "this is a new feature available on docker if the containerd store integration is enabled, it's kind of like the old images but now each image is actually an index and may contain multiple "images" for different platforms, etc.,". We could link there from a lot of other places, and we could also have a little table with the different commands that take a parameter to specify a platform if relevant, and I think it'd tie a lot of things together that we're kind of leaving implied/are spread out through a lot of different places/docs right now. Kind of like the multi-platform build doc for Build. cc @thaJeztah @dvdksn

dvdksn · 2024-10-04T18:32:24Z

@laurazard the multi-platform builds page has some of that. I agree it would be nice with a general "what's an image, actually" primer. We should probably consider OCI the typical format by now — buildx will default to them more often than not, and with containerd becoming the default storage, it feels like the transition is coming sooner rather than later. Single-platform manifests is a legacy format in my world.

thaJeztah · 2024-10-07T14:37:52Z

@dvdksn @laurazard I rewrote the PR to have a short description for the flags, and move the description to the docs.

There's still some issues/bugs to fix on the daemon side (as I found some issues while working on the docs; moby/moby#48589)

thaJeztah · 2024-10-07T16:00:06Z

Oh! Should mention that I didn't squash / cleanup commits yet; will do so before merging if this looks good.

thaJeztah · 2024-10-08T15:48:12Z

let me start cleaning up the commit history on this one to bring it in a mergable state

thaJeztah · 2024-10-08T16:45:58Z

+			_, _ = fmt.Fprintf(dockerCli.Err(), "Invalid platform %s", opts.platform)
+			return err


I just realised that this isn't correct; it tries to both print something as well as returning an error. The printed message is without a newline. But also, because the error is returned, both the printed message and the error get printed;

docker image save -o image-amd64.tar --platform in:valid/foo:bar alpine:latest Invalid platform in:valid/foo:bar"in:valid" is an invalid OS component of "in:valid/foo:bar": OSAndVersion specifier component must match "^([A-Za-z0-9_-]+)(?:\$([A-Za-z0-9_.-]*)\$)?$": invalid argument

I wondered if this was perhaps done for cases where no output file is specified (so the tar output would be redirected), but the same happens in that case;

docker image save --platform in:valid/foo:bar alpine:latest > file.tar Invalid platform in:valid/foo:bar"in:valid" is an invalid OS component of "in:valid/foo:bar": OSAndVersion specifier component must match "^([A-Za-z0-9_-]+)(?:\$([A-Za-z0-9_.-]*)\$)?$": invalid argument

So I think this should just wrap the error and return it.

Updated these to just return the error

thaJeztah · 2024-10-08T17:47:51Z

LOL, and there I thought "let me add some basic unit-tests" and scratching my head "why doesn't it fail??"

cli/command/image: fix TestNewSaveCommandSuccess to actually test #5520

thaJeztah · 2024-10-08T18:38:53Z

OK; this should probably be ready for review / merge now;

Squashed / cleaned up commits
Changed the fmt.FPrint to just return the error instead; c8d: Add --platform flag to history, save and load #5331 (comment)
Added some basic unit tests

laurazard · 2024-10-10T12:42:53Z

Oops, meant to comment/hit the other button. Just had found some typos/suggestions, otherwise it looks good.

thaJeztah · 2024-10-10T12:59:53Z

Updated! I think I got them all, @laurazard

laurazard

LGTM

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

dvdksn

LGTM 🚀

fix(ci): docker save 显式指定 --platform 修复 arm64 离线镜像导出失败 Created-by: zhongyiwei Commit-by: zhongyiwei Merged-by: jdfff Description: Paired: [GitHub #1306](#1306) ↔ [GitCode !2518](https://gitcode.com/openJiuwen/agent-studio/merge_requests/2518) ## 问题 CI `export-images (arm64)` 稳定失败（如 [run #39758270150](https://github.com/openJiuwen-ai/agent-studio/actions/runs/39758270150/job/119778399494)），`export-images (amd64)` 正常： ```text Status: Downloaded newer image for ghcr.io/.../studio-manager:dev-20261022174857 ← docker pull 成功 Error response from daemon: unable to create manifests file: NotFound: content digest sha256:b6c7677b...: not found ← 下一条命令 docker save 失败 ``` ## 根因 `ubuntu-latest` runner 的 Docker 已升级到 Engine 29+，[containerd image store 默认启用](https://docs.docker.com/engine/storage/containerd/)。该存储后端下 `docker save`（不带 `--platform`）**默认导出宿主机平台（amd64）的内容**；arm64 导出 job 在 amd64 runner 上只拉取了 arm64 内容，tag 又指向完整多架构 index → 按 amd64 找 manifest → `content digest not found`。 - 报错中缺失的 digest `sha256:b6c7677b...` 已确认是 GHCR 上该 tag 的 **linux/amd64** manifest - 上游已知问题：[docker/cli#5476](https://github.com/docker/cli/issues/5476)（open）、[#5493](https://github.com/docker/cli/issues/5493)（相对 24.0.7 的 regression） - 镜像本身完好，无需重建；原样重跑会确定性失败完整证据链与本地复现记录见 #1582。 ## 修改三个 workflow 的导出步骤各改一行，给 `docker save` 显式指定目标平台（该 flag 自 Docker 28.0 引入，见 [docker/cli#5331](https://github.com/docker/cli/issues/5331)，runner 必然支持；amd64 job 加了也无影响）： - `.github/workflows/build-push-ghcr-dev.yml`（本次失败） - `.github/workflows/build-push-ghcr.yml` - `.github/workflows/build-push-dev.yml` ```diff - docker save -o "images/${SERVICE}-${ARCH_SUFFIX}.tar" "${IMAGE}" + docker save --platform "linux/${ARCH}" -o "images/${SERVICE}-${ARCH_SUFFIX}.tar" "${IMAGE}" ``` ## 验证 Docker 29.9.1（containerd image store）本地验证： ```bash $ docker pull --platform linux/arm64 alpine:latest # 成功 $ docker save -o /tmp/repro.tar alpine:latest # 复现同款报错 $ docker save --platform linux/arm64 -o /tmp/fix.tar alpine:latest # 修复后成功 $ docker load -i /tmp/fix.tar # RepoTags 正常、architecture=arm64，回环成功 ``` Closes #1582   Linked Closing Issues: - [#1582](https://gitcode.com/openJiuwen/agent-studio/issues/1582)  See merge request: openJiuwen/agent-studio!2518

vvoland added impact/changelog area/api area/ux containerd-integration Issues and PRs related to containerd integration labels Aug 8, 2024

vvoland added this to the 28.0.0 milestone Aug 8, 2024

vvoland requested review from a team and dvdksn August 8, 2024 11:22

vvoland force-pushed the c8d-saveload-platform branch from 2ee583e to b3139f5 Compare August 8, 2024 13:45

vvoland self-assigned this Sep 6, 2024

thaJeztah mentioned this pull request Sep 12, 2024

vendor: github.com/docker/docker 164cae56ed95 (master, v-next) #5428

Merged

thaJeztah force-pushed the c8d-saveload-platform branch from b3139f5 to 8f20d2e Compare September 24, 2024 11:48

dvdksn reviewed Sep 24, 2024

View reviewed changes

thaJeztah force-pushed the c8d-saveload-platform branch from 8f20d2e to 400f5f0 Compare September 24, 2024 14:32

thaJeztah force-pushed the c8d-saveload-platform branch from 400f5f0 to 8ebf4cf Compare September 24, 2024 15:19

thaJeztah marked this pull request as ready for review September 24, 2024 15:19

thaJeztah requested a review from a team as a code owner September 24, 2024 15:19

thaJeztah reviewed Sep 24, 2024

View reviewed changes

thaJeztah mentioned this pull request Sep 24, 2024

Docker Containerd snapshotter Overlayfs, docker save fails for image pulled using --platform=linux/arm64 on amd64 host. #5476

Open

dvdksn mentioned this pull request Sep 27, 2024

add Docker Desktop 4.35.0 release notes docker/docs#20909

Merged

thaJeztah force-pushed the c8d-saveload-platform branch from 8ebf4cf to 3b97d52 Compare October 4, 2024 11:53

thaJeztah force-pushed the c8d-saveload-platform branch from 3b97d52 to 7731583 Compare October 7, 2024 13:37

thaJeztah mentioned this pull request Oct 8, 2024

add shell-completion for --platform flags #5516

Merged

Benehiko approved these changes Oct 8, 2024

View reviewed changes

thaJeztah force-pushed the c8d-saveload-platform branch from d30d61a to c656e2e Compare October 8, 2024 14:54

thaJeztah reviewed Oct 8, 2024

View reviewed changes

thaJeztah force-pushed the c8d-saveload-platform branch from c656e2e to c7cc278 Compare October 8, 2024 18:37

Benehiko approved these changes Oct 9, 2024

View reviewed changes

thaJeztah mentioned this pull request Oct 9, 2024

With containerd, I pulled images debian, ubuntu, nginx all latest, using --platform=linux/arm64/v8 and multiple strange observations. #5523

Open

laurazard requested changes Oct 10, 2024

View reviewed changes

thaJeztah force-pushed the c8d-saveload-platform branch from c7cc278 to 6acec20 Compare October 10, 2024 12:59

laurazard approved these changes Oct 10, 2024

View reviewed changes

dvdksn reviewed Oct 10, 2024

View reviewed changes

Comment thread docs/reference/commandline/image_save.md Resolved

Comment thread docs/reference/commandline/image_save.md Resolved

Comment thread docs/reference/commandline/image_load.md Resolved

vvoland added 3 commits October 10, 2024 16:35

image/save: Add --platform

a20eb45

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

image/load: Add --platform

b0bb4ba

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

image/history: Add --platform flag

d085e24

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah force-pushed the c8d-saveload-platform branch from 6acec20 to d085e24 Compare October 10, 2024 15:00

dvdksn approved these changes Oct 10, 2024

View reviewed changes

thaJeztah merged commit f483aac into docker:master Oct 10, 2024

thaJeztah mentioned this pull request Oct 10, 2024

Touch-up some errors for missing platforms moby/moby#48631

Merged

vvoland mentioned this pull request Oct 18, 2024

[27.x backport] Touch-up some errors for missing platforms moby/moby#48691

Merged

This was referenced Oct 25, 2024

c8d/load: Don't ignore missing platform when requested moby/moby#48718

Merged

[27.x backport] c8d/load: Don't ignore missing platform when requested moby/moby#48763

Closed

vvoland mentioned this pull request Jan 27, 2025

containerd integration: Platform selection moby/moby#47680

Closed

15 of 18 tasks

pullfrog Bot mentioned this pull request Jul 30, 2026

feat(cli): abctl claude — coding-agent TUI in a sandbox arcboxlabs/arcbox#505

Merged

openjiuwen-collaboration-bot Bot mentioned this pull request Oct 22, 2026

[Bug]: CI export-images (arm64) 失败：docker save 需显式 --platform（Docker 29 containerd image store） openJiuwen-ai/agent-studio#1305

Closed

openjiuwen-sync-bot Bot mentioned this pull request Oct 22, 2026

fix(ci): docker save 显式指定 --platform 修复 arm64 离线镜像导出失败 openJiuwen-ai/agent-studio#1306

Merged

	Example: "linux/amd64"`)
	Example: "docker image history --platform linux/amd64"`)

		`Specify a platform from a multi-platform image to show the history for.
		If the platform is not specified, the host platform is preferred if it's available, otherwise any available platform is used.

		_, _ = fmt.Fprintf(dockerCli.Err(), "Invalid platform %s", opts.platform)
		return err

Conversation

vvoland commented Aug 8, 2024 • edited by thaJeztah Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

thaJeztah Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laurazard commented Oct 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dvdksn commented Oct 4, 2024

Uh oh!

thaJeztah commented Oct 7, 2024

Uh oh!

thaJeztah commented Oct 7, 2024

Uh oh!

thaJeztah commented Oct 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Oct 8, 2024

Uh oh!

thaJeztah commented Oct 8, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

laurazard commented Oct 10, 2024

Uh oh!

thaJeztah commented Oct 10, 2024

Uh oh!

laurazard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dvdksn left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

vvoland commented Aug 8, 2024 •

edited by thaJeztah

Loading

codecov-commenter commented Sep 24, 2024 •

edited

Loading

thaJeztah Sep 24, 2024 •

edited

Loading

laurazard commented Oct 4, 2024 •

edited

Loading