CLI Plugin docs gone missing?

[christian-korneck]

the documentation on Docker CLI plugins has gone missing. It used to be here: https://docs.docker.com/engine/extend/cli_plugins/

As I'm using some homebrewn CLI plugins I found this info very helpful.

Is there a reason why it has been removed? Is support for CLI plugins deprecated?

[mehyedes]

Looks like they are deprecated
docker/cli@459c608

[christian-korneck]

thanks - does anyone know what their deprecated with? Is there a successor how to extend the Docker CLI?

[christian-korneck]

interestingly Docker have released a new Docker CLI plugin today (for AWS ECS integration):
https://github.com/docker/ecs-plugin

At a first look it uses the classic Docker CLI specs (the README even has a broken link to the deleted docs page).

Question: Are the Docker CLI Plugin specs really deprecated or was that page maybe deleted by mistake?

If someone from the Docker docs maintainers reads this, it would be really nice to get a clarification.
( @usha-mandya @chris-crone )

(I'm asking this as a Docker user who has homebrewed some custom CLI plugins, like this one.).

[chris-crone]

Hi @christian-korneck!

The plugin mechanism is deprecated and we consider the plugin API unstable. We will either remove it or re-architecture in a future release of the CLI.

The ECS integration was done as a plugin to decouple internal development work. We will be pulling it into the CLI in the near future so that the UX is the same as that of the ACI integration.

[mehyedes]

Right when I started developing a new Docker plugin 😞

[christian-korneck]

@chris-crone : Thanks. That's sad to hear. Is there anything that CLI plugin developers can do to avoid that their plugins just stop working with a future Docker version? Is there any github issue that I could read to better understand this deprecation?

(I had planned to cleanup and open source more of my homebrewed/inhouse CLI plugins, but probably won't do now as I don't want to encourage others to use something that might become suddenly broken).

[chris-crone]

Is there anything that CLI plugin developers can do to avoid that their plugins just stop working with a future Docker version?

The two ways forward are either: (1) to upstream the changes or (2) to create a standalone tool.

Is there any github issue that I could read to better understand this deprecation?

I don't think we opened an external issue for this. The decision was based on us not wanting to fragment the Docker CLI UX. We want users of the CLI to have the same experience and not be confused that some commands work in some places but not others (i.e.: when you do/don't have the plugin installed).

[christian-korneck]

hi @chris-crone

Many thanks for taking the time to clarify this topic. I appreciate it a lot. As you say this was an internal decision I'm not sure if feedback about this topic is welcome? Hope it's ok if I still give some, here it is:

The two ways forward are either: (1) to upstream the changes or (2) to create a standalone tool.

TLDR: I think the removal of support for Docker CLI plugins will leave a gap for use cases that are in the middle of these two extremes: Extensions of the CLI created with little effort by users (not core project developers) that usually solve small day to day problems and provide functionality that wouldn't work without the CLI host application.

In more detail:

either: (1) to upstream

As a Docker user (not developer, maybe poweruser) I find the current CLI plugin interface perfect for quickly and flexibly adding functionality to the docker command without having to fork and hack or wrap the full docker/cli project (with all the disadvantages that come with running a fork in production. Think: Enterprise workstation deployments, ci runners, etc). With "flexibly" I mean you can you can create a CLI plugin as shell script or in any language that fits, you're not bound to Golang.

My CLI plugins all started as shell scripts to solve day to day problems and when I needed a particular one to be more robust and/or portable I rewrote it in Golang.

I think that's not an uncommon way of how to extend a CLI host application. Not only with the Docker CLI: For example the kubectl CLI docs encourages users to extend kubectl with shell scripts. It's a pattern. And it is great that it was possible that I could apply it to Docker so far.

The ECS integration was done as a plugin to decouple internal development work.

isn't that a good example for why CLI plugins are useful? 😉

(2) to create a standalone tool

That's an option of course. Users then would need to run docker-mycommand instead of docker mycommand. But IMHO it's about more than just that dash in the command. I think there's a clear distinction between what should be a standalone tool or a CLI plugin (again this can be applied to all extendable CLIs like docker, kubectl, etc):

If the tool's core functionality depends on the existence of interfaces that the host software provides it should (imho) be a CLI plugin.
In case of Docker that's when the tool needs to talk to the Docker engine with docker system dial-stdio, access the Docker credentials store, etc.

(This dependency of the plugin to the host application (potentially in a specific version like >=19.03) can also be nicely reflected in package managers).

The decision was based on us not wanting to fragment the Docker CLI UX

I wouldn't expect user-created plugins to be a threat for Docker CLI consistency. With a CLI plugin you can't overwrite an existing command. When the CLI host introduces a new subcommand and an existing plugin by bad luck is using the same name then the plugin needs to be renamed. (Did that ever happen?)

I'm not sure how many users are out there who are using Docker CLI plugins (maybe you have telemetry data?). But I'll miss this interface for sure. 🙁

[chris-crone]

This is good feedback, thanks for taking the time to write it here @christian-korneck!

For the core functionality problem, Docker should help solve this but can do so with things like SDKs for access to credentials/sockets so that developers (of all stripes) can build on top of Docker. This approach would also allow non-CLI apps to provide functionality.

In terms of UX consistency, we have seen fragmentation internally that we want to avoid. Examples include Docker App's listing of apps vs docker image ls which is caused by plugins not being able to overwrite existing functionality. Another form of inconsistency comes from having different languages doing things like flag parsing.

Thank you again for the feedback, I have shared it internally so that it helps drive what we do next.

[SvenDowideit]

As I mentioned in the PR - I think the saddest thing is that there was no discussion of this in GitHub.

Removing the docs before the release that deprecates the API (and not just amending it to say that its deprecated, and then removing it when the code is removed) also seems hasty - as there are some of us that need to continue supporting our users, and debugging plugins

I guess https://github.com/SvenDowideit/docker-cli-plugins is still useful tho.

[kriansa]

@chris-crone Compose V2 has been recently released as a plugin. Does it mean you are revisiting that decision, or the intention is to upstream Compose?

Maybe @ndeloof or @ulyssessouza could chime in as well?

[chris-crone]

Hi @kriansa

We are still deciding exactly what we want to do for the CLI architecture for the long term.

Internally, we find plugins useful for separating codebases for the various Docker components and iterating quickly on them but plugins cause UX inconsistencies and complicate packaging. For external users, we think that CLI features should be contributed to docker/cli or, if they're not relevant to Docker users, be provided as a standalone tool outside of Docker.

As such, CLI plugins are considered an unstable Docker internal API for organizing components. We do not intend that the plugin experience of writing 'docker buildx' is exposed to users and this is the first step towards removing it, simplifying the build experience, and merging this more closely into the CLI.