Kubernetes orchestrator all namespaces #1059

mat007 · 2018-05-15T15:01:37Z

- What I did

I made --orchestrator=all behave as if also --all-namespaces unless specific namespaces have been queried with --namespace

- How I did it

Opts.AllNamespaces gets set to true if requesting for all orchestrators without specifying a namespace

- Description for the changelog

Setting --orchestrator=all also sets --all-namespaces unless specific --namespace are set.

- A picture of a cute animal (not mandatory but encouraged)

mat007 · 2018-05-15T15:02:03Z

Split from #1031, only the last commit is new.

mat007 · 2018-05-16T08:38:12Z

For reference, here is the current state of the discussion from #1031

@thaJeztah

I still have reservations on the magic relation between --orchestrator=all and --all-namespaces (copying my earlier comment #1031 (comment) here, as it was now hidden);

I'm having trouble conflating these; this may become very confusing:

configuration has "orchestrator: all": docker stack ls shows all stacks (both swarm and k8s)

I want to view only the k8s stacks, so I add --orchestrator=kubernetes. Now I no longer see all my stacks, only those in the default namespace

confused

@chris-crone

The thinking behind this was as follows: When you list stacks across orchestrators your intention is to see all the stacks running on your cluster. When you specify kubernetes as your orchestrator; you are doing something more specific and want similar behaviour to what kubectl gives you.

I can see how this might be confusing for your flow. There will be a NAMESPACE column which should help the user understand what has happened.

@chris-crone

Ah, see you're referring to the case when a user has set orchestrator to all in their config file. I don't think many users will do this as they will only be able to list resources. In order to manipulate resources, they will need to specify an orchestrator.

I don't think many users will do this as they will only be able to list resources.

@thaJeztah

I'm actually not sure: I can definitely see it being useful to just run any docker list or docker ps command, and see everything I have running, then use --filter and/or --orchestrator=x to limit the list to what I'm looking for, or (when deploying / interacting with objects), pass --orchestrator=foo --namespace=bar as part of the steps to deploy.

Which brings me to two things to consider:

Add orchestrator and namespace as filter options (--filter orchestrator=swarm) (no high-priority, but for consistency)

When interacting with existing objects, only require --orchestrator and/or --namespace if the object refered to is ambiguous, for example, if there's only one stack with a given name, there would be no need to ask for --orchestrator.

thaJeztah · 2018-05-16T13:08:54Z

So

connecting orchestrator=all with --all-namespaces means that
- --orchestrator=all: show all swarm objects and all kubernetes objects, in all namespaces
- switching to --orchestrator=swarm: shows all swarm objects from the list above
- switching to --orchestrator=kubernetes: show only kubernetes objects if they're in the "default" namespace (this is the confusing bit)

Thoughts: we could;

special-case the "default" namespace, and consider it (for list-views) equivalent of "no namespace set", in which case we always show "all namespaces", unless user manually passes a namespace --namespace=foo
irregardless of the configured namespace, default to show all namespaces for kubernetes (in list views), unless the user manually passes a namespace --namespace=foo (this one is tricky, because we then ignore the user's preference)
add a configuration option to enable --all-namespaces by default (or, the reverse: allow opt-out and dissable the default --all-namespaces)
add a configuration option to specify namespaces to use: ("namespaces":["all"] or "namespaces":["myteam-stage", "myteam-prod"])
combination of 3. and 4. (to prevent the "all" magic value), in which case "namespaces" defaults to the "default" namespace (as configured by the user), and --all-namespaces either enabled by default or not (whatever we decide on: opt-in/opt-out)

I'm leaning towards 4. (or 3.), as it keeps the user in control;

Allows to see all namespaces by default, both for --orchestrator=all and --orchestrator=kubernetes, which may be a use-case if the user manages multiple namespaces (and orchestrators)
Allows to only see namespaces that the user is working on (which could be "some" of the ones they have access to), both when viewing kubernetes-only and when viewing "all" orchestrators
No magic/unexpected behavior

silvin-lubecki · 2018-05-24T16:31:46Z

cc @thaJeztah @vdemeester @mat007 I'm opened to any comment on the config file option 😄
I took the third option among those proposed by @thaJeztah

silvin-lubecki · 2018-05-25T09:37:09Z

Some details on the UX:

# Use kubernetes as default orchestrator
$ cat ~/.docker/config.json
{
  "experimental" : "enabled",
  "orchestrator" : "kubernetes"
}

$ docker version
...
 Orchestrator: kubernetes
...

# Check there is no stack
$ docker stack ls
NAME                SERVICES            ORCHESTRATOR

# Deploy a stack on the default namespace, optionally defined in the current context in your ~/.kube/config file 
$ docker stack deploy mystack -c docker-compose.yml
...

# Add a new namespace
$ kubectl create namespace mynamespace
namespace "mynamespace" created

# And deploy another stack in it
$ docker stack deploy mystack -c docker-compose.yml --namespace mynamespace
...

# By default, list all stacks in all namespaces
$ docker stack ls
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default
mystack             5                   Kubernetes          mynamespace

# List can be restricted to one or multiple namespaces
$ docker stack ls --namespace default
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default

$ docker stack ls --namespace mynamespace --namespace default                                                                        NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default
mystack             5                   Kubernetes          mynamespace

# Deploy a stack on Swarm
$ docker stack deploy mystack -c docker-compose.yml --orchestrator swarm
...

$ docker stack ls --orchestrator swarm
NAME                SERVICES            ORCHESTRATOR
mystack             5                   Swarm

# Listing on all orchestrators also lists all namespaces
$ docker stack ls --orchestrator all
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default
mystack             5                   Kubernetes          mynamespace
mystack             5                   Swarm

# Listing all namespaces behavior by default can be disabled in the docker config file
$ cat ~/.docker/config.json
{
  "experimental" : "enabled",
  "orchestrator" : "kubernetes",
  "kubernetes": {
    "allNamespaces" : "disabled"
  }
}

# Now it lists using the default configured namespace
$ docker stack ls
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default

# List can be extended to one or multiple namespaces
$ docker stack ls --namespace mynamespace
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          mynamespace

$ docker stack ls --namespace mynamespace --namespace default                                                                        NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default
mystack             5                   Kubernetes          mynamespace

# Listing all namespaces is now explicit
$ docker stack ls --all-namespaces
NAME                SERVICES            ORCHESTRATOR        NAMESPACE
mystack             5                   Kubernetes          default
mystack             5                   Kubernetes          mynamespace

Updated

Renaming the config field to kubernetes/allNamespaces
Removing the docker stack ls listing the first stack, as it doesn't show the default all-namespaces feature

vdemeester · 2018-05-25T10:08:22Z

cli/config/configfile/file.go

+	Proxies                map[string]ProxyConfig      `json:"proxies,omitempty"`
+	Experimental           string                      `json:"experimental,omitempty"`
+	Orchestrator           string                      `json:"orchestrator,omitempty"`
+	StackListAllNamespaces string                      `json:"stackListAllNamespaces,omitempty"`


I wonder if we should start to using some nested struct, like

{ "orchestrator": "kubernetes", "stack": { "list": { "allNamespaces": "disabled", "format": "…", } } }

cc @thaJeztah

Why not, but maybe in a follow up?

Will we only use this configuration for stacks, or also for docker service ls ? (wondering about the Stack prefix.

Alternatively, if we want to use a boolean instead of the disabled string, we could use {"singleNamespace": true}

Ok so what about

{ "kubernetes": { "singleNamespace": true } }

And by the way, singleNamespace makes you think you can only list one namespace, but you still can use multiple --namespace in the docker stack ls command, so it could be a little confusing?

@thaJeztah hum good point.
I tend to agree with @silvin-lubecki on the singleNamespace name… but nothing really pop in my head for a correct name.

Yes, good point; I'd say:

use "allNamespaces": "disabled"

also accept "allNamespaces": "enabled" for those that want to be explicit

if no option is set in the configuration, "allNamespaces": "enabled" is the default

SGTY?

thaJeztah · 2018-05-28T08:28:26Z

Looks like there's a linting failure;

cli/config/configfile/file.go:60:6:warning: exported type KubernetesConfig should have comment or be unexported (golint)
Exited with code 1

silvin-lubecki · 2018-05-28T08:39:22Z

Indeed 😔

…or is all or Kubernetes * Add "kubernetes" struct in config file with "allNamespaces" option, to opt-out this behavior when set as "disabled" Signed-off-by: Mathieu Champlon <[email protected]>

Signed-off-by: Silvin Lubecki <[email protected]>

silvin-lubecki · 2018-05-28T08:56:55Z

PTAL 😄

vdemeester

LGTM 🐯

thaJeztah

LGTM

As a follow-up we need to update the CLI configuration documentation https://github.com/docker/cli/blob/master/docs/reference/commandline/cli.md#configuration-files

Unfortunately, that one is quite messy, and in need of a proper rewrite (with actual examples 😞)

thaJeztah · 2018-05-28T10:01:06Z

cli/command/stack/kubernetes/list.go

 }

+func isAllNamespacesDisabled(kubeCliConfig *configfile.KubernetesConfig) bool {
+	return kubeCliConfig == nil || kubeCliConfig != nil && kubeCliConfig.AllNamespaces != "disabled"


Oh, one thing: do we want a strict check to only accept "disabled", "enabled" or nil ?

yes, @silvin-lubecki will follow up 👼

Sure I will!

silvin-lubecki · 2018-05-28T10:16:49Z

Is this the only documentation we have about the CLI configuration? Nothing else on docs.docker.com?

thaJeztah · 2018-05-28T10:54:23Z

We may need some more in general (about working with multiple orchestrators); not sure if "orchestrator" was already documented in the configuration; also some flags/options are currently experimental, so we'll have to do a "sweep" after they leave experimental, so see if everything is there.

Would be good to have a tracking issue in which we can add things that we think about (could be a checkbox-list style)

silvin-lubecki · 2018-05-28T11:07:56Z

Not yet in the documentation as it was behind the experimental feature flag.

mat007 requested a review from vdemeester as a code owner May 15, 2018 15:01

GordonTheTurtle added the status/0-triage label May 15, 2018

thaJeztah added status/1-design-review area/kubernetes and removed status/0-triage labels May 15, 2018

chris-crone mentioned this pull request May 16, 2018

Better stack list experience with Kubernetes and Swarm #1031

Merged

silvin-lubecki mentioned this pull request May 18, 2018

Remove Kubernetes commands from experimental #1068

Merged

vdemeester reviewed May 25, 2018

View reviewed changes

mat007 and others added 2 commits May 28, 2018 10:43

Imply all Kubernetes namespaces for docker stack list when orchestrat…

1c1300b

…or is all or Kubernetes * Add "kubernetes" struct in config file with "allNamespaces" option, to opt-out this behavior when set as "disabled" Signed-off-by: Mathieu Champlon <[email protected]>

--orchestrator flag is now a persistent flag

770daef

Signed-off-by: Silvin Lubecki <[email protected]>

vdemeester approved these changes May 28, 2018

View reviewed changes

thaJeztah added status/2-code-review and removed status/1-design-review labels May 28, 2018

thaJeztah approved these changes May 28, 2018

View reviewed changes

thaJeztah added impact/changelog docs/revisit labels May 28, 2018

vdemeester merged commit 8205323 into docker:master May 28, 2018

GordonTheTurtle added this to the 18.06.0 milestone May 28, 2018

thaJeztah reviewed May 28, 2018

View reviewed changes

silvin-lubecki mentioned this pull request May 28, 2018

Complete documentation on CLI configuration file #1086

Open

3 tasks

Kubernetes orchestrator all namespaces #1059

Kubernetes orchestrator all namespaces #1059

Uh oh!

Conversation

mat007 commented May 15, 2018

Uh oh!

mat007 commented May 15, 2018

Uh oh!

mat007 commented May 16, 2018

Uh oh!

thaJeztah commented May 16, 2018

Uh oh!

silvin-lubecki commented May 24, 2018

Uh oh!

silvin-lubecki commented May 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented May 28, 2018

Uh oh!

silvin-lubecki commented May 28, 2018

Uh oh!

silvin-lubecki commented May 28, 2018

Uh oh!

vdemeester left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

silvin-lubecki commented May 28, 2018

Uh oh!

thaJeztah commented May 28, 2018

Uh oh!

silvin-lubecki commented May 28, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

silvin-lubecki commented May 25, 2018 •

edited

Loading