kubernetes rbac rules (still) not enforced

I have tried with the latest version of my channel (Stable or Edge)
I have uploaded Diagnostics

This is a refresh of #3694, as that issue was believed to be fixed but actually still exists in 2.3.0.3 (45519). The old issue has been locked due to age.

Run the following:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: test-sa

---

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  serviceAccountName: test-sa
  containers:
  - image: alpine
    name: test-container
    command: [sh]
    args:
    - -ec
    - |
      apk add curl;
      KUBE_NAMESPACE="$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)";
      curl \
        --cacert "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" \
        -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
        "https://kubernetes.default.svc/api/v1/namespaces/$KUBE_NAMESPACE/services";
      while true; do sleep 1; done;

kubectl apply -f test.yml
kubectl logs test-pod

Expected behavior

Permissions error (403)

Actual behavior

API request succeeds

Information

kubectl get clusterrolebinding docker-for-desktop-binding -o yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: "2020-06-22T13:01:34Z"
  name: docker-for-desktop-binding
  resourceVersion: "478"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/docker-for-desktop-binding
  uid: f4d43180-d35e-448c-ba41-db0a1330baff
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts
  namespace: kube-system

kubectl auth can-i --list --as system:serviceaccount

Resources                                       Non-Resource URLs   Resource Names   Verbs
*.*                                             []                  []               [*]
                                                [*]                 []               [*]
selfsubjectaccessreviews.authorization.k8s.io   []                  []               [create]
selfsubjectrulesreviews.authorization.k8s.io    []                  []               [create]
                                                [/api/*]            []               [get]
                                                [/api]              []               [get]
                                                [/apis/*]           []               [get]
                                                [/apis]             []               [get]
                                                [/healthz]          []               [get]
                                                [/healthz]          []               [get]
                                                [/livez]            []               [get]
                                                [/livez]            []               [get]
                                                [/openapi/*]        []               [get]
                                                [/openapi]          []               [get]
                                                [/readyz]           []               [get]
                                                [/readyz]           []               [get]
                                                [/version/]         []               [get]
                                                [/version/]         []               [get]
                                                [/version]          []               [get]
                                                [/version]          []               [get]

rnsv posted the necessary fix in the old issue here: #3694 (comment) (the namespace was applied to the wrong part of the spec)

Diagnostic logs

Docker for Mac: 2.3.0.3 (45519)

In the meantime, users can patch this issue on their own deploys by running:

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: docker-for-desktop-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:kube-system
EOF

(it will complain about "kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply" but it will work. I don't know if it will cause problems with docker-desktop upgrades in the future)

A patch might be a better workaround - it will fail if the binding doesn't exist:

 kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch $'[{"op":"replace", "path":"/subjects/0/name", "value":"system:serviceaccounts:kube-system"}]'

or in Powershell which needs the quotes to be escaped:

kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch '[{\"op\":\"replace\", \"path\":\"/subjects/0/name\", \"value\":\"system:serviceaccounts:kube-system\"}]'

@nebuk89

@nebuk89 this is quite an important issue, currently it gives all Service Accounts cluster admin role which pretty much defeats RBAC and makes Kube in Desktop (Windows and Mac) behave unlike every other distro :)

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

I have just tested this in 2.4.0.0 (48506) and the issue still exists, so I'm marking this not stale.

For users, the workaround posted by sixeyed seems to be the best option:

kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch $'[{"op":"replace", "path":"/subjects/0/name", "value":"system:serviceaccounts:kube-system"}]'

Alternatively I've found that minikube is pretty easy to get going with and seems to create a system which is a bit closer to a "real" kubernetes environment.

/remove-lifecycle stale

Still exists in Desktop 3.0.0

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

/remove-lifecycle stale

Still exists in 3.2.2

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@guillaumerose

maybe it's worth pinging @guillaumerose on this one since it keeps going stale and he previously asked people to let him know if this issue continued after the original attempted fix.

/remove-lifecycle stale

just hit this same issue and was pulling my hair out trying to see why... any update Docker ?

This "feature" does make teaching others about RBAC in k8s a little interesting for sure! 😆 While it's easy enough to nuke the binding, I'd love to hear why it's there in the first place. Should it be scoped to a specific namespace?

@thaJeztah

Pinging @thaJeztah - maybe he won't know what to do, but he will know who knows ♥

(I need to refresh my look-up table of Docker Inc. folks so that I know better who to ping for that kind of issue 😅)

Short recap:

Docker Desktop has a ClusterRoleBinding called docker-for-desktop-binding that gives cluster-admin privileges to all ServiceAccounts
this means that any Pod running on Docker Desktop has cluster-admin privileges
it should probably be changed to give these privileges only to the ServiceAccount kube-system:default (or even removed completely)

This behavior is present at least until Docker Desktop 4.1.1 (69879).

Slightly off-topic but as a first layer of restrictions you could opt out of automounting API credentials for a particular pod via automountServiceAccountToken directive ( https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ ), especially if your pods don't need to communicate with the control plane

This is painful, please fix it.

@chrisdoherty4

@chrisdoherty4 Hi! Can you delete the clusterrolebinding and report if there is any side effect? That would help us all to figure out what the best solution would be. Thank you!

@jpetazzo

@jpetazzo Done and it performs well. I can correctly define RBAC and kubectl auth can-i spits out the expected result (I ran various tests mixing and matching stuff).

I didn't explicitly try ClusterRole's but I assume it functions fine also.

I believe the binding should not be installed any more in development builds. If you have time to check, here is the latest one: https://desktop-stage.docker.com/mac/main/amd64/71478/Docker.dmg (replace amd64 with arm64 if you are using an M1). I don't think it will remove a binding that already exists-- it will probably only take effect when the cluster is next initialised.

@thaJeztah

Pinging @thaJeztah - maybe he won't know what to do, but he will know who knows ♥

Oh! Sorry, looks like I didn't reply on this thread. I see it's being worked on though, so 👍 ❤️

@djs55

@djs55 Awesome work. Thanks!

New release 4.3.0 (https://docs.docker.com/desktop/mac/release-notes/) is out and has a fix for this issue

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

Navigation Menu

Repository navigation

kubernetes rbac rules (still) not enforced #4774

Expected behavior

Actual behavior

Information

Diagnostic logs

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

kubernetes rbac rules (still) not enforced #4774

Description

Expected behavior

Actual behavior

Information

Diagnostic logs

Activity

davidje13 commented on Jul 14, 2020

sixeyed commented on Jul 22, 2020

sixeyed commented on Jul 22, 2020

docker-robott commented on Oct 20, 2020

davidje13 commented on Nov 1, 2020

sixeyed commented on Dec 16, 2020

docker-robott commented on Mar 16, 2021

sixeyed commented on Apr 13, 2021

docker-robott commented on Jul 12, 2021

davidje13 commented on Jul 12, 2021

6 remaining items

ncresswell commented on Oct 25, 2021

mikesir87 commented on Oct 28, 2021

jpetazzo commented on Nov 5, 2021

mstaicu commented on Nov 17, 2021

chrisdoherty4 commented on Nov 24, 2021

jpetazzo commented on Nov 25, 2021

chrisdoherty4 commented on Nov 25, 2021

djs55 commented on Nov 25, 2021

thaJeztah commented on Nov 29, 2021

chrisdoherty4 commented on Nov 29, 2021

trungutt commented on Dec 2, 2021

docker-robott commented on Jan 1, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions