[Request]: k8s plugin

[jshi991]

Feature or enhancement request details

Feature Request: k8s plugin — local Kubernetes cluster support

Important

This is an experimental feature. CLI options and configuration API will be unstable as the plugin evolves.

Summary

Add a k8s plugin to the container CLI that lets users spin up and manage local Kubernetes development clusters. Cluster nodes run upstream Kubernetes bootstrapped via kubeadm, using the kindest/node image from the KIND project as the base, running directly on the Apple container runtime.

Motivation

Local Kubernetes development on Apple silicon today requires third-party tools that either ship their own VMs or rely on Docker. Since container already manages lightweight Linux containers natively on macOS, it is a natural fit for running a local k8s cluster with minimal overhead and no extra dependencies.

Proposed UX

Initial feature set — create, start, delete, load-image, write-config, list

The initial feature set covers the core day-to-day workflow: boot a single-node cluster, load images into it, point kubectl at it, and tear it down when finished.

Node image — The plugin uses kindest/node as the cluster node image. This image ships with kubeadm, kubelet, kubectl, and containerd pre-installed. On first use the plugin pulls the image automatically if it is not already present.

create — Creates and starts a single control-plane cluster. The plugin pulls the node image if needed, boots the container, runs a prep script to configure the native containerd snapshotter and required sysctl values, runs kubeadm init, applies the kindnet CNI, removes the control-plane taint so pods can schedule on the single node, and merges a kubeconfig entry into ~/.kube/config automatically.

container k8s create --name my-cluster

start — Starts a previously stopped cluster and re-merges the kubeconfig into ~/.kube/config, updating the API server address in case the container IP changed on restart.

container k8s start --name my-cluster

delete — Stops and removes the cluster and cleans up its entries from ~/.kube/config.

container k8s delete --name my-cluster

load-image — Loads a locally-stored image into the cluster's containerd so kubelet can schedule pods using it without an external registry. The image is saved from the container image store to a temporary OCI tar and imported into the node via ctr images import. Short references (alpine) are automatically qualified to docker.io/library/alpine:latest and also tagged back under the short name so kubelet can find them. An optional --platform flag controls which platform variant is selected from the local image store when the image has multiple architectures (default: host architecture).

container k8s load-image --name my-cluster my-app:latest
container k8s load-image --name my-cluster --platform linux/amd64 my-app:latest

write-config — Merges the cluster's kubeconfig into ~/.kube/config (or a custom path via --kubeconfig). Rewrites the API server address to 127.0.0.1:<host-port> or to the machine's FQDN when one is detected. Does not change current-context — use kubectl config use-context to switch contexts.

container k8s write-config --name my-cluster
container k8s write-config --name my-cluster --kubeconfig ~/.kube/my-cluster.kubeconfig

list — Lists all clusters and their nodes in a table, discovered via the plugin=k8s container label.

container k8s list
# CLUSTER     NODE        ROLE           STATE    CPUS  MEMORY   ADDR          PORTS
# my-cluster  my-cluster  control-plane  running  4     4096 MB  192.168.64.5  6445->6443

Architecture

All components run on macOS. The container k8s CLI drives the container service to create, start, and exec into the cluster node container. Images are transferred out-of-band: the CLI reads from the container image store and streams an OCI tar into the node's containerd (k8s.io namespace). kubectl reaches the API server through a published host port (6445 → 6443). write-config merges credentials directly into ~/.kube/config.

Future Work

API support — Cluster management API calls use a well defined K8sClusterConfiguration object for creation, and a K8sClusterResource conforming to ManagedResource for existing clusters. k8s create accepts a --config config-path options as well as existing cluster configuration options.

Multi-worker clusters — create will accept a --workers N flag to start N worker nodes alongside the control plane. Each worker joins using a token fetched from the control plane. load-image will target all nodes by default so every kubelet can schedule pods regardless of placement.

HA control plane — create accepts a --control-planes N flag to start N control plane nodes, with a load balancing implementation that provides a single HA cluster API endpoint.

Service load balancing — Allow clusters to expose services through a host-accessible endpoint without manual port-forwarding, designed to map cleanly to the container networking model.

Registries — Let users point a cluster at a local registry (e.g. one started with container run) so that images pushed there are immediately pullable inside the cluster without running load-image manually.

Alternate cluster providers — Refine cluster and node provisioning into solid abstractions, so that nodes can be created and joined using distributions other than kind.

Questions for the Community

1. Is this something you'd use? Would this be valuable to your workflow?
2. What features are must-haves?
3. What use cases would this unlock for you?

---

Please share your thoughts and feedback. We want to hear from you!

Code of Conduct

• I agree to follow this project's Code of Conduct

[OneCricketeer]

Should the control be inverted? E.g Rather than "limiting" containers to use KIND (vs k3d or other), then KIND can optionally use container API - kubernetes-sigs/kind#3958

[jshi991]

Should the control be inverted? E.g Rather than "limiting" containers to use KIND (vs k3d or other), then KIND can optionally use container API - kubernetes-sigs/kind#3958

currently the project scope is in this direction of first giving the CLI to use kind to orchestrate containers however an api is planned in the future

[toonsevrin]

Is there any plan to support k3s as well for people who want to run a persistent, upgradable cluster on their machine?

[mrkoil-web]

Feature or enhancement request details

Feature Request: k8s plugin — local Kubernetes cluster support

[!IMPORTANT]

This is an experimental feature. CLI options and configuration API will be unstable as the plugin evolves.

Summary

Add a k8s plugin to the container CLI that lets users spin up and manage local Kubernetes development clusters. Cluster nodes run upstream Kubernetes bootstrapped via kubeadm, using the kindest/node image from the KIND project as the base, running directly on the Apple container runtime.

Motivation

Local Kubernetes development on Apple silicon today requires third-party tools that either ship their own VMs or rely on Docker. Since container already manages lightweight Linux containers natively on macOS, it is a natural fit for running a local k8s cluster with minimal overhead and no extra dependencies.

Proposed UX

Initial feature set — create, start, delete, load-image, write-config, list

The initial feature set covers the core day-to-day workflow: boot a single-node cluster, load images into it, point kubectl at it, and tear it down when finished.

Node image — The plugin uses kindest/node as the cluster node image. This image ships with kubeadm, kubelet, kubectl, and containerd pre-installed. On first use the plugin pulls the image automatically if it is not already present.

create — Creates and starts a single control-plane cluster. The plugin pulls the node image if needed, boots the container, runs a prep script to configure the native containerd snapshotter and required sysctl values, runs kubeadm init, applies the kindnet CNI, removes the control-plane taint so pods can schedule on the single node, and merges a kubeconfig entry into ~/.kube/config automatically.

container k8s create --name my-cluster

start — Starts a previously stopped cluster and re-merges the kubeconfig into ~/.kube/config, updating the API server address in case the container IP changed on restart.

container k8s start --name my-cluster

delete — Stops and removes the cluster and cleans up its entries from ~/.kube/config.

container k8s delete --name my-cluster

load-image — Loads a locally-stored image into the cluster's containerd so kubelet can schedule pods using it without an external registry. The image is saved from the container image store to a temporary OCI tar and imported into the node via ctr images import. Short references (alpine) are automatically qualified to docker.io/library/alpine:latest and also tagged back under the short name so kubelet can find them. An optional --platform flag controls which platform variant is selected from the local image store when the image has multiple architectures (default: host architecture).

container k8s load-image --name my-cluster my-app:latest
container k8s load-image --name my-cluster --platform linux/amd64 my-app:latest

write-config — Merges the cluster's kubeconfig into ~/.kube/config (or a custom path via --kubeconfig). Rewrites the API server address to 127.0.0.1:<host-port> or to the machine's FQDN when one is detected. Does not change current-context — use kubectl config use-context to switch contexts.

container k8s write-config --name my-cluster
container k8s write-config --name my-cluster --kubeconfig ~/.kube/my-cluster.kubeconfig

list — Lists all clusters and their nodes in a table, discovered via the plugin=k8s container label.

container k8s list
# CLUSTER     NODE        ROLE           STATE    CPUS  MEMORY   ADDR          PORTS
# my-cluster  my-cluster  control-plane  running  4     4096 MB  192.168.64.5  6445->6443

Architecture

All components run on macOS. The container k8s CLI drives the container service to create, start, and exec into the cluster node container. Images are transferred out-of-band: the CLI reads from the container image store and streams an OCI tar into the node's containerd (k8s.io namespace). kubectl reaches the API server through a published host port (6445 → 6443). write-config merges credentials directly into ~/.kube/config.

Future Work

API support — Cluster management API calls use a well defined K8sClusterConfiguration object for creation, and a K8sClusterResource conforming to ManagedResource for existing clusters. k8s create accepts a --config config-path options as well as existing cluster configuration options.

Multi-worker clusters — create will accept a --workers N flag to start N worker nodes alongside the control plane. Each worker joins using a token fetched from the control plane. load-image will target all nodes by default so every kubelet can schedule pods regardless of placement.

HA control plane — create accepts a --control-planes N flag to start N control plane nodes, with a load balancing implementation that provides a single HA cluster API endpoint.

Service load balancing — Allow clusters to expose services through a host-accessible endpoint without manual port-forwarding, designed to map cleanly to the container networking model.

Registries — Let users point a cluster at a local registry (e.g. one started with container run) so that images pushed there are immediately pullable inside the cluster without running load-image manually.

Alternate cluster providers — Refine cluster and node provisioning into solid abstractions, so that nodes can be created and joined using distributions other than kind.

Questions for the Community

1. Is this something you'd use? Would this be valuable to your workflow?
2. What features are must-haves?
3. What use cases would this unlock for you?

---

Please share your thoughts and feedback. We want to hear from you!

Code of Conduct

• I agree to follow this project's Code of Conduct

Off