Skip to content

Comments

add zstd support to squashfs (release 3732)#1296

Merged
t-lo merged 2 commits intoflatcar-3732from
t-lo/3732-add-squashfs-zstd
Oct 23, 2023
Merged

add zstd support to squashfs (release 3732)#1296
t-lo merged 2 commits intoflatcar-3732from
t-lo/3732-add-squashfs-zstd

Conversation

@t-lo
Copy link
Member

@t-lo t-lo commented Oct 23, 2023

No description provided.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
@t-lo t-lo merged commit 48e903f into flatcar-3732 Oct 23, 2023
@t-lo t-lo deleted the t-lo/3732-add-squashfs-zstd branch October 23, 2023 14:07
@github-actions
Copy link

Test report for 3732.1.0+nightly-20231020-2100 / amd64 arm64

Platforms tested : qemu_uefi-amd64 qemu_update-amd64 qemu_uefi-arm64 qemu_update-arm64

ok bpf.execsnoop 🟢 Succeeded: qemu_uefi-amd64 (1)

ok bpf.local-gadget 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cgroupv1 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.multipart-mime 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.cloudinit.script 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid0.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid0.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid1.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.disk.raid1.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.discovery 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.etcdctlv3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.etcd-member.v2-backup-restore 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.filesystem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.flannel.udp 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.flannel.vxlan 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.instantiated.enable-unit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.kargs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.luks 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.indirect 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.indirect.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.regular 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.regular.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.reuse 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.oem.wipe 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.symlink 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.translation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v1.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.ext4checkexisting 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.swap 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.ignition.v2_1.vfat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.install.cloudinit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.internet 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.locksmith.cluster 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.misc.falco 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.network.initramfs.second-boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.network.listeners 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.network.wireguard 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.omaha.ping 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.osreset.ignition-rerun 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.overlay.cleanup 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.swap_activation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.sysext.boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.sysext.fallbackdownload # SKIP 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.toolbox.dnf-install 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.update.badverity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.update.grubnop 🟢 Succeeded: qemu_uefi-amd64 (1)

ok cl.update.payload 🟢 Succeeded: qemu_update-amd64 (1); qemu_update-arm64 (1)

ok cl.update.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.users.shells 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok cl.verity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.auth.verify 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.local 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.resource.s3.versioned 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.security.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.ignition.systemd.enable-service 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.locksmith.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.locksmith.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.selinux.boolean 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.selinux.enforce 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.tls.fetch-urls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok coreos.update.badusr 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok devcontainer.docker 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok devcontainer.systemd-nspawn 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.btrfs-storage 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.containerd-restart 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.enable-service.torcx 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.lib-coreos-dockerd-compat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.network 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.selinux 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.torcx-manifest-pkgs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok docker.userns 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok extra-test.[first_dual].cl.update.oem 🟢 Succeeded: qemu_update-amd64 (1); qemu_update-arm64 (5) ❌ Failed: qemu_update-arm64 (1, 2, 3, 4)

                Diagnostic output for qemu_update-arm64, run 4
    L1: " Error: _cluster.go:125: Created symlink /etc/systemd/system/locksmithd.service ??? /dev/null."
    L2: "update.go:254: Triggering update_engine"
    L3: "update.go:273: Rebooting test machine"
    L4: "cluster.go:125: Warning: found hardcoded GROUP=developer in /etc/flatcar/update.conf - make sure it fits the release channel you want to follow"
    L5: "update.go:320: Rebooting test machine after flatcar-update run (2nd reboot)"
    L6: "update.go:322: reboot failed: machine __5753c7e0-76ce-4ac6-a3b3-853ea19c72b2__ failed basic checks: systemctl: : dial tcp 10.0.0.2:22: connect: connection refused:_"
    L7: " "
                Diagnostic output for qemu_update-arm64, run 3
    L1: " Error: _cluster.go:125: Created symlink /etc/systemd/system/locksmithd.service ??? /dev/null."
    L2: "update.go:254: Triggering update_engine"
    L3: "update.go:273: Rebooting test machine"
    L4: "cluster.go:125: Warning: found hardcoded GROUP=developer in /etc/flatcar/update.conf - make sure it fits the release channel you want to follow"
    L5: "update.go:320: Rebooting test machine after flatcar-update run (2nd reboot)"
    L6: "cluster.go:145: __rootdev -s /usr__ failed: output , status dial tcp 10.0.0.2:22: connect: connection refused_"
    L7: " "
                Diagnostic output for qemu_update-arm64, run 2
    L1: " Error: _cluster.go:125: Created symlink /etc/systemd/system/locksmithd.service ??? /dev/null."
    L2: "update.go:254: Triggering update_engine"
    L3: "update.go:273: Rebooting test machine"
    L4: "cluster.go:125: Warning: found hardcoded GROUP=developer in /etc/flatcar/update.conf - make sure it fits the release channel you want to follow"
    L5: "update.go:320: Rebooting test machine after flatcar-update run (2nd reboot)"
    L6: "cluster.go:145: __rootdev -s /usr__ failed: output , status dial tcp 10.0.0.2:22: connect: connection refused_"
    L7: " "
                Diagnostic output for qemu_update-arm64, run 1
    L1: "  "
    L2: " Error: _cluster.go:125: Created symlink /etc/systemd/system/locksmithd.service ??? /dev/null."
    L3: "update.go:254: Triggering update_engine"
    L4: "update.go:273: Rebooting test machine"
    L5: "cluster.go:125: Warning: found hardcoded GROUP=developer in /etc/flatcar/update.conf - make sure it fits the release channel you want to follow"
    L6: "update.go:320: Rebooting test machine after flatcar-update run (2nd reboot)"
    L7: "cluster.go:145: __test ! -e /oem/python/shouldbedeleted && test ! -e /etc/systemd/system/waagent.service__ failed: output , status dial tcp 10.0.0.2:22: connect: connection refused_"
    L8: " "

ok extra-test.[first_dual].cl.update.payload 🟢 Succeeded: qemu_update-amd64 (1); qemu_update-arm64 (1, 2, 3, 4, 5)

ok kubeadm.v1.25.10.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (3) ❌ Failed: qemu_uefi-arm64 (1, 2)

                Diagnostic output for qemu_uefi-arm64, run 2
    L1: " Error: _cluster.go:125: I1122 15:49:47.619415    1484 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.25"
    L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.25.16"
    L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.25.16"
    L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.25.16"
    L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.25.16"
    L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.8"
    L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.9.3"
    L9: "cluster.go:125: I1122 15:50:00.151982    1645 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.25"
    L10: "cluster.go:125: [init] Using Kubernetes version: v1.25.16"
    L11: "cluster.go:125: [preflight] Running pre-flight checks"
    L12: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:125: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:125: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.4?]"
    L19: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:125: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:125: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:125: [apiclient] All control plane components are healthy after 6.504253 seconds"
    L42: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:125: [bootstrap-token] Using token: 01iouq.zlk4317j08ynez0f"
    L48: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:125: "
    L58: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:125: "
    L60: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:125: "
    L62: "cluster.go:125:   mkdir -p $HOME/.kube"
    L63: "cluster.go:125:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:125:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:125: "
    L66: "cluster.go:125: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:125: "
    L68: "cluster.go:125:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:125: "
    L70: "cluster.go:125: You should now deploy a pod network to the cluster."
    L71: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:125:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:125: "
    L74: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:125: "
    L76: "cluster.go:125: kubeadm join 10.0.0.4:6443 --token 01iouq.zlk4317j08ynez0f _"
    L77: "cluster.go:125:  --discovery-token-ca-cert-hash sha256:8901fb49408517add3469db1faf898a31094b981f13f19827db30ebe211e6c25 "
    L78: "cluster.go:125: namespace/tigera-operator created"
    L79: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgpfilters.crd.projectcalico.org created"
    L81: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L82: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L83: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L84: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L85: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L86: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L87: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L88: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L89: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L90: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L91: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L92: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L93: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L94: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L95: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L96: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L97: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L98: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L99: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L100: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L101: "cluster.go:125: serviceaccount/tigera-operator created"
    L102: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:125: deployment.apps/tigera-operator created"
    L105: "cluster.go:125: error: .status.conditions accessor error: <nil_ is of the type <nil_, expected []interface{}"
    L106: "kubeadm.go:285: unable to setup cluster: unable to run master script: Process exited with status 1_"
    L107: " "
                Diagnostic output for qemu_uefi-arm64, run 1
    L1: "  "
    L2: " Error: _cluster.go:125: I1122 15:43:54.552169    1477 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.25"
    L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.25.16"
    L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.25.16"
    L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.25.16"
    L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.25.16"
    L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.8"
    L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L9: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.9.3"
    L10: "cluster.go:125: I1122 15:44:06.445931    1638 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.25"
    L11: "cluster.go:125: [init] Using Kubernetes version: v1.25.16"
    L12: "cluster.go:125: [preflight] Running pre-flight checks"
    L13: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:125: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L17: "cluster.go:125: [certs] Generating __ca__ certificate and key"
    L18: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
    L19: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?21]"
    L20: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L21: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
    L22: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
    L23: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L24: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L27: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L28: "cluster.go:125: [certs] Generating __sa__ key and public key"
    L29: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L30: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L31: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L32: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L33: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L34: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L35: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L36: "cluster.go:125: [kubelet-start] Starting the kubelet"
    L37: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L41: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L42: "cluster.go:125: [apiclient] All control plane components are healthy after 8.004611 seconds"
    L43: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L44: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L45: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
    L46: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L47: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L48: "cluster.go:125: [bootstrap-token] Using token: 2wzay5.m8xw1y8s939o9of9"
    L49: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L50: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L51: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L52: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L53: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L54: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L55: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L56: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
    L57: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
    L58: "cluster.go:125: "
    L59: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
    L60: "cluster.go:125: "
    L61: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
    L62: "cluster.go:125: "
    L63: "cluster.go:125:   mkdir -p $HOME/.kube"
    L64: "cluster.go:125:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L65: "cluster.go:125:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L66: "cluster.go:125: "
    L67: "cluster.go:125: Alternatively, if you are the root user, you can run:"
    L68: "cluster.go:125: "
    L69: "cluster.go:125:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L70: "cluster.go:125: "
    L71: "cluster.go:125: You should now deploy a pod network to the cluster."
    L72: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L73: "cluster.go:125:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L74: "cluster.go:125: "
    L75: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
    L76: "cluster.go:125: "
    L77: "cluster.go:125: kubeadm join 10.0.0.121:6443 --token 2wzay5.m8xw1y8s939o9of9 _"
    L78: "cluster.go:125:  --discovery-token-ca-cert-hash sha256:935fb67a5e0bbb7cd2f2dc4d095be14ea9859e34dcda3ccb61b684f16306b8ff "
    L79: "cluster.go:125: namespace/tigera-operator created"
    L80: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L81: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgpfilters.crd.projectcalico.org created"
    L82: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L83: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L84: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L85: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L86: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L87: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L88: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L89: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L90: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L91: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L92: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L93: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L94: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L95: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L96: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L97: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L98: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L99: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L100: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L102: "cluster.go:125: serviceaccount/tigera-operator created"
    L103: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L105: "cluster.go:125: deployment.apps/tigera-operator created"
    L106: "cluster.go:125: error: .status.conditions accessor error: <nil_ is of the type <nil_, expected []interface{}"
    L107: "kubeadm.go:285: unable to setup cluster: unable to run master script: Process exited with status 1_"
    L108: " "

ok kubeadm.v1.25.10.calico.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.cilium.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.25.10.flannel.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.26.5.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.27.2.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.27.2.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (2) ❌ Failed: qemu_uefi-arm64 (1)

                Diagnostic output for qemu_uefi-arm64, run 1
    L1: "  "
    L2: " Error: _cluster.go:125: I1122 15:40:21.481642    1443 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.27"
    L3: "cluster.go:125: W1122 15:40:21.586908    1443 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.8, falling back to the nearest etcd version (3.5.7-0)"
    L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.27.8"
    L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.27.8"
    L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.27.8"
    L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.27.8"
    L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.9"
    L9: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.5.7-0"
    L10: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.10.1"
    L11: "cluster.go:125: I1122 15:40:37.856133    1605 version.go:256] remote version is much newer: v1.28.4; falling back to: stable-1.27"
    L12: "cluster.go:125: [init] Using Kubernetes version: v1.27.8"
    L13: "cluster.go:125: [preflight] Running pre-flight checks"
    L14: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L15: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L16: "cluster.go:125: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L17: "cluster.go:125: W1122 15:40:38.129895    1605 checks.go:835] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended that using __registry.k8s.io/pause:3.9__ as the CRI sandbox image."
    L18: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L19: "cluster.go:125: [certs] Generating __ca__ certificate and key"
    L20: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
    L21: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.4?1]"
    L22: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L23: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
    L24: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
    L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L29: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L30: "cluster.go:125: [certs] Generating __sa__ key and public key"
    L31: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L32: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L33: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L34: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L35: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L36: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L37: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L38: "cluster.go:125: [kubelet-start] Starting the kubelet"
    L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L43: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L44: "cluster.go:125: [apiclient] All control plane components are healthy after 7.503905 seconds"
    L45: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L46: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L47: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
    L48: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L49: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L50: "cluster.go:125: [bootstrap-token] Using token: 398732.b46x4wh8pzo8u17q"
    L51: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L52: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L53: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L54: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L55: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L56: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L57: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L58: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
    L59: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
    L60: "cluster.go:125: "
    L61: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
    L62: "cluster.go:125: "
    L63: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
    L64: "cluster.go:125: "
    L65: "cluster.go:125:   mkdir -p $HOME/.kube"
    L66: "cluster.go:125:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L67: "cluster.go:125:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L68: "cluster.go:125: "
    L69: "cluster.go:125: Alternatively, if you are the root user, you can run:"
    L70: "cluster.go:125: "
    L71: "cluster.go:125:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L72: "cluster.go:125: "
    L73: "cluster.go:125: You should now deploy a pod network to the cluster."
    L74: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L75: "cluster.go:125:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L76: "cluster.go:125: "
    L77: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
    L78: "cluster.go:125: "
    L79: "cluster.go:125: kubeadm join 10.0.0.41:6443 --token 398732.b46x4wh8pzo8u17q _"
    L80: "cluster.go:125:  --discovery-token-ca-cert-hash sha256:508d6d422c226c4403cd251e63c3d5a107e7a421b34512646991d7e44c653898 "
    L81: "cluster.go:125: i  Using Cilium version 1.12.5"
    L82: "cluster.go:125: ? Auto-detected cluster name: kubernetes"
    L83: "cluster.go:125: ? Auto-detected datapath mode: tunnel"
    L84: "cluster.go:125: ? Auto-detected kube-proxy has been installed"
    L85: "cluster.go:125: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L86: "cluster.go:125: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L87: "cluster.go:125: ? Created CA in secret cilium-ca"
    L88: "cluster.go:125: ? Generating certificates for Hubble..."
    L89: "cluster.go:125: ? Creating Service accounts..."
    L90: "cluster.go:125: ? Creating Cluster roles..."
    L91: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..."
    L92: "cluster.go:125: i  Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L93: "cluster.go:125: i  Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L94: "cluster.go:125: ? Creating Agent DaemonSet..."
    L95: "cluster.go:125: ? Creating Operator Deployment..."
    L96: "cluster.go:125: ? Waiting for Cilium to be installed and ready..."
    L97: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L98: "cluster.go:125: daemonset.apps/cilium patched"
    L99: "cluster.go:125: ?[33m    /??_"
    L100: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L101: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L102: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L103: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L104: "cluster.go:125: ?[34m    ___/"
    L105: "cluster.go:125: ?[0m"
    L106: "cluster.go:125: Deployment       cilium-operator    "
    L107: "cluster.go:125: DaemonSet        cilium             "
    L108: "cluster.go:125: Containers:      cilium             "
    L109: "cluster.go:125:                  cilium-operator    "
    L110: "cluster.go:125: Cluster Pods:    0/0 managed by Cilium"
    L111: "cluster.go:125: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L112: "harness.go:583: Found emergency shell on machine f905fd36-c9a3-4f9d-b200-2f8249b333ad console"
    L113: "harness.go:583: Found systemd unit failed to start (?[0;1;39mignition-f???es.service?[0m - Ignition (files). ) on machine f905fd36-c9a3-4f9d-b200-2f8249b333ad console"
    L114: "harness.go:583: Found systemd dependency unit failed to start (?[0;1;39migni???te.target?[0m - Ignition Complete. ) on machine f905fd36-c9a3-4f9d-b200-2f8249b333ad console"
    L115: "harness.go:583: Found emergency shell on machine 5309bf77-6953-43ba-9938-5a1545decc1c console"
    L116: "harness.go:583: Found systemd unit failed to start (?[0;1;39mignition-f???es.service?[0m - Ignition (files). ) on machine 5309bf77-6953-43ba-9938-5a1545decc1c console"
    L117: "harness.go:583: Found systemd dependency unit failed to start (?[0;1;39migni???te.target?[0m - Ignition Complete. ) on machine 5309bf77-6953-43ba-9938-5a1545decc1c console_"
    L118: " "

ok kubeadm.v1.27.2.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.28.1.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok kubeadm.v1.28.1.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

                Diagnostic output for qemu_uefi-amd64, run 1
    L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.28.4"
    L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.28.4"
    L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.28.4"
    L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.28.4"
    L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.9"
    L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.5.9-0"
    L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.10.1"
    L8: "cluster.go:125: [init] Using Kubernetes version: v1.28.4"
    L9: "cluster.go:125: [preflight] Running pre-flight checks"
    L10: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L11: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L12: "cluster.go:125: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L13: "cluster.go:125: W1122 15:39:41.510518    1675 checks.go:835] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended that using __registry.k8s.io/pause:3.9__ as the CRI sandbox image."
    L14: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L15: "cluster.go:125: [certs] Generating __ca__ certificate and key"
    L16: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
    L17: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.9?8]"
    L18: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L19: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
    L20: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
    L21: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L22: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L23: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L24: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L25: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L26: "cluster.go:125: [certs] Generating __sa__ key and public key"
    L27: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L28: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L29: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L30: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L31: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L32: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L33: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L34: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L35: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L36: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L37: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L38: "cluster.go:125: [kubelet-start] Starting the kubelet"
    L39: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L40: "cluster.go:125: [apiclient] All control plane components are healthy after 4.001877 seconds"
    L41: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L42: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L43: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
    L44: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L45: "cluster.go:125: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L46: "cluster.go:125: [bootstrap-token] Using token: mmy7ae.689qyg9kx5rw24iu"
    L47: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L48: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L49: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L50: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L51: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L52: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L53: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L54: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
    L55: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
    L56: "cluster.go:125: "
    L57: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
    L58: "cluster.go:125: "
    L59: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
    L60: "cluster.go:125: "
    L61: "cluster.go:125:   mkdir -p $HOME/.kube"
    L62: "cluster.go:125:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L63: "cluster.go:125:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L64: "cluster.go:125: "
    L65: "cluster.go:125: Alternatively, if you are the root user, you can run:"
    L66: "cluster.go:125: "
    L67: "cluster.go:125:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L68: "cluster.go:125: "
    L69: "cluster.go:125: You should now deploy a pod network to the cluster."
    L70: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L71: "cluster.go:125:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L72: "cluster.go:125: "
    L73: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
    L74: "cluster.go:125: "
    L75: "cluster.go:125: kubeadm join 10.0.0.98:6443 --token mmy7ae.689qyg9kx5rw24iu _"
    L76: "cluster.go:125:  --discovery-token-ca-cert-hash sha256:5f8d0c008348c91eef5a9243049df367e90e757179090bdabc44f9908500c5da "
    L77: "cluster.go:125: i  Using Cilium version 1.12.5"
    L78: "cluster.go:125: ? Auto-detected cluster name: kubernetes"
    L79: "cluster.go:125: ? Auto-detected datapath mode: tunnel"
    L80: "cluster.go:125: ? Auto-detected kube-proxy has been installed"
    L81: "cluster.go:125: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L82: "cluster.go:125: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L83: "cluster.go:125: ? Created CA in secret cilium-ca"
    L84: "cluster.go:125: ? Generating certificates for Hubble..."
    L85: "cluster.go:125: ? Creating Service accounts..."
    L86: "cluster.go:125: ? Creating Cluster roles..."
    L87: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..."
    L88: "cluster.go:125: i  Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L89: "cluster.go:125: i  Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L90: "cluster.go:125: ? Creating Agent DaemonSet..."
    L91: "cluster.go:125: ? Creating Operator Deployment..."
    L92: "cluster.go:125: ? Waiting for Cilium to be installed and ready..."
    L93: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L94: "cluster.go:125: daemonset.apps/cilium patched"
    L95: "cluster.go:125: ?[33m    /??_"
    L96: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L97: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L98: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L99: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L100: "cluster.go:125: ?[34m    ___/"
    L101: "cluster.go:125: ?[0m"
    L102: "cluster.go:125: Deployment       cilium-operator    "
    L103: "cluster.go:125: DaemonSet        cilium             "
    L104: "cluster.go:125: Containers:      cilium             "
    L105: "cluster.go:125:                  cilium-operator    "
    L106: "cluster.go:125: Cluster Pods:    0/0 managed by Cilium"
    L107: "cluster.go:125: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L108: "--- FAIL: kubeadm.v1.28.1.cilium.base/node_readiness (91.69s)"
    L109: "kubeadm.go:301: nodes are not ready: ready nodes should be equal to 2: 1"
    L110: "--- FAIL: kubeadm.v1.28.1.cilium.base/IPSec_encryption (64.90s)"
    L111: "cluster.go:125: 2023/11/22 15:41:57 Unable to start gops: mkdir /home/core/.config: no space left on device"
    L112: "cluster.go:125: Error: Unable to determine status:  timeout while waiting for status to become successful: context deadline exceeded"
    L113: "cluster.go:145: __/opt/bin/cilium status --wait --wait-duration 1m__ failed: output ?[33m    /????_"
    L114: "?[36m /?????[33m___/?[32m????_?[0m    Cilium:         ?[31m1 errors?[0m, ?[33m1 warnings?[0m"
    L115: "?[36m ___?[31m/????_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L116: "?[32m /?????[31m___/?[35m????_?[0m    Hubble:         ?[36mdisabled?[0m"
    L117: "?[32m ___?[34m/????_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L118: "?[34m    ___/"
    L119: "?[0m"
    L120: "Deployment        cilium-operator    Desired: 1, Ready: ?[32m1/1?[0m, Available: ?[32m1/1?[0m"
    L121: "DaemonSet         cilium             Desired: 2, Ready: ?[33m1/2?[0m, Available: ?[33m1/2?[0m, Unavailable: ?[31m1/2?[0m"
    L122: "Containers:       cilium-operator    Running: ?[32m1?[0m"
    L123: "cilium             Pending: ?[32m1?[0m, Running: ?[32m1?[0m"
    L124: "Cluster Pods:     3/3 managed by Cilium"
    L125: "Image versions    cilium-operator    quay.io/cilium/operator-generic:v1.12.5@sha256:b296eb7f0f7656a5cc19724f40a8a7121b7fd725278b7d61dc91fe0b7ffd7c0e: 1"
    L126: "cilium             quay.io/cilium/cilium:v1.12.5@sha256:06ce2b0a0a472e73334a7504ee5c5d8b2e2d7b72ef728ad94e564740dd505be5: 2"
    L127: "Errors:           cilium             cilium          1 pods of DaemonSet cilium are not ready"
    L128: "Warnings:         cilium             cilium-9k5lx    pod is pending, status Process exited with status 1_"
    L129: " "
    L130: "  "

ok kubeadm.v1.28.1.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.nfs.v3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.nfs.v4 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok linux.ntp 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok misc.fips 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok packages 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok sysext.custom-docker.torcx 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok sysext.custom-oem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok sysext.simple 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.journal.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.journal.user 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

ok systemd.sysusers.gshadow 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants