We proudly announce that we will publish the most comprehensive FREE AWS EKS tutorial. In this series of articles, you will learn how to work deeply with AWS EKS and deploy, manage, upgrade, and operate EKS clusters through the command line “also planned for CloudFormation, Terraform, OpenTofu, Pulumi and Ansible”. EKS stands for Elastic Kubernetes Service and is the AWS-managed Kubernetes you can deploy and use with ease. Easily? Although deploying and managing EKS clusters is easier than On-premises Kubernetes deployments, it has its own challenges.

Follow our social media:

https://www.linkedin.com/in/ssbostan

https://www.linkedin.com/company/kubedemy

https://www.youtube.com/@kubedemy

https://telegram.me/kubedemy

Watch on YouTube:

EKS cluster pre-requirements:

To deploy an EKS cluster, we need a couple of other AWS resources:

• An IAM user to deploy cluster and resources and be the cluster owner.
• AWS CLI setup and configured to create required resources.
• An AWS VPC in any EKS-supported region to act as the network infrastructure.
• A couple of public and private subnets to deploy Kubernetes worker nodes.
• An Internet Gateway to be able to connect to the internet from public subnets.
• A NAT Gateway for private subnets to be able to access the internet.
• A new Route Table and NAT Gateway route for private subnets.
• An EC2 keypair to be able to connect to EC2 instances.

Step 1 — Create an IAM user:

Connect to your AWS account with the root user and create a new IAM user for the next steps. We use this IAM user to create, deploy, manage, etc., in this article and all future articles. Never delete this IAM user, as it will be the owner of the EKS cluster.

IAM > Users > Add users > Set username > Set AdministratorAccess policy.

After creating the user, create an access key from the Security credentials.

Note: I set the AdministratorAccess policy to avoid challenging permissions. You must create an IAM principal with the least privileges to avoid security pitfalls. AWS IAM service and permissions will be explained in a separate thread.

Step 2 — Setup AWS CLI:

There are many ways to install and use AWS CLI; as we love containers, we run it inside one using the lovely Docker. You can also use other methods.

docker run -d --name aws-cli --entrypoint sleep amazon/aws-cli infinity

docker exec -it aws-cli bash

Now, we are inside the AWS CLI container.

Step 3 — Configure AWS CLI and environment:

To setup our environment, we use the previously created IAM user credentials.

bash-4.2# aws configure
AWS Access Key ID [None]: YOUR ACCESS KEY HERE
AWS Secret Access Key [None]: YOUR SECRET KEY HERE
Default region name [None]: eu-west-2
Default output format [None]: json

Note: You can use any EKS-supported region you want.

To confirm your configuration, run the following command:

aws sts get-caller-identity

The output should be something like this:

Here is the architecture diagram of what we did in the previous steps:

Step 4 — Create VPC:

Run the following command to create a new VPC in your region:

bash-4.2# aws ec2 create-vpc \
  --cidr-block 10.0.0.0/16 \
  --tag-specifications "ResourceType=vpc,Tags=[{Key=owner,Value=kubedemy}]"

{
    "Vpc": {
        "CidrBlock": "10.0.0.0/16",
        "DhcpOptionsId": "dopt-0c3ad00db38b27d36",
        "State": "pending",
        "VpcId": "vpc-09a71d02067cf835d",
        "OwnerId": "231144931069",
        "InstanceTenancy": "default",
        "Ipv6CidrBlockAssociationSet": [],
        "CidrBlockAssociationSet": [
            {
                "AssociationId": "vpc-cidr-assoc-0aa226721dfc1c806",
                "CidrBlock": "10.0.0.0/16",
                "CidrBlockState": {
                    "State": "associated"
                }
            }
        ],
        "IsDefault": false,
        "Tags": [
            {
                "Key": "owner",
                "Value": "kubedemy"
            }
        ]
    }
}

By running the aws ec2 create-vpc command, you create a new VPC, giving you the VPC ID and some information. We need VPC ID for further steps.

aws ec2 describe-vpcs

This command also shows you the list of available VPCs in this region.

Note: You can ignore this step if you already have a VPC in your active region. Just run aws ec2 describe-vpcs and find the VPC ID for further steps.

Step 5 — Enable VPC DNS Hostnames feature:

Kubernetes worker nodes join the cluster control plane using their DNS hostname, and the control plane verifies them in the same way. So, we must enable the DNS Hostnames feature of VPC to make it possible to setup the EKS cluster.

Replace the VPC ID with your own VPC ID from the last command.

aws ec2 modify-vpc-attribute --vpc-id vpc-09a71d02067cf835d --enable-dns-hostnames

Step 6 — Create Public Subnets:

In each Availability Zone, we need at least two subnets, one public and one private. You can ignore creating any of them, but I will create both public and private subnets for future articles and explain how we can setup clusters in both cases.

To see all availability zones in the active region, run the following command:

aws ec2 describe-availability-zones

Setup public subnets, Internet gateway and route:

Replace the VPC ID and Availability Zone name with your own.

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.1.0/24 \
  --availability-zone eu-west-2a \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.2.0/24 \
  --availability-zone eu-west-2b \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.3.0/24 \
  --availability-zone eu-west-2c \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

Running each command will show you the Subnet ID of the created subnet.

Important: To deploy worker nodes to a public subnet, the subnet must automatically assign a public IPv4 IP address to nodes. Without a public IP address, the process of joining worker nodes to the cluster will fail.

To enable it, run the following command for each public subnet:

aws ec2 modify-subnet-attribute --subnet-id subnet-0ff015478090c2174 --map-public-ip-on-launch

aws ec2 modify-subnet-attribute --subnet-id subnet-01b107cea804fdff1 --map-public-ip-on-launch

aws ec2 modify-subnet-attribute --subnet-id subnet-09b7d720aca170608 --map-public-ip-on-launch

Now create an Internet Gateway using the following command:

aws ec2 create-internet-gateway --tag-specifications "ResourceType=internet-gateway,Tags=[{Key=owner,Value=kubedemy}]"

Assign it to your VPC using this command:

aws ec2 attach-internet-gateway \
  --vpc-id vpc-09a71d02067cf835d \
  --internet-gateway-id igw-046c41e4bc7f0ad14

Find the default route table ID and create a new route to the internet gateway:

aws ec2 describe-route-tables

aws ec2 create-route \
  --route-table-id rtb-0c418972dc98d43ed \
  --destination-cidr-block 0.0.0.0/0 \
  --gateway-id igw-046c41e4bc7f0ad14

If it returns True, it means everything is good.

Step 7 — Create Private Subnets:

AWS recommends deploying Kubernetes worker nodes in private subnets, and to access the internet, they must use NAT Gateway “for IPv4” and Egress-only Internet Gateway “for IPv6” Egress-only is free, but for NAT Gateway, you have to pay at least $1/day without any usage. You can ignore this step if you don’t want to pay for NAT Gateway, and you can come back to this step when I explain deploying clusters in Private subnets.

Setup private subnets, NAT Gateway, Route table and route:

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.11.0/24 \
  --availability-zone eu-west-2a \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.12.0/24 \
  --availability-zone eu-west-2b \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

aws ec2 create-subnet \
  --vpc-id vpc-09a71d02067cf835d \
  --cidr-block 10.0.13.0/24 \
  --availability-zone eu-west-2c \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=owner,Value=kubedemy}]"

To create a new route table:

aws ec2 create-route-table \
  --vpc-id vpc-09a71d02067cf835d \
  --tag-specifications "ResourceType=route-table,Tags=[{Key=owner,Value=kubedemy}]"

Now associate the new route table to private subnets:

aws ec2 associate-route-table \
  --route-table-id rtb-04e5f753ba648101f \
  --subnet-id subnet-02a9b470ef3111fd4

aws ec2 associate-route-table \
  --route-table-id rtb-04e5f753ba648101f \
  --subnet-id subnet-066e961abfc243923

aws ec2 associate-route-table \
  --route-table-id rtb-04e5f753ba648101f \
  --subnet-id subnet-0774243a62c6808aa

Important: Now we have to create a NAT Gateway. NAT Gateway should be created in a public subnet with internet access through Internet Gateway. We have three public subnets in our example, and we can use one of them.

Before creating a NAT Gateway, we need an Elastic IP. To create one, run:

aws ec2 allocate-address \
  --domain vpc \
  --tag-specifications "ResourceType=elastic-ip,Tags=[{Key=owner,Value=kubedemy}]"

Now run the following command to create a NAT Gateway:

aws ec2 create-nat-gateway \
  --allocation-id eipalloc-0c87f9d1007e5ed0a \
  --subnet-id subnet-0ff015478090c2174 \
  --tag-specifications "ResourceType=natgateway,Tags=[{Key=owner,Value=kubedemy}]" \
  --connectivity-type public

Attention: AWS charges you for NAT Gateway and Elastic IP. Delete them after implementing this lab and recreate them in the next labs as needed.

To allow private subnets to connect to the internet, add the following route:

aws ec2 create-route \
  --route-table-id rtb-04e5f753ba648101f \
  --destination-cidr-block 0.0.0.0/0 \
  --nat-gateway-id nat-0930f9f4426b9c43d

If it returns True, it means everything is good.

Here is the architecture diagram of what we did in the previous steps:

Step 8 — Import SSH Key pair:

Although we recommend not doing SSH to worker nodes and using the in-cluster method for debugging purposes, I will introduce some related things in the next articles. So, we create an SSH key on our local machine and import the public key into our AWS account. You can also create a keypair in the AWS Dashboard and download its private key.

To create a new key pair, run the following command:

aws ec2 create-key-pair \
  --key-name kubedemy \
  --key-type ed25519 \
  --tag-specifications "ResourceType=key-pair,Tags=[{Key=owner,Value=kubedemy}]" \
  --key-format pem \
  --query "KeyMaterial" \
  --output text > kubedemy.pem

It creates a key pair and saves the private key into kubedemy.pem file.

Conclusion:

So far, we created all requirements to deploy EKS clusters in both public and private subnets. In the next articles, we will work on deploying the Kubernetes cluster.

If you like this series of articles, please share them and write your thoughts as comments here. Your feedback encourages me to complete this massively planned program. Just share them and provide feedback. I’ll make you an AWS EKS black belt.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Follow Kubedemy LinkedIn https://www.linkedin.com/company/kubedemy

Follow Kubedemy Telegram https://telegram.me/kubedemy

AWS EKS — Part 1 — Deploy EKS Cluster Requirements was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Register at https://kubedemy.io/aws-eks-black-belt-course

Hello and welcome to the most complete AWS EKS course, Black Belt Course.

This course is created for those who want to learn AWS EKS and Kubernetes in depth. AWS EKS is an AWS-managed Kubernetes service, which stands for Elastic Kubernetes Service. Many companies are looking to migrate their applications to Kubernetes, especially EKS, and because of that, it’s super important for all Cloud engineers, Platform engineers, DevOps engineers and Kubernetes engineers to have hands-on experience with EKS and Kubernetes. So, if you want to learn EKS, join this course for free.

The last update was on 29 Jan 2023 to help you find the latest changes.

AWS EKS is super easy as it’s a managed service:

Although EKS is a managed service, there are many topics you should learn before using EKS “also Kubernetes” to run your production workloads. Upgrading cluster, worker nodes, addons. Configuring storage, network, network policies, load balancers, ingress controller, and subnets. Managing costs, monitoring, logging, and application performance. Cluster security, network security, application security, supply chain security and many more topics that we will cover all of them in this course.

Course program and materials:

• 54 EKS-focused articles specially written for AWS EKS environments. (FREE)
• All related awscli commands to deploy and test scenarios. (FREE)
• Terraform codes to setup and deploy environments through automation. (FREE)
• Question and answer chat forum to support students. (FREE)
• Face-to-face online consultation meetings for companies and enterprises. (Paid)
• Design and Implementation services for companies and enterprises. (Paid)
• Troubleshooting sessions for companies and enterprises. (Paid)

Course details:

The course starts on 06 Nov 2023 and ends on 11 Nov 2024, and every article will be published at 9 a.m. UTC every Monday. We may continue the course for another year with new articles, which depends on participants’ and sponsors’ welcome.

AWS EKS tutorial free registration:

AWS EKS Black Belt Course | Kubedemy

Become sponsor:

We always welcome our sponsors, and we believe that together, we can build better communities and experiences. If you’re interested in supporting this course and promoting your products, please send a sponsoring request. Right now, we accept Credit sponsors and Content sponsors, explained in the following form:

Become the AWS EKS Black Belt course sponsor

AWS EKS Black Belt Course topics:

AWS EKS — Part 1 — Deploy EKS cluster requirements

AWS EKS — Part 2 — Deploy managed cluster control plane

AWS EKS — Part 3 — Deploy worker nodes using managed node groups

AWS EKS — Part 4 — Deploy worker nodes using custom launch templates

AWS EKS — Part 5 — Deploy self-managed worker nodes

AWS EKS — Part 6 — Deploy Bottlerocket worker nodes and update operator

AWS EKS — Part 7 — Deploy ARM-based Kubernetes Worker nodes

AWS EKS — Part 8 — Deploy Worker nodes using Spot Instances

AWS EKS — Part 9 — Deploy Worker nodes using Fargate Instances

AWS EKS — Part 10 — Deploy cluster with private API endpoint

AWS EKS — Part 11 — Deploy workers in private subnets behind the NAT gateway

AWS EKS — Part 12 — Deploy fully air-gapped EKS clusters without internet

AWS EKS — Part 13 — Setup IAM Roles for Service Accounts (IRSA)

AWS EKS — Part 14 — Restrict IMDS access to secure AWS account access

AWS EKS — Part 15 — Enable Secrets encryption at rest with AWS KMS Service

AWS EKS — Part 16 — Enable Cluster logging with AWS CloudWatch Service

AWS EKS — Part 17 — Kubernetes Authentication with AWS IAM Principal

AWS EKS — Part 18 — Kubernetes Authentication with OIDC and Kubelogin

AWS EKS — Part 19 — Deploy and manage Amazon EKS Cluster Addons

AWS EKS — Part 20 — Upgrade EKS Cluster and Worker nodes

AWS EKS — Part 21 — Delete EKS Cluster and dependent resources

AWS EKS — Part 22 — The title is Hidden — Exposed Soon.

AWS EKS — Part 23 — The title is Hidden — Exposed Soon.

AWS EKS — Part 24 — The title is Hidden — Exposed Soon.

AWS EKS — Part 25 — The title is Hidden — Exposed Soon.

AWS EKS — Part 26 — The title is Hidden — Exposed Soon.

AWS EKS — Part 27 — The title is Hidden — Exposed Soon.

AWS EKS — Part 28 — The title is Hidden — Exposed Soon.

AWS EKS — Part 29 — The title is Hidden — Exposed Soon.

AWS EKS — Part 30 — The title is Hidden — Exposed Soon.

AWS EKS — Part 31 — The title is Hidden — Exposed Soon.

AWS EKS — Part 32 — The title is Hidden — Exposed Soon.

AWS EKS — Part 33 — The title is Hidden — Exposed Soon.

AWS EKS — Part 34 — The title is Hidden — Exposed Soon.

AWS EKS — Part 35 — The title is Hidden — Exposed Soon.

AWS EKS — Part 36 — The title is Hidden — Exposed Soon.

AWS EKS — Part 37 — The title is Hidden — Exposed Soon.

AWS EKS — Part 38 — The title is Hidden — Exposed Soon.

AWS EKS — Part 39 — The title is Hidden — Exposed Soon.

AWS EKS — Part 40 — The title is Hidden — Exposed Soon.

AWS EKS — Part 41 — The title is Hidden — Exposed Soon.

AWS EKS — Part 42 — The title is Hidden — Exposed Soon.

AWS EKS — Part 43 — The title is Hidden — Exposed Soon.

AWS EKS — Part 44 — The title is Hidden — Exposed Soon.

AWS EKS — Part 45 — The title is Hidden — Exposed Soon.

AWS EKS — Part 46 — The title is Hidden — Exposed Soon.

AWS EKS — Part 47 — The title is Hidden — Exposed Soon.

AWS EKS — Part 48 — The title is Hidden — Exposed Soon.

AWS EKS — Part 49 — The title is Hidden — Exposed Soon.

AWS EKS — Part 50 — The title is Hidden — Exposed Soon.

AWS EKS — Part 51 — The title is Hidden — Exposed Soon.

AWS EKS — Part 52 — The title is Hidden — Exposed Soon.

AWS EKS — Part 53 — The title is Hidden — Exposed Soon.

AWS EKS — Part 54 — The title is Hidden — Exposed Soon.

Course materials and license:

https://github.com/kubedemy/aws-eks-black-belt-course

If you like this series of articles, please share them and write your thoughts as comments here. Your feedback encourages me to complete this massively planned program. Just share them and provide feedback. I’ll make you an AWS EKS black belt.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Follow Kubedemy LinkedIn https://www.linkedin.com/company/kubedemy

Follow Kubedemy Telegram https://telegram.me/kubedemy

In the previous article, we learnt about QoS classes in Kubernetes and how Kubernetes evicts Pods if node resources are running out. Now it’s time to be familiar with the PriorityClass resource in Kubernetes. Imagine you have a bunch of Pods with the same QoS class in the worker node, and the worker node’s resources are running out! How should Kubernetes decide which Pod should be evicted to free up node resources? Here is where that PriorityClass comes into the picture. Priority class lets you prioritise your workloads and tell Kubernetes which ones are more important and in the time of node pressure, which ones should be evicted first.

Read the previous article about Kubernetes QoS classes:

Kubernetes Hacks and Tricks — #7 Pod and Container Resource and QoS classes

In Kubernetes, PriorityClass has two use cases:

1- Helps kube-scheduler to evict low-priority Pods when a high-priority Pod is waiting in Pending mode in the queue due to lack of resources.

2- Helps kubelet to evict Pods when the node is under pressure and other high-priority Pods need more resources to do their job. Note that there is a hidden rule in this type of eviction.

Default Kubernetes Priority classes:

Kubernetes comes with two default “system” priority classes to save cluster and node critical workloads in pressure situation.

Cluster-critical priority class should be used when the location of the application does not matter. For example Prometheus monitoring system.

Node-critical priority class should be used for all applications that need to be run on specific nodes only. For example Node-exporter, CSI node plugin, CNI node plugin, etc.

Create your own PriorityClass:

PriorityClass name cannot be started with system- as it is reserved for Kubernetes itself. Value can be -2147483648 to 1000000000. The higher value higher the priority. The default priority class is used for all the Pods that have not specified priorityClassName.

When preemptionPolicy is set to Never, the scheduler will not evict any other Pods to schedule pending Pods of this priority class.

How to use PriorityClass:

If you don’t assign any priorityClassName to the Pod, the global default priority class will be used for pod.

Important note about node pressure eviction:

This is the hidden rule! kubelet node-pressure eviction does not evict Pods when their usage does not exceed their requests. If a Pod with lower priority is not exceeding its requests, it won’t be evicted. Another Pod with higher priority that exceeds its requests may be evicted.

Conclusion and Best Practices:

Understanding Kubernetes scheduling and the kubelet eviction process can help you deploy reliable and highly available workloads on Kubernetes. For every workload, you need to use proper QoS and Priority classes. For all cluster and node critical workloads, you have to specify system priority classes created by Kubernetes itself. Kubernetes has not defined any default priority class, and you need to create your default priority class.

If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Follow my Twitter https://twitter.com/saeidbostan

Good luck.

Kubernetes Hacks and Tricks — #8 Pod Preemption, Eviction and Priority classes was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

You know how Kubernetes decide to find which Pods should be evicted if node resource is not enough or is running out? In this article, we’ll talk about this and introduce Kubernetes Quality of Service classes related to this topic.

What is Node Pressure?

Node pressure is a term that explains the worker node’s resources are running out. Pressure means something got limited on the worker node, which can be Memory, Storage, PID space, etc… When the worker node gets pressured, Kubelet tries to free up its resources, which means some Pods should be evicted on the node, meaning the application should go down and run somewhere else. Finally, it means you will experience some downtime in your system, and it’s a nightmare for everyone.

Kubernetes Quality of Service(QoS) classes:

Kubernetes comes with 3 default QoS classes which help Kubernetes determine which pods should be evicted first on rainy days. The possible QoS classes are Guaranteed, Burstable and BestEffort. When a Node runs out of resources, Kubernetes will first evict BestEffort Pods running on that Node, followed by Burstable and finally Guaranteed Pods.

BestEffort QoS class:

When you don’t specify any resources for containers within a Pod, the Pod will be run in this QoS class. This class has the minimum priority in the cluster, and the Pods of this class are the first candidate to be evicted when the node is under pressure.

BestEffort means containers can use resources as much as they can, and because you didn’t provide any requests and limits, there is no guarantee for them to be alive as you expect. They can use node resources if any resource is available, and they are the first target for eviction if the node is faced with pressure. They make their best effort, and you need to pray and hope they will be running as long as you expect.

Burstable QoS class:

Burstable class is the one you provide requests and limits for all the containers in the Pod, but the amount of memory and CPU of requests is less than the limits. Kubernetes scheduler schedules Pods based on their resource requests and has nothing to do with limits. Resource limits get enforced when the Pod runs on the worker node, and if it tries to consume more than limits, Kubelet kills it. Still, there is another hidden fact about this type of QoS class! there is no guarantee that the worker node can provide resources up to Pod’s limits. If Pod get scheduled on a worker node “based on requests”, and the worker node resources go under pressure due to “lack of available physical resource”, and the container tries to consume more resources than requests “as it expects to use up to limits”, Kubelet may kill containers of this QoS class if some Pods with Guaranteed QoS classes need to consume more resources in that node.

So you need to be cautious and know what exactly you are doing in cluster and worker node and how much resources you need now and in the future. Note that if you set an inappropriate amount of resources, you may go overutilization and BOOOM.

Guaranteed QoS class:

The best one for app, bad one for your pocket. When you provide both requests and limits with the same value, your Pod will run in the Guaranteed QoS class. It means it will be the last candidate evicted from the node. You should be aware of two important things. The first one is how much resources your Pod really needs, and the second one is how to spread your Pods across worker nodes to prevent node underutilization. Kubernetes scheduler schedules based on resource requests, and if you set requests and limits with the same values but less than Pod really needs, your Pod will be scheduled on the node, but it gets restarted with “Out Of Memory” kill status when it tries to consume more than limits. On the other hand, if you set more than what it needs, your node will go underutilization, and you should pay for what you do not use.

To run Pods with Guaranteed QoS class, create a proper spread topology to avoid going underutilized, and run fair stress tests just like your real-world traffic on the application to find the right resource needs.

Conclusion:

BestEffort has the minimum priority and it’s the first candidate to be evicted from worker node. There is no guarantee that it will be run properly with adequate resources. It can get evicted anytime.

Burstable has the second priority, and in the case of node pressure, it will be evicted after BestEffort containers. Setting the wrong amount of requests/limits can cause overutilization and significantly impact all of the running Pods on the node.

Guaranteed has the last priority and will get evicted after Burstable pods. Setting an inappropriate amount of resources causes Pod getting into an infinity restart loop, node underutilization, or, if you set more than what you have, in Pending state.

Keep your eyes open for the following articles about spread topologies and priority classes to learn how you should deploy production-grade workloads in Kubernetes clusters.

If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #7 Pod and Container Resource and QoS classes was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

As you know, recently, Kubernetes changed its main repository from k8s.gcr.io to registry.k8s.io, meaning you should update your manifests as soon as possible. In this note, I tell you how you can find running Pods using the legacy Kubernetes image registry.

https://kubernetes.io/blog/2023/03/10/image-registry-redirect

The following command list all running Pods in your cluster, along with their namespaces and image address.

kubectl get po -A -o jsonpath='{range .items[*]}{"namespace="}{.metadata.namespace}{" "}{"pod="}{.metadata.name}{" "}{range .spec.containers[*]}{"image="}{.image}{" "}{end}{"\n"}{end}'

Now, to find Pods using k8s.gcr.io you can grep the result.

kubectl get po -A -o jsonpath='{range .items[*]}{"namespace="}{.metadata.namespace}{" "}{"pod="}{.metadata.name}{" "}{range .spec.containers[*]}{"image="}{.image}{" "}{end}{"\n"}{end}' | grep k8s.gcr.io

After finding target Pods, you can describe them and find they are running by which Deployment, StatefulSet or DaemonSet, and update manifest to use new Kubernetes image registry.

Important note:
This change from the Kubernetes community has nothing to do with your applications, and in most cases, you need to update static Pods manifests from worker nodes. For example, Kube-apiserver Pod which runs directly by Kubelet on the worker node.

https://kubernetes.io/docs/tasks/configure-pod-container/static-pod

If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #6 Find Pods running from deprecated registry was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

What really is dnsPolicy in Pod spec, and when should we change it? By setting the dnsPolicy option in Pod spec, you define how DNS requests should be resolved. If you want to resolve DNS requests with just the internal DNS server, or you want to use the internal DNS server first and then use upstream DNS, or resolve all of them using an external DNS server or something else, you can do it by configuring Pod’s dnsPolicy and dnsConfig options. In Kubernetes, DNS policy can be configured per Pod, which means you can customize the DNS resolving method for every Pod in your cluster.

Allowed values in dnsPolicy option:

• Default Pod inherits worker node’s DNS configuration.
• None all DNS configurations are ignored and dnsConfig option will be used to configure Pod DNS configuration.
• ClusterFirst use cluster DNS first, CoreDNS. If the DNS query does not match any domains in cluster DNS, forward it to upstream DNS servers.
• ClusterFirstWithHostNet in the case of using hostNetwork it uses the cluster DNS server first and then forward unmatched queries to upstream DNS servers. Will be explained more in the following article.

Although it seems Default should be the default, but ClusterFirst is the default option. So to speak, if you have not set this option, ClusterFirst will be used as the default value.

ClusterFirst (real default value):

Test result:

As you can see, the request was resolved by the cluster’s DNS server through 10.43.0.10 (CoreDNS IP address), and it used my cluster domain.

Default (its name is just Default, but it’s not default):

Test result:

As you can see, it used my node’s DNS server to resolve the DNS query, and because of that, it couldn’t resolve it. Moreover, it used the worker node’s domain instead of the cluster domain.

None (to customize DNS configuration):

Test result:

As you can see, it tried to resolve the specified address with our search list through the Google DNS server.

Note: To understand how containers resolve domains, see /etc/resolve.conf inside the container for every configuration explained above.

If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #5 Pod DNS Policy and DNS Config options was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Long time no see! After a long time, I got a chance to release and continue the complete Jenkins tutorial. In this article, we will learn how to use Git in our pipelines. Using Git is a most important topic in CI/CD pipelines as we store everything in Git, our source codes and pipeline codes themselves. To work with Git, you need the Jenkins Git plugin. I’ve prepared everything you need in the following Jenkins stack. The following Jenkins stack is available for both Docker and Kubernetes environments.

• GitHub - ssbostan/jenkins-stack-kubernetes: Scripts and manifests to deploy complete Jenkins on Kubernetes
• GitHub - ssbostan/jenkins-stack-docker: Docker-compose version of jenkins-stack-kubernetes

Jenkins Pipeline in Git repository:

When you’re about to create a new Jenkins job, pipeline, you can connect Jenkins to a specific Git repository to read your pipeline code, the Jenkinsfile. In the case of CI pipelines, your Jenkinsfile is along with your source code, and Jenkins will clone your Git repository, which consists of your application source code and Jenkinsfile. You need your source code in the Jenkins CI pipeline to be able to build it, test it, and release it. In CD pipelines, you may have multiple things like Kubernetes manifests, GitOps manifests, Ansible playbooks, Docker-compose files, Terraform codes, Pulumi codes, etc. along with your Jenkinsfile.

In the Pipeline section you can configure:

• Git repository URL address
• Credentials to clone private repository (User/Pass or SSH)
• Target branch or Ref name
• Script path, Jenkins pipeline file (Default: Jenkinsfile)

It’s super easy, but what if you want to clone the Git repository inside your pipeline or commit and push something into Git repo using pipeline? Jenkins provides you with a couple of ways to clone a Git repository inside the pipeline. The straight forward way is to use the Git plugin.

Clone Git repository in Jenkins Pipeline:

git(
    url: "REPO_URL_OR_SSH_URI_ADDRESS",
    branch: "REPO_BRANCH_NAME",
    credentialsId: "REPO_CREDENTIAL_ID_IN_JENKINS",
    changelog: true,
    poll: true
)

For public repositories, you don’t need to specify credentialsId option.

If poll option is set to true, repo will be polled for changes.

If changelog option is set to true, changelog will be computed.

Here is the full example of how to clone a Git repository:

Attention: If you read pipeline code from Git using Jenkinsfile “We always use this method”, you should use dir to go to another directory and clone the other repository to prevent conflict between the main repo and the newly cloned repo.

Commit and Push to Git using Jenkins Pipeline:

Now it’s time to commit something new and push it back to the Git repository. Why do we need it? In many cases, you may need to add, commit and push from your pipeline. For example, updating GitOps manifests after building the image of the new application version, or pushing terraform.tfplan file into artifact repo.

First you need to configure Git plugin:

Dashboard > Manage Jenkins > Configure System > Git plugin

To be able to push to the GitHub repository, you need to have a token. To create a token, go to Developer Settings in your GitHub account.

Then back to Jenkins and create a new Username/Password credential.

Manage Jenkins > Credentials > System > Global credentials

For GitHub, write TOKEN in the Password field.

Now go to your pipeline, commit and push something to the Git repository. It doesn’t matter if you’ve cloned Git repository in the pipeline or Jenkins has cloned it because of Jenkinsfile. You just need a Git repository with .git directory and credentials to push to the origin.

withCredentials is a pipeline step to inject credentials to being used by applications in its block. As you can see, I injected a gitUsernamePassword to be able to run git push command. I will explain it more in future articles.

If you are faced with this error, you should solve it by running git config command inside the Jenkins container.

git config --global user.name "Jenkins Pipeline"
git config --global user.email "jenkins@localhost"

Now, check your Git repository. You should be able to see a new commit pushed by Jenkins pipeline.

Conclusion:

Using Git repositories in CI/CD pipelines is a daily task you should learn well. In the following articles, we will work more with Git repositories as everything we have is stored in Git repositories. To be honest, I’m super busy these days, but I promise to spend more time writing new articles.

If you liked it, please like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

GitHub - ssbostan/jenkins-tutorial: The completest Jenkins tutorial, reference, awesome, examples

Jenkins Tutorial — Part 10 — Work with Git in Pipeline was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

In access-limited clusters where you don’t have access to cluster resources, you may need worker nodes’ info like worker node name, node IP address, etc. You can’t run kubectl get nodes in such clusters because the cluster admin has restricted your access to only particular namespaces and namespaced resources. This trick is for you if you are blocked in such a situation. First, you must run the following DaemonSet to ensure you have a Pod in every worker node to be able to find worker nodes’ info.

After that, run the following command to show you all worker nodes’ names and IP addresses. Easier than you think.

kubectl get pod -l app=test -o jsonpath='{range .items[*]}{.spec.nodeName}{"\t"}{.status.hostIP}{"\n"}{end}'

That’s it. Short but practical. If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #4 Find worker node info from Pod spec info was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Do you want to know which images are downloaded and available on your worker nodes? If yes, you need to run the following command.

kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{range .status.images[*]}{"\t"}{.names[0]}{"\n"}{end}{end}'

This command list all Kubernetes nodes and their available images.

That’s it. Short but practical. If you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #3 List all downloaded images on worker nodes was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

In some applications, the application configurations are set by default in the codebase (hard-coded) and if you want to override them, you can provide your override values through environment variables. Sometimes and for some configurations, it doesn’t matter if the override values exist. However, you want to make sure the application runs without any issues. If ConfigMaps/Secrets are available, use them, and if there are no ConfigMaps/Secrets for the application, run the application with its default configuration. In such a case, Kubernetes provides us “optional” field for using ConfigMaps/Secrets as environment variables in Pods.

If you run the pod with a regular manifest and the ConfigMap/Secret does not exist, it will face with error.

apiVersion: v1
kind: Pod
metadata:
  name: optional-configmap-secret
spec:
  containers:
    - name: registry
      image: registry:latest
      envFrom:
        - configMapRef:
            name: registry-config
        - secretRef:
            name: registry-secret

Now we can use the optional field to run the pod even if ConfigMap/Secret is not available to be used by pod container.

That’s it. You did it. if you liked it, like it.

Follow my LinkedIn https://www.linkedin.com/in/ssbostan

Good luck.

Kubernetes Hacks and Tricks — #2 Optional ConfigMaps and Secrets in Pods was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.