Skip to content

2023

Implementing Blue/Green and Canary Deployment Strategies in Kubernetes

CKAD

Overview

Learn how to implement blue/green and canary deployment strategies in Kubernetes. These methods enhance stability and reliability when deploying new versions of applications.

Key Concepts

Blue/Green and Canary deployments are strategies to reduce risks during application updates, allowing gradual and controlled rollouts.

Blue/Green Deployment

What is Blue/Green Deployment?

Blue/Green Deployment involves two identical environments: one active (Blue) and one idle (Green). New versions are deployed to Green and, after testing, traffic is switched from Blue to Green.

Blue Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: blue-deployment
  labels:
    app: bluegreen-test
    color: blue
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bluegreen-test
      color: blue
  template:
    metadata:
      labels:
        app: bluegreen-test
        color: blue
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80
Green Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: green-deployment
  labels:
    app: bluegreen-test
    color: green
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bluegreen-test
      color: green
  template:
    metadata:
      labels:
        app: bluegreen-test
        color: green
    spec:
      containers:
        - name: nginx
          image: nginx:1.15.8
          ports:
            - containerPort: 80
Service to Switch Traffic
apiVersion: v1
kind: Service
metadata:
  name: bluegreen-test-svc
spec:
  selector:
    app: bluegreen-test
    color: blue  # Change to green to switch traffic
  ports:
    - protocol: TCP
      port: 80

Switching Traffic

Update the color label in the Service from blue to green to direct traffic to the new version.

Canary Deployment

What is Canary Deployment?

Canary Deployment involves rolling out a new version to a small subset of users before deploying it to the entire user base, allowing for gradual and controlled updates.

Main Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: main-deployment
spec:
  replicas: 5  # Main user base
  selector:
    matchLabels:
      app: canary-test
      environment: main
  template:
    metadata:
      labels:
        app: canary-test
        environment: main
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80

Canary Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: canary-deployment
spec:
  replicas: 1  # Subset of users
  selector:
    matchLabels:
      app: canary-test
      environment: main
  template:
    metadata:
      labels:
        app: canary-test
        environment: main
    spec:
      containers:
        - name: nginx
          image: nginx:1.15.8
          ports:
            - containerPort: 80

Service to Direct Traffic:

apiVersion: v1
kind: Service
metadata:
  name: canary-test-svc
spec:
  selector:
    app: canary-test
  ports:
    - protocol: TCP
      port: 80

Managing Traffic

Control user exposure to the new version by adjusting the number of replicas in the canary deployment.

Conclusion

Blue/Green and Canary deployment strategies in Kubernetes offer a methodical approach to manage application updates, reducing risks and ensuring a smoother rollout process.

CKAD kubectl Cheat Sheet

CKAD

0. Context Management

0.1 View Current Context

kubectl config current-context

0.2 List All Contexts

kubectl config get-contexts

0.3 Switch Context

kubectl config use-context my-context
  • my-context: Name of the context to switch to.

1. Pods

1.1 Managing a Pod

Creating a Pod
kubectl run my-pod --image=nginx:latest --restart=Never --env=VAR1=value1
  • --image nginx:latest: Specifies the container image.
  • --restart Never: Controls the restart policy.
  • --env VAR1=value1: Sets environment variables.

Declarative:

  • Generate YAML:
kubectl run my-pod --image=nginx:latest --restart=Never --env=VAR1=value1 --dry-run=client -o yaml > my-pod.yaml
  • Apply YAML:
kubectl apply -f my-pod.yaml
Getting Pods
kubectl get pods -o wide --watch
  • -o wide: Provides more detailed output.
  • --watch: Watches for changes in real-time.
Describing a Pod
kubectl describe pod my-pod

2. Deployments

2.1 Managing Deployments

Creating a Deployment
kubectl create deployment my-deployment --image=nginx:latest --replicas=2
  • --image nginx:latest: Specifies the container image.
  • --replicas 2: Number of desired replicas.

Declarative:

  • Generate YAML:
kubectl create deployment my-deployment --image=nginx:latest --replicas=2 --dry-run=client -o yaml > my-deployment.yaml
  • Apply YAML:
kubectl apply -f my-deployment.yaml
Scaling a Deployment
kubectl scale deployment my-deployment --replicas=5
  • --replicas 5: Sets the number of desired replicas.

Declarative:

  • Update YAML: Adjust replicas in my-deployment.yaml file.
  • Apply YAML:
kubectl apply -f my-deployment.yaml

3. Services

3.1 Creating a Service

kubectl expose deployment my-deployment --port=80 --type=ClusterIP
  • --port 80: Specifies the port number.
  • --type ClusterIP: Defines the type of service.

Declarative:

  • Generate YAML:
kubectl expose deployment my-deployment --port=80 --type=ClusterIP --dry-run=client -o yaml > my-service.yaml
  • Apply YAML:
kubectl apply -f my-service.yaml

4. Namespaces

4.1 Managing Namespaces

Creating a Namespace
kubectl create namespace my-namespace

Declarative:

  • Generate YAML:
kubectl create namespace my-namespace --dry-run=client -o yaml > my-namespace.yaml
  • Apply YAML:
kubectl apply -f my-namespace.yaml
Listing Namespaces
kubectl get namespaces

5. Configuration

5.1 Managing ConfigMaps and Secrets

Creating a ConfigMap
kubectl create configmap my-configmap --from-literal=key1=value1 --from-file=./config-file.txt
  • --from-literal key1=value1: Sets a key-value pair directly.
  • --from-file ./config-file.txt: Creates a ConfigMap from a file.

Declarative:

  • Generate YAML:
kubectl create configmap my-configmap --from-literal=key1=value1 --from-file=./config-file.txt --dry-run=client -o yaml > my-configmap.yaml

-

Apply YAML:

kubectl apply -f my-configmap.yaml
Creating a Secret
kubectl create secret generic my-secret --from-literal=key1=value1 --from-file=./secret-file.txt
  • --from-literal key1=value1: Sets a key-value pair for the secret.
  • --from-file ./secret-file.txt: Creates a Secret from a file.

Declarative:

  • Generate YAML:
kubectl create secret generic my-secret --from-literal=key1=value1 --from-file=./secret-file.txt --dry-run=client -o yaml > my-secret.yaml
  • Apply YAML:
kubectl apply -f my-secret.yaml

6. Monitoring and Logging

6.1 Getting Logs

kubectl logs my-pod -f --since=1h
  • -f: Follow log output in real-time.
  • --since 1h: Show logs since a certain time.

7. Jobs and CronJobs

7.1 Managing Jobs and CronJobs

Creating a Job
kubectl create job my-job --image=busybox
  • --image busybox: Specifies the container image.

Declarative:

  • Generate YAML:
kubectl create job my-job --image=busybox --dry-run=client -o yaml > my-job.yaml
  • Apply YAML:
kubectl apply -f my-job.yaml
Creating a CronJob
kubectl create cronjob my-cronjob --schedule="*/5 * * * *" --image=busybox
  • --schedule "*/5 * * * *": Sets the cron schedule in cron format.

Declarative:

  • Generate YAML:
kubectl create cronjob my-cronjob --schedule="*/5 * * * *" --image=busybox --dry-run=client -o yaml > my-cronjob.yaml
  • Apply YAML:
kubectl apply -f my-cronjob.yaml

8. Rolling Updates and Rollbacks

8.1 Managing Updates and Rollbacks

Updating a Deployment
kubectl set image deployment/my-deployment nginx=nginx:1.9.1

Declarative:

  • Update YAML: Adjust image in my-deployment.yaml.
  • Apply YAML:
kubectl apply -f my-deployment.yaml
Rolling Back a Deployment
kubectl rollout undo deployment/my-deployment

Declarative:

  • Use previous version of my-deployment.yaml.
  • Apply YAML:
kubectl apply -f my-deployment.yaml

9. Resource Management

9.1 Setting Resource Requests and Limits

kubectl set resources deployment/my-deployment --limits=cpu=200m,memory=512Mi --requests=cpu=100m,memory=256Mi
  • --limits cpu=200m,memory=512Mi and --requests cpu=100m,memory=256Mi: Set resource constraints.

Declarative:

  • Update YAML: Adjust resources in my-deployment.yaml.
  • Apply YAML:
kubectl apply -f my-deployment.yaml

10. Debugging

10.1 Diagnosing and Fixing Issues

Executing into a Container
kubectl exec -it my-pod -- /bin/bash

Declarative: Not applicable for exec command.

Port Forwarding
kubectl port-forward my-pod 8080:80
  • 8080:80: Forwards local port 8080 to the Pod's port 80.

Declarative: Not applicable for port-forward command.

Copying Files to/from a Container
kubectl cp /path/on/local/file.txt my-pod:/path/in/container/file.txt

Declarative: Not applicable for cp command.

11. Labels and Selectors

11.1 Managing Labels

Adding Labels to a Pod
kubectl label pods my-pod key1=value1 key2=value2
  • Adds labels key1=value1 and key2=value2 to my-pod.
Updating Labels of a Pod
kubectl label pods my-pod key1=value1 --overwrite
  • Updates the value of key1 to value1 on my-pod, overwriting if it exists.
Removing Labels from a Pod
kubectl label pods my-pod key1-
  • Removes the label key1 from my-pod.
Filtering Resources by Labels
kubectl get pods -l key1=value1,key2=value2
  • Lists all pods with labels key1=value1 and key2=value2.
Using Labels for Resource Management
  • Imperative:
  • Assigning a label:
kubectl label pods my-pod env=dev
  • Selecting resources:
kubectl get pods -l env=dev
  • Declarative:
  • Update YAML: Add labels under metadata.labels in resource definition files.
  • Apply YAML:
kubectl apply -f <resource-definition-file>.yaml

Terraform - HashiCorp Infrastructure Automation Certification

CKA

Exam Objectives

HashiCorp Certified: Terraform Associate

For in-depth information on Terraform, including certification details. visit the HashiCorp Certified: Terraform Associate page.

1. Understand Infrastructure as Code (IaC) Concepts

2. Understand the Purpose of Terraform

  • 2a. Explain multi-cloud and provider-agnostic benefits
  • 2b. Explain the benefits of state

3. Understand Terraform Basics

  • 3a. Install and version Terraform providers
  • 3b. Describe plugin-based architecture
  • 3c. Write Terraform configuration using multiple providers
  • 3d. Describe how Terraform finds and fetches providers

4. Use Terraform Outside of Core Workflow

  • 4a. Describe using terraform import to import existing infrastructure into your Terraform state
  • 4b. Use terraform state to view Terraform state
  • 4c. Describe enabling verbose logging and its value

5. Interact with Terraform Modules

  • 5a. Contrast and use different module source options including the public Terraform Module Registry
  • 5b. Interact with module inputs and outputs
  • 5c. Describe variable scope within modules/child modules
  • 5d. Set module version

6. Use the Core Terraform Workflow

  • 6a. Describe Terraform workflow (Write -> Plan -> Create)
  • 6b. Initialize a Terraform working directory (terraform init)
  • 6c. Validate a Terraform configuration (terraform validate)
  • 6d. Generate and review an execution plan for Terraform (terraform plan)
  • 6e. Execute changes to infrastructure with Terraform (terraform apply)
  • 6f. Destroy Terraform managed infrastructure (terraform destroy)
  • 6g. Apply formatting and style adjustments to a configuration (terraform fmt)

7. Implement and Maintain State

  • 7a. Describe default local backend
  • 7b. Describe state locking
  • 7c. Handle backend and cloud integration authentication methods
  • 7d. Differentiate remote state back end options
  • 7e. Manage resource drift and Terraform state
  • 7f. Describe backend block and cloud integration in configuration
  • 7g. Understand secret management in state files

8. Read, Generate, and Modify Configuration

  • 8a. Demonstrate use of variables and outputs
  • 8b. Describe secure secret injection best practice
  • 8c. Understand the use of collection and structural types
  • 8d. Create and differentiate resource and data configuration
  • 8e. Use resource addressing and resource parameters to connect resources together
  • 8f. Use HCL and Terraform functions to write configuration
  • 8g. Describe built-in dependency management (order of execution based)

9. Understand Terraform Cloud Capabilities

  • 9a. Explain how Terraform Cloud helps to manage infrastructure
  • 9b. Describe how Terraform Cloud enables collaboration and governance

Deploying and Managing MySQL with Helm in Kubernetes

CKAD

Overview

This guide explains how to deploy and manage the MySQL database using Helm in a Kubernetes environment. Helm, a package manager for Kubernetes, simplifies the process of managing Kubernetes applications.

Note

For detailed Helm installation instructions, refer to Installing Helm. Helm Charts package all the resource definitions necessary to deploy an application in Kubernetes.

Deploying MySQL with Helm

Helm streamlines the deployment of applications in Kubernetes, and here’s how you can use it to deploy MySQL:

1. Add a Helm Repository

First, add the Bitnami Helm repository which contains the MySQL chart:

helm repo add bitnami https://charts.bitnami.com/bitnami

2. Update the Repository

Ensure you have the latest charts by updating the repository:

helm repo update

3. Install MySQL Chart

Replace $MYSQL_ROOT_PASSWORD with your desired root password.

kubectl create ns my-database

export MYSQL_ROOT_PASSWORD=strong-password

To install the MySQL chart with a custom password:

helm install --set mysqlRootPassword=$MYSQL_ROOT_PASSWORD --set volumePermissions.enabled=true -n my-database my-mysql bitnami/mysql

The volumePermissions.enabled=true setting helps avoid potential permission issues with persistent volumes.

Tip

Use helm search repo [repository-name] to find available charts in a repository.

4. Intentionally Update to an Incompatible MySQL Image Tag

To simulate a real-world problem where an update might cause issues, let's intentionally update to an incompatible MySQL image tag:

helm upgrade my-mysql bitnami/mysql -n my-database --set image.tag=nonexistent

Info

The purpose of this command is to simulate a problematic update, allowing us to demonstrate the rollback process. This update intentionally uses a non-existent tag, which will cause the update to fail, resembling a common real-world issue.

5. Viewing Helm Release History

To view the history of the MySQL release:

helm history my-mysql -n my-database

6. Listing Installed Helm Charts

List all installed Helm charts in a specific namespace:

helm list -n my-database

7. Rolling Back a Helm Release

To rollback to the first version of the MySQL release:

helm rollback my-mysql 1 -n my-database

Caution

Rollbacks cannot be undone. Be sure of the revision number.

8. Uninstalling the MySQL Release

To remove the MySQL release:

helm uninstall my-mysql -n my-database

Conclusion

Using Helm to deploy and manage applications like MySQL in Kubernetes simplifies the process considerably. Following these steps, including addressing common deployment challenges like permission issues, will allow you to effectively manage MySQL in your Kubernetes clusters.

Restricting network access with UFW

Introduction

  • Uncomplicated Firewall (UFW) is a user-friendly interface for managing firewall rules in Linux distributions.
  • It simplifies the process of configuring the iptables firewall, providing an easy-to-use command-line interface.

Installation

  • UFW is typically installed by default on many Linux distributions.

  • If not installed, it can be easily installed using the package manager of your distribution.

    sudo apt-get install ufw   # For Ubuntu/Debian
sudo yum install ufw       # For CentOS/RHEL

Fundamentals

  • Basic Usage: Enable the firewall:

    sudo ufw enable

  • Disable the firewall:

    sudo ufw disable

  • Check the firewall status:

    sudo ufw status

Managing Rules

  • Allow incoming traffic on specific ports (e.g., SSH, HTTP, HTTPS)

    sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

  • Allow incoming traffic from specific IP addresses:

    sudo ufw allow from 192.168.1.100

  • Deny incoming traffic on specific ports:

    sudo ufw deny 25/tcp

  • Delete a rule:

    sudo ufw delete allow 22/tcp

Advanced Configuration

  • UFW supports more advanced configurations such as port ranges and specifying protocols.

    sudo ufw allow 8000:9000/tcp
sudo ufw allow proto udp to any port 53

Logging

  • UFW can log denied connections for troubleshooting purposes.

    sudo ufw logging on

Default Policies

  • By default, UFW denies all incoming connections and allows all outgoing connections.

  • Default policies can be changed if needed.

    sudo ufw default deny incoming
sudo ufw default allow outgoing

Integration with CKS Preparation

  • Understanding UFW is valuable for Certified Kubernetes Security Specialist (CKS) preparation.
  • CKS candidates may need to configure network policies and ingress/egress rules within Kubernetes clusters.
  • Knowledge of UFW can help in securing access to Kubernetes nodes and ensuring only necessary traffic is allowed.

Conclusion

  • Uncomplicated Firewall (UFW) is a powerful tool for managing firewall rules in Linux environments.
  • Its simplicity makes it suitable for both beginners and advanced users.
  • Understanding UFW is beneficial for CKS preparation, particularly for configuring network policies and securing Kubernetes clusters.

Understanding Kubeconfig in Kubernetes

What is Kubeconfig?

Kubeconfig is a configuration file used by kubectl and other Kubernetes tools to access and manage Kubernetes clusters. It contains information about clusters, users, contexts, and other settings needed to authenticate and communicate with Kubernetes clusters.

Location of Config File

By default, the kubeconfig file is located at:

  • Linux/Mac: $HOME/.kube/config
  • Windows: %USERPROFILE%\.kube\config

You can also specify a different location using the KUBECONFIG environment variable or the --kubeconfig flag with kubectl commands.

Clusters, Contexts, Users

A kubeconfig file typically contains several sections:

  • Clusters: Defines the Kubernetes clusters you can connect to. Each cluster entry includes the cluster name, server URL, and certificate authority data.

  • Contexts: Represents a combination of a cluster, a user, and a namespace. Contexts allow you to switch between different cluster-user combinations easily.

  • Users: Specifies the credentials needed to authenticate to a cluster. This can include client certificates, tokens, or other authentication methods.

Example Kubeconfig File

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /path/to/ca.crt
    server: https://your-kubernetes-cluster-server
  name: my-cluster

contexts:
- context:
    cluster: my-cluster
    namespace: default
    user: my-user
  name: my-context

current-context: my-context

kind: Config
preferences: {}
users:
- name: my-user
  user:
    client-certificate: /path/to/client.crt
    client-key: /path/to/client.key

Embedding Certificates in Kubeconfig

Instead of referring to certificate files, you can embed the certificate data directly in the kubeconfig file. This makes the configuration portable and easier to manage.

Example Kubeconfig File with Embedded Certificates

apiVersion: v1
kind: Config
clusters:
- name: my-cluster
  cluster:
    server: https://your-kubernetes-cluster-server
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURrVENDQWVrZ0F3SUJBZ0lKQU9XWFpXK0pqOTRmTUEwR0NTcUdTSWIzRFFFQkJRVUFNQTB4Q3pBSkJnTlYKQkFNTUdWUjBMbWgwZEhCekxtWnZiM1F1YzJWeU1TNW5jR0YwWVRBMU1TNHdPVEV6TVRJd1pEQXpNRm9YRFRNMwpNamM1T0RjeU1EWXhPRFl4TUZvWFRUUTVNamN4TkRreU1EWTBNelF3TkRFVk1CTUdBMVVFQXhNR2J6RXhN

contexts:
- name: my-context
  context:
    cluster: my-cluster
    user: my-user
    namespace: default

current-context: my-context

users:
- name: my-user
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURrVENDQWVrZ0F3SUJBZ0lKQU9XWFpXK0pqOTRmTUEwR0NTcUdTSWIzRFFFQkJRVUFNQTB4Q3pBSkJnTlYKQkFNTUdWUjBMbWgwZEhCekxtWnZiM1F1YzJWeU1TNW5jR0YwWVRBMU1TNHdPVEV6TVRJd1pEQXpNRm9YRFRNMwpNamM1T0RjeU1EWXhPRFl4TUZvWFRUUTVNamN4TkRreU1EWTBNelF3TkRFVk1CTUdBMVVFQXhNR2J6RXhN
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQo...

How to Embed Certificates

  1. Convert Certificate Files to Base64: Use the base64 command to encode your certificate and key files.
base64 -w 0 /path/to/ca.crt > ca.crt.base64
base64 -w 0 /path/to/client.crt > client.crt.base64
base64 -w 0 /path/to/client.key > client.key.base64

  1. Embed the Encoded Data in the Kubeconfig: Replace the file paths in your kubeconfig with the Base64-encoded data.

  2. For the certificate-authority-data key:

    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t...

  3. For the client-certificate-data key:

    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t...

  4. For the client-key-data key:

    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQo...

Example Commands

base64 -w 0 /path/to/ca.crt > ca.crt.base64
base64 -w 0 /path/to/client.crt > client.crt.base64
base64 -w 0 /path/to/client.key > client.key.base64

These commands will generate files containing the Base64-encoded data, which you can then copy and paste into your kubeconfig file under the appropriate keys.

By embedding the certificate data directly in the kubeconfig file, you make the configuration self-contained, which can be particularly useful for automation and sharing configurations across different environments

Current Context

The current context determines which cluster, user, and namespace are active when you run kubectl commands. It is specified in the current-context field of the kubeconfig file.

Viewing the Current Context

To view the current context:

kubectl config current-context

Setting the Current Context

To set the current context:

kubectl config use-context my-context

KubeConfig Management Commands

Here are some useful kubectl commands for working with kubeconfig:

  • View Current Context:
kubectl config current-context
  • Set Current Context:
kubectl config use-context <context-name>
  • View All Contexts:
kubectl config get-contexts
  • Switch to a Different Context:
kubectl config use-context <context-name>
  • View Cluster Information:
kubectl config view
  • Add a New Cluster:
kubectl config set-cluster <cluster-name> --server=<server-url> --certificate-authority=<path-to-ca.crt>
  • Add a New User:
kubectl config set-credentials <user-name> --client-certificate=<path-to-client.crt> --client-key=<path-to-client.key>
  • Add a New Context:
kubectl config set-context <context-name> --cluster=<cluster-name> --namespace=<namespace> --user=<user-name>

By understanding and managing the kubeconfig file, you can efficiently switch between different Kubernetes clusters and user configurations, making cluster management more streamlined and effective.

Securing Kubelet - A Guide for CKS

Introduction

The Certified Kubernetes Security Specialist (CKS) exam requires a deep understanding of securing the Kubelet, the primary "node agent" in Kubernetes. This guide covers key aspects of securing the Kubelet, including configuration, authentication, and best practices.

Kubelet Configuration File

The Kubelet's behavior is configured via a configuration file, typically found at:

  • Linux: /var/lib/kubelet/config.yaml
  • Windows: C:\var\lib\kubelet\config.yaml

Viewing the Configuration

To view the contents of the Kubelet configuration file, use:

cat /var/lib/kubelet/config.yaml

Viewing Kubelet Options

To see all available configuration options for the Kubelet, use:

kubelet --help

This command displays all command-line flags and options that can be used to configure the Kubelet.

Kubelet Serving Ports and Their Functions

The Kubelet uses several ports for different functions:

  • 10250: Main Kubelet API port for communication with the Kubernetes API server.
  • 10255: Read-only port for health checks and metrics (deprecated and should be disabled).

Example Kubelet Configuration

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
port: 10250
readOnlyPort: 0

Enable/Disable Anonymous Authentication in Kubelet

Anonymous authentication allows unauthenticated requests to access the Kubelet's API. For security, it is recommended to disable this feature.

Disabling Anonymous Authentication

To disable anonymous authentication, ensure the following configuration is set:

authentication:
  anonymous:
    enabled: false

Enabling Anonymous Authentication

If necessary, you can enable anonymous authentication by setting:

authentication:
  anonymous:
    enabled: true

After modifying the configuration file, restart the Kubelet to apply the changes:

sudo systemctl restart kubelet

Kubelet Authentication: Certificates and API Bearer Tokens

Kubelet supports multiple authentication methods to secure communication with the Kubernetes API server and other components.

Certificate-Based Authentication

Configure the Kubelet to use client certificates for authentication:

authentication:
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt

API Bearer Token Authentication

Configure the Kubelet to use API bearer tokens for authentication:

authentication:
  webhook:
    enabled: true
  token:
    enabled: true

Example Configuration with Authentication

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
serverTLSBootstrap: true
port: 10250
readOnlyPort: 0

Best Practices for Securing Kubelet

  1. Disable Read-Only Port: Ensure the read-only port (10255) is disabled.
  2. Enable Webhook Authorization: Use webhook authorization to enforce fine-grained access control.
  3. Use TLS: Always use TLS for secure communication.
  4. Rotate Certificates Regularly: Implement a process to rotate Kubelet certificates regularly.
  5. Restrict Node Access: Limit access to nodes and the Kubelet API to trusted sources only.

By following these practices, you can enhance the security of the Kubelet and maintain a secure Kubernetes environment, essential for passing the CKS exam and ensuring robust cluster security.

Guide to CIS Kubernetes Benchmarking with Kube-bench

What is CIS?

The Center for Internet Security (CIS) is a non-profit organization that develops globally recognized best practices for securing IT systems and data. CIS provides benchmarks, controls, and guidelines to help organizations improve their cybersecurity posture.

CIS Benchmark for Kubernetes

The CIS Kubernetes Benchmark provides a comprehensive set of guidelines for securing Kubernetes clusters. These benchmarks cover various aspects such as configuration, management, and monitoring, aiming to enhance the overall security of Kubernetes deployments.

You can download the CIS Kubernetes Benchmark from the official CIS website: CIS Kubernetes Benchmark Download.

Introduction to Kube-bench Tool

Kube-bench is an open-source tool developed by Aqua Security that automates the process of checking Kubernetes clusters against the CIS Kubernetes Benchmark. It provides a detailed report highlighting which controls are compliant and which need remediation.

Deploy Kube-bench Options

There are multiple ways to deploy Kube-bench in your Kubernetes environment:

  • Running as a Job: Execute Kube-bench as a Kubernetes job, which runs the checks and exits.
  • Running as a DaemonSet: Deploy Kube-bench as a DaemonSet to run on every node in the cluster.
  • Running locally: Run Kube-bench directly on the command line for individual nodes.
  • Kube-bench Installation
  • Killer Coda Labs

Run Kube-bench

kube-bench run --benchmark <benchmark-version> --targets master,node

Fix One Sample Issue

Let's fix an issue identified by Kube-bench:

Ensure that the --anonymous-auth argument is set to false (CIS 1.2.5).

Steps to Fix

1. Identify the Configuration File

Locate the kube-apiserver configuration file, usually found in /etc/kubernetes/manifests/kube-apiserver.yaml.

2. Edit the Configuration

Open the kube-apiserver.yaml file and add or modify the --anonymous-auth flag to be false.

- --anonymous-auth=false
3. Apply the Changes

Save the file and the kubelet will automatically restart the kube-apiserver with the new settings.

4. Verify the Fix

Run Kube-bench again to ensure that the issue is resolved.

kube-bench run --check 1.2.5

Conclusion

Using the CIS Kubernetes Benchmark and Kube-bench tool is a robust approach to enhance the security of your Kubernetes clusters. Regularly running these checks and addressing identified issues helps maintain a secure and compliant environment.

Certificate API

Info

Certificates and Certificate Signing Requests

Overview of what a Certificate Signing Request (CSR).

  1. Request for Certificate: A CSR is a request sent to a Certificate Authority (CA) asking for a digital certificate.

  2. Contains Public Key: It includes the applicant's public key along with identity information (e.g., common name, organization details).

  3. Signed by Private Key: The CSR is signed by the applicant's private key to prove ownership and authenticity of the public key.

  4. Establishes Trust: By submitting a CSR, the applicant seeks validation of their identity by the CA to establish trust.

  5. Enables Secure Communications: Once approved, the CA issues a digital certificate that binds the applicant's identity to their public key, enabling secure encrypted communications (e.g., HTTPS, SSL/TLS connections).

1. Create a CSR
  • First, generate a private key and CSR using OpenSSL:
openssl genrsa -out my-key.key 2048
openssl req -new -key my-key.key -out my-csr.csr -subj "/CN=my-user"
  • Encode CSR in Base64 Encode the CSR in base64 (to be included in the YAML):
cat my-csr.csr | base64 | tr -d '\n'
  • Next, create a Kubernetes CSR manifest (my-csr.yaml):
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: my-csr
spec:
  request: <encoded-base64-goes-here>
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
  • Apply the CSR manifest:
kubectl apply -f my-csr.yaml
2. Approve or Deny the CSR
  • To approve the CSR:
kubectl certificate approve my-csr
  • To deny the CSR:
kubectl certificate deny my-csr
3. List CSRs
  • List all CertificateSigningRequests:
kubectl get csr
4. View YAML and Describe CSR
  • View the CSR in YAML format:
kubectl get csr my-csr -o yaml
  • Describe the CSR:
kubectl describe csr my-csr
5. Delete the CSR
kubectl delete csr my-csr

Locating Certificates on the Master Node

Once a CSR is approved, the certificate will typically be signed by the Kubernetes Certificate Authority and returned. To locate certificates:

  • Certificates are generally stored in etcd and are not directly accessible from the file system. However, if you are managing custom certificates, they may be stored in specific directories like /etc/kubernetes/pki.

  • For cluster components (e.g., kube-apiserver, kubelet), certificates are usually stored in /etc/kubernetes/pki on the master node.

  • Example locations:

  • CA certificates: /etc/kubernetes/pki/ca.crt
  • API server certificates: /etc/kubernetes/pki/apiserver.crt
  • Kubelet certificates: /var/lib/kubelet/pki/kubelet-client-current.pem
Accessing the Master Node
  • SSH into the master node:
ssh user@master-node-ip
  • Navigate to the directory containing certificates:
cd /etc/kubernetes/pki
  • List the files to see the certificates:
ls -l

Additional Tips

  • Ensure you have appropriate permissions to manage CSRs and access the file system on the master node.
  • Regularly back up your certificate files and maintain secure practices around certificate management.
  • Refer to the Kubernetes documentation for detailed information on the CSR API and certificate management best practices.

By following these steps, you should be able to create, manage, and locate certificates effectively in a Kubernetes cluster, which is essential for your CKS certification preparation.

Verify Kubernetes Plantform Binaries

Directory Check and Listing

  • Change directory to the location where the binaries are stored.
  • List the files in the directory using ls

SHA-512 Sum Generation

  • Use sha512sum to generate the SHA-512 checksum for each binary

    1. sha512sum kube-apiserver
    2. sha512sum kube-controller-manager
    3. sha512sum kube-proxy
    4. sha512sum kubelet

Verification

  • Compare the generated checksums with the provided ones to ensure they match.
  • Identify any discrepancies.

Detailed Comparison for Verification

  • For a closer look at the kube-controller-manager checksum, redirect the output to a file: sha512sum kube-controller-manager > compare
  • Open the compare file in Vim: vim compare
  • Edit the file to ensure that each line contains only the checksum value.

  • Use cat and uniq to compare the contents of the compare file and ensure that there are no duplicate lines indicating a discrepancy:

    cat compare | uniq

Cleanup

  • Remove the binaries (kubelet and kube-controller-manager) that showed discrepancies in the checksums: rm kubelet kube-controller-manager

This process ensures that the binaries in the Kubernetes cluster match the expected checksums, helping to verify their integrity and authenticity.