Skip to content

Kubernetes

An open-source system for automating deployment, scaling, and management of containerized applications
Open sourced in 2014 by Google
Originates from internal project named Borg

The components

Control Plane Components

The control plane's components make global decisions about the cluster (for example, scheduling), as well as detecting and responding to cluster events.

  • kube-apiserver : the component that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane.

  • etcd (Stacked etcd topology ) : consistent and highly-available key value store used as Kubernetes' backing store for all cluster data.

  • kube-scheduler : the component that watches for newly created Pods with no assigned node, and selects a node for them to run on.

  • kube-controller-manager : the component that runs controller processes.

    • node controller : responsible for noticing and responding when nodes go down.
    • replication : manage correct number of pods.
    • endpoints : Populates the Endpoints object (that is, joins Services & Pods).
    • Service Account & Token Controllers : Create default accounts and API access tokens for new namespaces.
    • etc ...
  • cloud-controller-manager : the component that embeds cloud-specific control logic

Nodes Components

Node components run on every node, maintaining running pods and providing the Kubernetes runtime environment.

  • kubelet: agent that runs on each node in the cluster. It makes sure that containers are running in a pod.

  • kube-proxy: enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding.

    • For large workloard you can use IPVS mode instead of iptables mode.
  • Container Runtime: responsible for running containers: Docker(mostly), containerd.

  • Addons: https://kubernetes.io/docs/concepts/cluster-administration/addons/.

    • DNS : required
    • Network Plugins: CNI (Cilium, Canal, Calico, ...)
    • Web UI (Dashboard).
    • Monitoring resources.
    • etc...

Pods

Containers in a pod share an IP address and port space, and can communicate via localhost
OR can use inter-process communications(IPCs)

Create - Manage

Minikube

Some tools such as minikube, kind, k3s let you run a lightweight kubernetes locally.

Helm - Helmfile

Helm/Helmfile the package manager for Kubernetes.

Kubectl

Install kubectl for Linux
  curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

  chmod +x kubectl

kubectl Basic Commands

kubectl Basic Commands
  kubectl cluster-info

  kubectl apply -f ./my_file.yml

  kubectl config get-contexts
  kubectl config current-context
  kubectl config use-context xxxxmycontext

  # apply default namespace to curent conext
  kubectl config set-context --current --namespace=my_app

  kubectl get deployments
  kubectl get deployment deployment-my_app
  kubectl describe deployments
  kubectl describe deployment deployment-my_app

  kubectl get rs
  kubectl get pods -o wide
  kubectl get pods -l=app=my_app
  kubectl get pods -l 'environment in (production, qa)'
  kubectl get pods --field-selector=status.phase=Running

  kubectl exec -ti pod_name -- /bin/sh
  kubectl logs --previous pod_na
  kubectl exec my-pod -- ls /

  kubectl delete pods,services -l name=myLabel
  kubectl delete -f deployment.yaml

  kubectl edit type_obj(=ingress, service...) objet_name

  kubectl get services --sort-by=.metadata.name # List Services Sorted by Name
Prepare Node for maintenance
  kubectl cordon  node # Mark node as unschedulable
  kubectl drain node --ignore-daemonsets # Drain node in preparation for maintenance
  kubectl uncordon  node # Mark node as schedulable

Other examples of kubectl usage

In rancher section

If you have to create kubeconfig manually
      K8S_SERVER="https://k8s_example.com"
      K8S_CONTEXT_USER="username"
      NAMESPACE="xxx"
      SERVICE_TOKEN=""
      CLUSTER="xxx"

      echo "$K8S_CA_CRT" > ca.crt
      kubectl config set-cluster $CLUSTER --server=$K8S_SERVER --certificate-authority=ca.crt

      kubectl config set-credentials $K8S_CONTEXT_USER --certificate-authority=ca.crt --token=$SERVICE_TOKEN

      kubectl config set-context dev-context --cluster=$CLUSTER --user=$K8S_CONTEXT_USER --namespace=$NAMESPACE

      kubectl config use-context dev-context
  • OIDC authentification : https://kubernetes.io/docs/reference/access-authn-authz/authentication/#option-1---oidc-authenticator

Create Secret

https://v1-13.docs.kubernetes.io/docs/concepts/configuration/secret/

Create secret
      # Create manually
          echo -n 'admin' | base64
          echo -n 'pwd' | base64
          echo 'MWYyZDFlMmU2N2Rm' | base64 --decode

      # write object in secret.yaml
      cat > secret.yaml <<EOF
      apiVersion: v1
      kind: Secret
      metadata:
        name: mysecret
      type: Opaque
      data:
        username: YWRtaW4=
        password: MWYyZDFlMxxxxN2Rm
      EOF
      kubectl apply -f ./secret.yaml

      # create from file
      # Create files needed for rest of example.
      echo -n 'admin' > ./username.txt
      echo -n '1f2d1xxxe67df' > ./password.txt
      kubectl create secret generic basic-secret --from-file=./username.txt --from-file=./password.txt
Use Secret
    ##  Expose secret as Env variable
    env:
      - name: SECRET_USERNAME
        valueFrom:
          secretKeyRef:
            name: basic-secret
            key: username

    echo $SECRET_USERNAME ## test inside a container

    ## Use secret as file from a pod

    containers:
      - image: nginx:latest
        name: nginx
        volumeMounts:
        - name: mounted-basic-secret
          mountPath: "/opt/nginx/"
          readOnly: true
    volumes:
    - name: basic-secret
      secret:
        secretName: basic-auth
        # defaultMode: 256  # ~= 0400
        volumes:
        - name: mounted-basic-secret
          secret:
            secretName: basic-secret  # username & password files will be mounted
            # to mount only specif key as file, use items
            # items:
            #  - key: username # only username  will be mounted as username.txt
            #    path: username.txt

Docker Registry Secret

Create & Use
    kubectl create secret docker-registry registry-secret --docker-username="isda" \
    --docker-password="nexusisda" --docker-server="registry.example.com" --namespace dev

    kubectl get secret registry-secret -o yaml --namespace dev

    ## Add in your manifest - in container block
      imagePullSecrets:
        - name: registry-secret

Ingress - tls secret -certificate

Create & Add
    kubectl create secret tls mon_certif-tls --key my_certificate.key --cert my_certificate.cer -n mon_namespace

    # To add in Ingress
      hosts: app.ingress.com
    tls:
    - secretName: chart-example-tls
      hosts:
          - kibana.my_app.caas-cnp-apps.com.intraorange

Ingress Nginx Basic Authentification

Create basic token
    htpasswd -c auth foo
    New password: <bar>
    New password:
    Re-type new password:
    Adding password for user foo
Create a secret resource
    kubectl create secret generic basic-auth --from-file=auth
    secret "basic-auth" created

    kubectl get secret basic-auth -o yaml
    apiVersion: v1
    data:
      auth: Zm9vOiRhcHIxJE9GRzNYeWJwJGNrTDBGSERBa29YWUlsSDkuY3lzVDAK
    kind: Secret
    metadata:
      name: basic-auth
      namespace: default
    type: Opaque
Add annotation in Ingress
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: ingress-with-auth
      annotations:
        # type of authentication
        nginx.ingress.kubernetes.io/auth-type: basic
        # name of the secret that contains the user/password definitions
        nginx.ingress.kubernetes.io/auth-secret: basic-auth
        # message to display with an appropriate context why the authentication is required
        nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
    spec:

Storage

https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes

Create and mount a conf dir/folder - configMap
      kubectl create configmap <map-name> <data-source>
      kubectl create configmap myconf --from-file=/path/to/

      containers:
        - name: test
          image: busybox
          volumeMounts:
            - name: config-vol
              mountPath: /etc/config
      volumes:
        - name: config-vol
          configMap:
            name: log-config
            items:
              - key: log_level
                path: log_level
To mount a specific file contained in configmap - subpath
      volumeMounts:
          - name: config-vol
            mountPath: /etc/config/test.cfg
            subPath: test.cfg
Create and mount a volume object like with Docker - emptyDir
      volumes:
      - name: cache-volume
        emptyDir: {}
Mount a server folder - hostPath
      volumes:
        - name: test-volume
          hostPath:
            # directory location on host
            path: /data
            # this field is optional
            type: Directory

Create and mount a storage block/space - local or Cloud type
PersistentVolume nodeAffinity is required when using local volumes.

View
      apiVersion: v1
      kind: PersistentVolume
      metadata:
        name: example-pv
      spec:
        capacity:
          storage: 100Gi
        # volumeMode field requires BlockVolume Alpha feature gate to be enabled.
        volumeMode: Filesystem
        accessModes:
        - ReadWriteOnce
        persistentVolumeReclaimPolicy: Delete
        storageClassName: local-storage
        local:
          path: /mnt/disks/ssd1
        nodeAffinity:
          required:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - example-node
A simple way - persistentVolumeClaim
      #  to manage persistent volumes: local, cloud blocks

        apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: wp-pv-claim
          labels:
            app: wordpress
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 5Gi
All volumes in one - projected
      # A projected volume maps several existing volume sources(secret, configMap) into the same dir

      volumes:
      - name: all-in-one
        projected:
          sources:
          - secret:
              name: mysecret
              items:
                - key: username
                  path: my-group/my-username
          - configMap:
              name: myconfigmap
              items:
                - key: config
                  path: my-group/my-config

Kubernetes Cluster Management Tools

Some interesting tools to create, manage a Kubernetes clusters across various platform - hybrid.

A tool from Kubernetes that provide best-practice "fast paths", domain Knowledge of Kubernetes clusters' life cycle management
for creating Kubernetes clusters.
My notes/examples related to Kubeadm

Ansible Automation for K8S
Kubespray runs on bare metal and most clouds,
using Ansible as its substrate for provisioning and orchestration.

Kubespray VS Kops & Kubeadm

Kops(Kubernetes operations) helps deploy, destroy, update, and maintain highly
available and production-grade Kubernetes clusters using the CLI kops as gcloud container on Google Cloud
It can also generate terraform configurations for your IAC purpose

However, Kops is more tightly integrated with the unique features of the clouds it supports.
Ex: Currently OpenStack support is Beta mode.

Mutilpe clusters management platform, Rancher addresses the operational and security challenges of
managing multiple Kubernetes clusters across any infrastructure. It offers:

  • Unified multi-cluster management
  • Centralized App Catalog
  • Consistent security policy and compliance
  • Hybrid & multi-cloud support

Some Rancher notes

Gardener to build your own Kubernetes as a Service and deliver fully managed clusters

I need to test it on Openstack Platform !