Kubernetes
An open-source system for automating deployment, scaling, and management of containerized applications
Open sourced in 2014 by Google
Originates from internal project named Borg
-
Overview : https://kubernetes.io/fr/docs/concepts/overview/what-is-kubernetes/
- Kubeadm Cluster Administration :
- Admission Controllers: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
- Nubenetes : Awesome Kubernetes & Cloud Docs.
The components
Control Plane Components
The control plane's components make global decisions about the cluster (for example, scheduling), as well as detecting and responding to cluster events.
-
kube-apiserver : the component that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane.
-
etcd (Stacked etcd topology ) : consistent and highly-available key value store used as Kubernetes' backing store for all cluster data.
- The etcd cluster can be separated from the control plane nodes: External etcd topology
-
kube-scheduler : the component that watches for newly created Pods with no assigned node, and selects a node for them to run on.
-
kube-controller-manager : the component that runs controller processes.
- node controller : responsible for noticing and responding when nodes go down.
- replication : manage correct number of pods.
- endpoints : Populates the Endpoints object (that is, joins Services & Pods).
- Service Account & Token Controllers : Create default accounts and API access tokens for new namespaces.
- etc ...
- cloud-controller-manager : the component that embeds cloud-specific control logic
Nodes Components
Node components run on every node, maintaining running pods and providing the Kubernetes runtime environment.
-
kubelet: agent that runs on each node in the cluster. It makes sure that containers are running in a pod.
-
kube-proxy: enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding.
- For large workloard you can use IPVS mode instead of iptables mode.
-
Container Runtime: responsible for running containers: Docker(mostly), containerd.
-
Addons:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
.- DNS : required
- Network Plugins: CNI (Cilium, Canal, Calico, ...)
- Web UI (Dashboard).
- Monitoring resources.
- etc...
Pods
Containers in a pod share an IP address and port space, and can communicate via localhost
OR can use inter-process communications(IPCs)
Create - Manage
Minikube
Some tools such as minikube, kind, k3s let you run a lightweight kubernetes locally.
Helm - Helmfile
Helm/Helmfile the package manager for Kubernetes.
Kubectl
Install kubectl for Linux
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod +x kubectl
kubectl Basic Commands
kubectl cluster-info
kubectl apply -f ./my_file.yml
kubectl config get-contexts
kubectl config current-context
kubectl config use-context xxxxmycontext
# apply default namespace to curent conext
kubectl config set-context --current --namespace=my_app
kubectl get deployments
kubectl get deployment deployment-my_app
kubectl describe deployments
kubectl describe deployment deployment-my_app
kubectl get rs
kubectl get pods -o wide
kubectl get pods -l=app=my_app
kubectl get pods -l 'environment in (production, qa)'
kubectl get pods --field-selector=status.phase=Running
kubectl exec -ti pod_name -- /bin/sh
kubectl logs --previous pod_na
kubectl exec my-pod -- ls /
kubectl delete pods,services -l name=myLabel
kubectl delete -f deployment.yaml
kubectl edit type_obj(=ingress, service...) objet_name
kubectl get services --sort-by=.metadata.name # List Services Sorted by Name
Prepare Node for maintenance
kubectl cordon node # Mark node as unschedulable
kubectl drain node --ignore-daemonsets # Drain node in preparation for maintenance
kubectl uncordon node # Mark node as schedulable
Other examples of kubectl
usage
If you have to create kubeconfig manually
K8S_SERVER="https://k8s_example.com"
K8S_CONTEXT_USER="username"
NAMESPACE="xxx"
SERVICE_TOKEN=""
CLUSTER="xxx"
echo "$K8S_CA_CRT" > ca.crt
kubectl config set-cluster $CLUSTER --server=$K8S_SERVER --certificate-authority=ca.crt
kubectl config set-credentials $K8S_CONTEXT_USER --certificate-authority=ca.crt --token=$SERVICE_TOKEN
kubectl config set-context dev-context --cluster=$CLUSTER --user=$K8S_CONTEXT_USER --namespace=$NAMESPACE
kubectl config use-context dev-context
- OIDC authentification :
https://kubernetes.io/docs/reference/access-authn-authz/authentication/#option-1---oidc-authenticator
Create Secret
https://v1-13.docs.kubernetes.io/docs/concepts/configuration/secret/
Create secret
# Create manually
echo -n 'admin' | base64
echo -n 'pwd' | base64
echo 'MWYyZDFlMmU2N2Rm' | base64 --decode
# write object in secret.yaml
cat > secret.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMxxxxN2Rm
EOF
kubectl apply -f ./secret.yaml
# create from file
# Create files needed for rest of example.
echo -n 'admin' > ./username.txt
echo -n '1f2d1xxxe67df' > ./password.txt
kubectl create secret generic basic-secret --from-file=./username.txt --from-file=./password.txt
Use Secret
## Expose secret as Env variable
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: basic-secret
key: username
echo $SECRET_USERNAME ## test inside a container
## Use secret as file from a pod
containers:
- image: nginx:latest
name: nginx
volumeMounts:
- name: mounted-basic-secret
mountPath: "/opt/nginx/"
readOnly: true
volumes:
- name: basic-secret
secret:
secretName: basic-auth
# defaultMode: 256 # ~= 0400
volumes:
- name: mounted-basic-secret
secret:
secretName: basic-secret # username & password files will be mounted
# to mount only specif key as file, use items
# items:
# - key: username # only username will be mounted as username.txt
# path: username.txt
Docker Registry Secret
Create & Use
kubectl create secret docker-registry registry-secret --docker-username="isda" \
--docker-password="nexusisda" --docker-server="registry.example.com" --namespace dev
kubectl get secret registry-secret -o yaml --namespace dev
## Add in your manifest - in container block
imagePullSecrets:
- name: registry-secret
Ingress - tls secret -certificate
Create & Add
kubectl create secret tls mon_certif-tls --key my_certificate.key --cert my_certificate.cer -n mon_namespace
# To add in Ingress
hosts: app.ingress.com
tls:
- secretName: chart-example-tls
hosts:
- kibana.my_app.caas-cnp-apps.com.intraorange
Ingress Nginx Basic Authentification
Create basic token
htpasswd -c auth foo
New password: <bar>
New password:
Re-type new password:
Adding password for user foo
Create a secret resource
kubectl create secret generic basic-auth --from-file=auth
secret "basic-auth" created
kubectl get secret basic-auth -o yaml
apiVersion: v1
data:
auth: Zm9vOiRhcHIxJE9GRzNYeWJwJGNrTDBGSERBa29YWUlsSDkuY3lzVDAK
kind: Secret
metadata:
name: basic-auth
namespace: default
type: Opaque
Add annotation in Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
Storage
https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes
Create and mount a conf dir/folder - configMap
kubectl create configmap <map-name> <data-source>
kubectl create configmap myconf --from-file=/path/to/
containers:
- name: test
image: busybox
volumeMounts:
- name: config-vol
mountPath: /etc/config
volumes:
- name: config-vol
configMap:
name: log-config
items:
- key: log_level
path: log_level
To mount a specific file contained in configmap - subpath
volumeMounts:
- name: config-vol
mountPath: /etc/config/test.cfg
subPath: test.cfg
Create and mount a volume object like with Docker - emptyDir
volumes:
- name: cache-volume
emptyDir: {}
Mount a server folder - hostPath
volumes:
- name: test-volume
hostPath:
# directory location on host
path: /data
# this field is optional
type: Directory
Create and mount a storage block/space - local or Cloud type
PersistentVolume nodeAffinity is required when using local volumes.
View
apiVersion: v1
kind: PersistentVolume
metadata:
name: example-pv
spec:
capacity:
storage: 100Gi
# volumeMode field requires BlockVolume Alpha feature gate to be enabled.
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
local:
path: /mnt/disks/ssd1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- example-node
A simple way - persistentVolumeClaim
# to manage persistent volumes: local, cloud blocks
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wp-pv-claim
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
All volumes in one - projected
# A projected volume maps several existing volume sources(secret, configMap) into the same dir
volumes:
- name: all-in-one
projected:
sources:
- secret:
name: mysecret
items:
- key: username
path: my-group/my-username
- configMap:
name: myconfigmap
items:
- key: config
path: my-group/my-config
Kubernetes Cluster Management Tools
Some interesting tools to create, manage a Kubernetes clusters across various platform - hybrid.
A tool from Kubernetes that provide best-practice "fast paths", domain Knowledge of Kubernetes clusters' life cycle management
for creating Kubernetes clusters.
My notes/examples related to Kubeadm
Ansible Automation for K8S
Kubespray runs on bare metal and most clouds,
using Ansible as its substrate for provisioning and orchestration.
Kops(Kubernetes operations) helps deploy, destroy, update, and maintain highly
available and production-grade Kubernetes clusters using the CLI kops
as gcloud container
on Google Cloud
It can also generate terraform
configurations for your IAC purpose
However, Kops is more tightly integrated with the unique features of the clouds it supports.
Ex: Currently OpenStack support is Beta mode.
Mutilpe clusters management platform, Rancher addresses the operational and security challenges of
managing multiple Kubernetes clusters across any infrastructure. It offers:
- Unified multi-cluster management
- Centralized App Catalog
- Consistent security policy and compliance
- Hybrid & multi-cloud support
Some Rancher notes
Gardener to build your own Kubernetes as a Service and deliver fully managed clusters
I need to test it on Openstack Platform !