Kubernetes First App

Init

export KUBECONFIG=/etc/kubernetes/admin.conf

Nginx

Create nginx-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - image: nginx
          name: nginx

Apply nginx-deploy.yaml

kubectl apply -f nginx-deploy.yaml

Check deployment

kubectl get deployment.apps/nginx

Describe deployment

kubectl describe deployment.apps/nginx

Results

Name:                   nginx
Namespace:              default
CreationTimestamp:      Fri, 17 Dec 2021 19:48:51 +0800
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=nginx
Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      False   MinimumReplicasUnavailable
  Progressing    True    ReplicaSetUpdated
OldReplicaSets:  <none>
NewReplicaSet:   nginx-6799fc88d8 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  79s   deployment-controller  Scaled up replica set nginx-6799fc88d8 to 1

From the result, we can see a replica set was created by deployment-controller

Check replica set

kubectl get replicaset  nginx-6799fc88d8

Describe replica set

kubectl describe replicaset nginx-6799fc88d8

Result

Name:           nginx-6799fc88d8
Namespace:      default
Selector:       app=nginx,pod-template-hash=6799fc88d8
Labels:         app=nginx
                pod-template-hash=6799fc88d8
Annotations:    deployment.kubernetes.io/desired-replicas: 1
                deployment.kubernetes.io/max-replicas: 2
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/nginx
Replicas:       1 current / 1 desired
Pods Status:    1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=nginx
           pod-template-hash=6799fc88d8
  Containers:
   nginx:
    Image:        nginx
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age    From                   Message
  ----    ------            ----   ----                   -------
  Normal  SuccessfulCreate  3m29s  replicaset-controller  Created pod: nginx-6799fc88d8-5pvr8

From the result, we can see a pod was created by replicaset-controller

Check the pod

kubectl get pod  nginx-6799fc88d8-5pvr8 -owide

Result

NAME                     READY   STATUS    RESTARTS   AGE     IP              NODE             NOMINATED NODE   READINESS GATES
nginx-6799fc88d8-5pvr8   1/1     Running   0          5m10s   10.244.145.65   ecs-75909-0003   <none>           <none>

Validate the pod

curl 10.244.145.65

Result

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Replica Set

Change replica set to 3

kubectl edit deploy nginx

Check pods

kubectl get pods -owide

Result (NOTE: not balanced on three machines)

NAME                     READY   STATUS    RESTARTS   AGE     IP              NODE             NOMINATED NODE   READINESS GATES
nginx-6799fc88d8-5pvr8   1/1     Running   0          46m     10.244.145.65   ecs-75909-0003   <none>           <none>
nginx-6799fc88d8-t7rx5   1/1     Running   0          2m14s   10.244.145.66   ecs-75909-0003   <none>           <none>
nginx-6799fc88d8-vbjvc   1/1     Running   0          2m14s   10.244.145.67   ecs-75909-0003   <none>           <none>

Service

Get a selector

kubectl get pods --show-labels

Result

NAME                     READY   STATUS    RESTARTS   AGE     LABELS
nginx-6799fc88d8-5pvr8   1/1     Running   0          51m     app=nginx,pod-template-hash=6799fc88d8
nginx-6799fc88d8-t7rx5   1/1     Running   0          6m58s   app=nginx,pod-template-hash=6799fc88d8
nginx-6799fc88d8-vbjvc   1/1     Running   0          6m58s   app=nginx,pod-template-hash=6799fc88d8

Expose 3 pods as a single service

kubectl expose deploy nginx --selector app=nginx --port=80 --type=NodePort

Get service

kubectl get service

Result

NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.1.0.1      <none>        443/TCP        36d
nginx        NodePort    10.1.25.160   <none>        80:30884/TCP   2m46s

Validate result

curl 10.1.25.160

Result

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Kubeadm, kubectl, kubelet

Check kubectl logs

kubectl get ns default -v 9

Check config file

cat /etc/kubernetes/admin.conf

Add key from aliyun

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - 

Add source form aliyun

sudo apt-add-repository "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main"

Update again

sudo apt-get update

Install all

sudo apt install kubelet kubeadm kubectl

Disable auto update

sudo apt-mark hold kubeadm kubelet kubectl

Test kub...

kubelet --version

Disable swap

sudo swapoff -a

Free

free -h

Add conf

vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Add line

Environment="KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --fail-swap-on=false"

Reload daemon

systemctl daemon-reload

Restart kubelet

systemctl restart kubelet

Kubeadm init on master

kubeadm init --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --service-cidr=10.1.0.0/16 --kubernetes-version v1.22.3

Export

export KUBECONFIG=/etc/kubernetes/admin.conf

Join on other node

kubeadm join 192.168.0.18:6443 --token 2pwout.v81y9ss6f4fjed87 \
    --discovery-token-ca-cert-hash sha256:20bfbde6985f6337e7585742572707b72b4dd4da6c1a4af9260c8295129c8d01

Validate system

watch kubectl get pods -n kube-system

Check nodes

kubectl get nodes -o wide

Example results

NAME             STATUS   ROLES                  AGE    VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
ecs-75909-0001   Ready    <none>                 123m   v1.22.3   192.168.0.61   <none>        Ubuntu 18.04.5 LTS   4.15.0-136-generic   docker://20.10.10
ecs-75909-0002   Ready    control-plane,master   136m   v1.22.3   192.168.0.18   <none>        Ubuntu 18.04.5 LTS   4.15.0-136-generic   docker://20.10.10
ecs-75909-0003   Ready    <none>                 122m   v1.22.3   192.168.0.67   <none>        Ubuntu 18.04.5 LTS   4.15.0-136-generic   docker://20.10.10

Calico (Only on master)

Create tigera-operator.yaml

kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml

Download file

https://docs.projectcalico.org/manifests/custom-resources.yaml

Change cidr ip to 10.244.0.0/16 in the file

Create pods

kubectl create -f custom-resources.yaml

Validate system

watch kubectl get pods -n calico-system

Validate all

watch kubectl get pods --all-namespaces

Example results

NAMESPACE          NAME                                       READY   STATUS    RESTARTS   AGE
calico-apiserver   calico-apiserver-6fb4f77d54-ljbgg          1/1     Running   0          116m
calico-apiserver   calico-apiserver-6fb4f77d54-wxx96          1/1     Running   0          116m
calico-system      calico-kube-controllers-7bbdc8dbd7-s9xrb   1/1     Running   0          124m
calico-system      calico-node-b6rg5                          1/1     Running   0          117m
calico-system      calico-node-dsx2r                          1/1     Running   0          116m
calico-system      calico-node-t4xwh                          1/1     Running   0          124m
calico-system      calico-typha-879f64854-jbh8c               1/1     Running   0          116m
calico-system      calico-typha-879f64854-rmgdw               1/1     Running   0          124m
kube-system        coredns-7f6cbbb7b8-76dwt                   1/1     Running   0          130m
kube-system        coredns-7f6cbbb7b8-f74v2                   1/1     Running   0          130m
kube-system        etcd-ecs-75909-0002                        1/1     Running   0          131m
kube-system        kube-apiserver-ecs-75909-0002              1/1     Running   0          131m
kube-system        kube-controller-manager-ecs-75909-0002     1/1     Running   0          131m
kube-system        kube-proxy-8vzwg                           1/1     Running   0          117m
kube-system        kube-proxy-jchsf                           1/1     Running   0          116m
kube-system        kube-proxy-jnkbs                           1/1     Running   0          130m
kube-system        kube-scheduler-ecs-75909-0002              1/1     Running   0          131m
tigera-operator    tigera-operator-78b8976b89-6h86f           1/1     Running   0          130m

Taint master (Optional)

kubectl taint nodes --all node-role.kubernetes.io/master-

Debug

Reset kubeadm

kubeadm reset -f

Remove pod

kubectl delete -f https://docs.projectcalico.org/manifests/custom-resources.yaml