Kubernetes的安装过程还是比较繁琐的，网上也提供了不少的教程，总结下来安装方式大概可以分为三类。

  • 二进制安装（上手难度高，不推荐）
  • kubeadm安装（官方推荐，基于命令行形式，安装过程相对简化）
  • 基于第三方工具（以容器安装容器，安装简单，但是也屏蔽了安装过程，快速体验可以采用这种方式安装）

这里使用腾讯云服务器来安装，准备了以下四台节点。其中三台服务器作为k8s集群节点，一台服务器辅助安装使用，在使用kubeadm安装时非必要，使用第三方辅助工具时候需要用到。

服务器配置 IP地址 节点名称 备注
2核心4G 10.0.4.17 master 集群主节点
2核心4G 10.0.4.5 node1 集群工作节点
2核心4G 10.0.4.8 node2 集群工作节点
2核心2G 10.0.16.15 support 辅助节点（非集群必须）

注：以下安装操作使用节点名称来代替具体的服务器

本篇安装教程主要包含两部分内容，k8s集群的安装方式以及web管理界面的安装方式。

k8s集群安装

KuboardSpray

KuboardSpray是一个图形化的 K8S 集群离线安装、维护工具，可以通过Docker启动，并且提供图形化配置功能，实现一键安装k8s集群。
首先support节点需要提供Docker支持，在support节点上使用Docker安装KuboardSpray，执行以下命令：

  1. docker run -d \
  2.   --privileged \
  3.   --restart=unless-stopped \
  4.   --name=kuboard-spray \
  5.   -p 80:80/tcp \
  6.   -e TZ=Asia/Shanghai \
  7.   -v /var/run/docker.sock:/var/run/docker.sock \
  8.   -v ~/kuboard-spray-data:/data \
  9.   eipwork/kuboard-spray:latest-amd64
  10.   # 如果您是 arm64 环境，请将标签里的 amd64 修改为 arm64，例如 eipwork/kuboard-spray:latest-arm64
  11.   # 如果抓不到这个镜像，可以尝试一下这个备用地址：
  12.   # swr.cn-east-2.myhuaweicloud.com/kuboard/kuboard-spray:latest-amd64

安装启动，默认登录信息admin/Kuboard123，登录后界面如下所示，可以修改密码
image.png
选择系统设置—>资源包管理，列出了一些列所支持的k8s版本，此处选择v1.23.1。
image.png
等待资源安装过程，安装完成如下所示
image.png
返回主界面，选择集群管理，配置集群信息
image.png
安装前注意事项：

安装后的k8s集群软件版本为：

  • Kubernetes v1.21.x
    • calico 3.17.1
    • nginx-ingress 1.9.1
  • Containerd.io 1.4.3

确保没有问题后，开始准备安装。等待安装过程，取决网络速度，安装containerd/kubelet/kubeadm/kubectl。
image.png
当出现如下提示信息时，k8s集群安装完成。安装内容主要包含

  • master节点
    • kubeadm、kubectl、kubelet
  • node1/node2节点
    • kubectl、kubelet
    • kubectl、kubelet

image.png

Kubeadm

kubeadm 是 Kubernetes 官方支持的安装方式，“二进制” 不是。关于使用kubeadm安装的方式就不再赘述，这里推荐kuboard官方提供的安装教程：

关于安装版本的说明：

  • Kubernetes v1.21 开始，默认移除 docker 的依赖，如果宿主机上安装了 docker 和 containerd，将优先使用 docker 作为容器运行引擎，如果宿主机上未安装 docker 只安装了 containerd，将使用 containerd 作为容器运行引擎

  • 本文使用 containerd 作为容器运行引擎

    Dashboard安装

    k8s提供了强大的集群管理功能，但是对于使用kubectl操作的方式过于专业，因此官方和第三方都提供了web图形化界面支持的工具，下面介绍两种安装方式。

    Kubernetes Dashboard

    Kubernetes Dashboard 是 Kubernetes 的官方 Web UI。使用 Kubernetes Dashboard，您可以：

  • 向 Kubernetes 集群部署容器化应用

  • 诊断容器化应用的问题
  • 管理集群的资源
  • 查看集群上所运行的应用程序
  • 创建、修改Kubernetes 上的资源（例如 Deployment、Job、DaemonSet等）
  • 展示集群上发生的错误

下面介绍如何安装，使用下面命令获取安装模板yaml文件，这里使用v2.5.0版本的dashboard。

  1. wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.0/aio/deploy/recommended.yaml

下载yaml文件后，注意修改service kubernetes-dashboard的类型为NodePort，可以在集群外部访问。

  1. # Copyright 2017 The Kubernetes Authors.
  2. #
  3. # Licensed under the Apache License, Version 2.0 (the "License");
  4. # you may not use this file except in compliance with the License.
  5. # You may obtain a copy of the License at
  6. #
  7. #     http://www.apache.org/licenses/LICENSE-2.0
  8. #
  9. # Unless required by applicable law or agreed to in writing, software
  10. # distributed under the License is distributed on an "AS IS" BASIS,
  11. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  12. # See the License for the specific language governing permissions and
  13. # limitations under the License.
  15. apiVersion: v1
  16. kind: Namespace
  17. metadata:
  18.   name: kubernetes-dashboard
  20. ---
  22. apiVersion: v1
  23. kind: ServiceAccount
  24. metadata:
  25.   labels:
  26.     k8s-app: kubernetes-dashboard
  27.   name: kubernetes-dashboard
  28.   namespace: kubernetes-dashboard
  30. ---
  32. kind: Service
  33. apiVersion: v1
  34. metadata:
  35.   labels:
  36.     k8s-app: kubernetes-dashboard
  37.   name: kubernetes-dashboard
  38.   namespace: kubernetes-dashboard
  39. spec:
  40.   ports:
  41.     - port: 443
  42.       targetPort: 8443
  43.   selector:
  44.     k8s-app: kubernetes-dashboard
  46. ---
  48. apiVersion: v1
  49. kind: Secret
  50. metadata:
  51.   labels:
  52.     k8s-app: kubernetes-dashboard
  53.   name: kubernetes-dashboard-certs
  54.   namespace: kubernetes-dashboard
  55. type: Opaque
  57. ---
  59. apiVersion: v1
  60. kind: Secret
  61. metadata:
  62.   labels:
  63.     k8s-app: kubernetes-dashboard
  64.   name: kubernetes-dashboard-csrf
  65.   namespace: kubernetes-dashboard
  66. type: Opaque
  67. data:
  68.   csrf: ""
  70. ---
  72. apiVersion: v1
  73. kind: Secret
  74. metadata:
  75.   labels:
  76.     k8s-app: kubernetes-dashboard
  77.   name: kubernetes-dashboard-key-holder
  78.   namespace: kubernetes-dashboard
  79. type: Opaque
  81. ---
  83. kind: ConfigMap
  84. apiVersion: v1
  85. metadata:
  86.   labels:
  87.     k8s-app: kubernetes-dashboard
  88.   name: kubernetes-dashboard-settings
  89.   namespace: kubernetes-dashboard
  91. ---
  93. kind: Role
  94. apiVersion: rbac.authorization.k8s.io/v1
  95. metadata:
  96.   labels:
  97.     k8s-app: kubernetes-dashboard
  98.   name: kubernetes-dashboard
  99.   namespace: kubernetes-dashboard
  100. rules:
  101.   # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  102.   - apiGroups: [""]
  103.     resources: ["secrets"]
  104.     resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
  105.     verbs: ["get", "update", "delete"]
  106.     # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  107.   - apiGroups: [""]
  108.     resources: ["configmaps"]
  109.     resourceNames: ["kubernetes-dashboard-settings"]
  110.     verbs: ["get", "update"]
  111.     # Allow Dashboard to get metrics.
  112.   - apiGroups: [""]
  113.     resources: ["services"]
  114.     resourceNames: ["heapster", "dashboard-metrics-scraper"]
  115.     verbs: ["proxy"]
  116.   - apiGroups: [""]
  117.     resources: ["services/proxy"]
  118.     resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
  119.     verbs: ["get"]
  121. ---
  123. kind: ClusterRole
  124. apiVersion: rbac.authorization.k8s.io/v1
  125. metadata:
  126.   labels:
  127.     k8s-app: kubernetes-dashboard
  128.   name: kubernetes-dashboard
  129. rules:
  130.   # Allow Metrics Scraper to get metrics from the Metrics server
  131.   - apiGroups: ["metrics.k8s.io"]
  132.     resources: ["pods", "nodes"]
  133.     verbs: ["get", "list", "watch"]
  135. ---
  137. apiVersion: rbac.authorization.k8s.io/v1
  138. kind: RoleBinding
  139. metadata:
  140.   labels:
  141.     k8s-app: kubernetes-dashboard
  142.   name: kubernetes-dashboard
  143.   namespace: kubernetes-dashboard
  144. roleRef:
  145.   apiGroup: rbac.authorization.k8s.io
  146.   kind: Role
  147.   name: kubernetes-dashboard
  148. subjects:
  149.   - kind: ServiceAccount
  150.     name: kubernetes-dashboard
  151.     namespace: kubernetes-dashboard
  153. ---
  155. apiVersion: rbac.authorization.k8s.io/v1
  156. kind: ClusterRoleBinding
  157. metadata:
  158.   name: kubernetes-dashboard
  159. roleRef:
  160.   apiGroup: rbac.authorization.k8s.io
  161.   kind: ClusterRole
  162.   name: kubernetes-dashboard
  163. subjects:
  164.   - kind: ServiceAccount
  165.     name: kubernetes-dashboard
  166.     namespace: kubernetes-dashboard
  168. ---
  170. kind: Deployment
  171. apiVersion: apps/v1
  172. metadata:
  173.   labels:
  174.     k8s-app: kubernetes-dashboard
  175.   name: kubernetes-dashboard
  176.   namespace: kubernetes-dashboard
  177. spec:
  178.   replicas: 1
  179.   revisionHistoryLimit: 10
  180.   selector:
  181.     matchLabels:
  182.       k8s-app: kubernetes-dashboard
  183.   template:
  184.     metadata:
  185.       labels:
  186.         k8s-app: kubernetes-dashboard
  187.     spec:
  188.       securityContext:
  189.         seccompProfile:
  190.           type: RuntimeDefault
  191.       containers:
  192.         - name: kubernetes-dashboard
  193.           image: kubernetesui/dashboard:v2.5.0
  194.           imagePullPolicy: Always
  195.           ports:
  196.             - containerPort: 8443
  197.               protocol: TCP
  198.           args:
  199.             - --auto-generate-certificates
  200.             - --namespace=kubernetes-dashboard
  201.             # Uncomment the following line to manually specify Kubernetes API server Host
  202.             # If not specified, Dashboard will attempt to auto discover the API server and connect
  203.             # to it. Uncomment only if the default does not work.
  204.             # - --apiserver-host=http://my-address:port
  205.           volumeMounts:
  206.             - name: kubernetes-dashboard-certs
  207.               mountPath: /certs
  208.               # Create on-disk volume to store exec logs
  209.             - mountPath: /tmp
  210.               name: tmp-volume
  211.           livenessProbe:
  212.             httpGet:
  213.               scheme: HTTPS
  214.               path: /
  215.               port: 8443
  216.             initialDelaySeconds: 30
  217.             timeoutSeconds: 30
  218.           securityContext:
  219.             allowPrivilegeEscalation: false
  220.             readOnlyRootFilesystem: true
  221.             runAsUser: 1001
  222.             runAsGroup: 2001
  223.       volumes:
  224.         - name: kubernetes-dashboard-certs
  225.           secret:
  226.             secretName: kubernetes-dashboard-certs
  227.         - name: tmp-volume
  228.           emptyDir: {}
  229.       serviceAccountName: kubernetes-dashboard
  230.       nodeSelector:
  231.         "kubernetes.io/os": linux
  232.       # Comment the following tolerations if Dashboard must not be deployed on master
  233.       tolerations:
  234.         - key: node-role.kubernetes.io/master
  235.           effect: NoSchedule
  237. ---
  239. kind: Service
  240. apiVersion: v1
  241. metadata:
  242.   labels:
  243.     k8s-app: dashboard-metrics-scraper
  244.   name: dashboard-metrics-scraper
  245.   namespace: kubernetes-dashboard
  246. spec:
  247.   ports:
  248.     - port: 8000
  249.       targetPort: 8000
  250.   selector:
  251.     k8s-app: dashboard-metrics-scraper
  253. ---
  255. kind: Deployment
  256. apiVersion: apps/v1
  257. metadata:
  258.   labels:
  259.     k8s-app: dashboard-metrics-scraper
  260.   name: dashboard-metrics-scraper
  261.   namespace: kubernetes-dashboard
  262. spec:
  263.   replicas: 1
  264.   revisionHistoryLimit: 10
  265.   selector:
  266.     matchLabels:
  267.       k8s-app: dashboard-metrics-scraper
  268.   template:
  269.     metadata:
  270.       labels:
  271.         k8s-app: dashboard-metrics-scraper
  272.     spec:
  273.       securityContext:
  274.         seccompProfile:
  275.           type: RuntimeDefault
  276.       containers:
  277.         - name: dashboard-metrics-scraper
  278.           image: kubernetesui/metrics-scraper:v1.0.7
  279.           ports:
  280.             - containerPort: 8000
  281.               protocol: TCP
  282.           livenessProbe:
  283.             httpGet:
  284.               scheme: HTTP
  285.               path: /
  286.               port: 8000
  287.             initialDelaySeconds: 30
  288.             timeoutSeconds: 30
  289.           volumeMounts:
  290.           - mountPath: /tmp
  291.             name: tmp-volume
  292.           securityContext:
  293.             allowPrivilegeEscalation: false
  294.             readOnlyRootFilesystem: true
  295.             runAsUser: 1001
  296.             runAsGroup: 2001
  297.       serviceAccountName: kubernetes-dashboard
  298.       nodeSelector:
  299.         "kubernetes.io/os": linux
  300.       # Comment the following tolerations if Dashboard must not be deployed on master
  301.       tolerations:
  302.         - key: node-role.kubernetes.io/master
  303.           effect: NoSchedule
  304.       volumes:
  305.         - name: tmp-volume
  306.           emptyDir: {}

执行命令，开始应用yaml文件

  1. kubectl apply -f recommended.yaml

创建完成后，查看deployments或pods对应状态。

  1. [root@master ~]# kubectl get deployments -n kubernetes-dashboard
  2. NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
  3. dashboard-metrics-scraper   1/1     1            1           6m7s
  4. kubernetes-dashboard        1/1     1            1           6m7s
  6. [root@master ~]# kubectl get pods -n kubernetes-dashboard
  7. NAME                                         READY   STATUS    RESTARTS   AGE
  8. dashboard-metrics-scraper-799d786dbf-p5h8c   1/1     Running   0          6m48s
  9. kubernetes-dashboard-546cbc58cd-p7j7d        1/1     Running   0          6m48s

创建ServiceAccount以及ClusterRoleBinding，新建k8s-auth.yaml文件

  1. apiVersion: v1
  2. kind: ServiceAccount
  3. metadata:
  4.   name: admin-user
  5.   namespace: kubernetes-dashboard
  6. ---
  7. apiVersion: rbac.authorization.k8s.io/v1
  8. kind: ClusterRoleBinding
  9. metadata:
  10.   name: admin-user
  11. roleRef:
  12.   apiGroup: rbac.authorization.k8s.io
  13.   kind: ClusterRole
  14.   name: cluster-admin
  15. subjects:
  16. - kind: ServiceAccount
  17.   name: admin-user
  18.   namespace: kubernetes-dashboard

应用RBAC yaml文件

  1. [root@master ~]# kubectl apply -f k8s-auth.yaml
  2. serviceaccount/admin-user created
  3. clusterrolebinding.rbac.authorization.k8s.io/admin-user unchanged

然后通过浏览器访问Kubernetes Dashboard，如果出现不能访问的情况：可能跟集群使用的证书有关。查看名称空间所应用的证书，kubernetes-dashboard-certs对应的DATA为0，因此这种场景下就需要自签发证书了。

  1. [root@master ~]# kubectl get secrets -n kubernetes-dashboard
  2. NAME                               TYPE                                  DATA   AGE
  3. admin-user-token-zqfhp             kubernetes.io/service-account-token   3      101s
  4. default-token-p2szn                kubernetes.io/service-account-token   3      11m
  5. kubernetes-dashboard-certs         Opaque                                0      11m
  6. kubernetes-dashboard-csrf          Opaque                                1      11m
  7. kubernetes-dashboard-key-holder    Opaque                                2      11m
  8. kubernetes-dashboard-token-dqhdt   kubernetes.io/service-account-token   3      11m

删除集群中kubernetes-dashboard名称空间下的证书

  1. kubectl delete secret kubernetes-dashboard-certs -n kubernetes-dashboard

生成证书

  1. openssl genrsa -out dashboard.key 2048 
  2. openssl req -new -out dashboard.csr -key dashboard.key -subj '/CN=your ip address'
  3. openssl x509 -req -in dashboard.csr -signkey dashboard.key -out dashboard.crt

生成新的secret

  1. kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.key --from-file=dashboard.crt -n kubernetes-dashboard

删除原有pod，由于使用development创建资源，会自动创建新Pod资源。

  1. [root@master ~]# kubectl delete pod kubernetes-dashboard-546cbc58cd-p7j7d -n kubernetes-dashboard
  2. pod "kubernetes-dashboard-546cbc58cd-p7j7d" deleted

查看新Pod创建状况，kubernetes-dashboard-546cbc58cd-pgs8s已经处于运行状态。

  1. [root@master ~]# kubectl get pods -n kubernetes-dashboard
  2. NAME                                         READY   STATUS    RESTARTS   AGE
  3. dashboard-metrics-scraper-799d786dbf-p5h8c   1/1     Running   0          22m
  4. kubernetes-dashboard-546cbc58cd-pgs8s        1/1     Running   0          72s

然后通过https://ip:端口访问
image.png
通过命令获取登录token

  1. kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')
  3. # 输出结果
  4. Name:         admin-user-token-zqfhp
  5. Namespace:    kubernetes-dashboard
  6. Labels:       <none>
  7. Annotations:  kubernetes.io/service-account.name: admin-user
  8.               kubernetes.io/service-account.uid: 7b0b0752-cda6-4931-8762-28d1683aab7d
  10. Type:  kubernetes.io/service-account-token
  12. Data
  13. ====
  14. token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IjY5akgzTk1kTWVQWjR1TUdwUXBGdE53a1kxcWF1TDJTLTRDVU1jSTB1ZkUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXpxZmhwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI3YjBiMDc1Mi1jZGE2LTQ5MzEtODc2Mi0yOGQxNjgzYWFiN2QiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.QtJ0dY4K97w77cBpS3dJ_7X3s_eu_O6uEIT5aQ6Pi0dq4jIuNtsQEx6dcRcN2rdQH6loDpsaZjA3AFiDkE1tdmgab-A1nhyPFp8TV7aQ4Ga7bMstx4azUo1E71-9ynh43O69JgmP2995p10eRjA8ZA74uwSEjREBKVki5g9daWUfZ6d7vupbjkmqgOcIbb4UCclgBDKBL4UM0CXi7v2eWGJOOOcaTv2UpibifLhHSMqJ-Tl3U96iXGyKkEk41D_fDvtUT7h7PSqwZyvP6amOXRxE4kuI2kwlzv9SNol8oXLYjlE3q5m2OSuZ6YHOX9bH4gFy_H9_tCOcxwKky4hKDA
  15. ca.crt:     1099 bytes
  16. namespace:  20 bytes

在登录页面输入token，进入首页看到集群的资源信息
image.png
配置用户名/密码登录
写入鉴权文件

  1. echo 'admin,admin,1' > /etc/kubernetes/pki/basic_auth_file

修改配置文件

  1. vim /etc/kubernetes/manifests/kube-apiserver.yaml
  2. # 增加如下参数
  3. - --basic-auth-file=/etc/kubernetes/pki/basic_auth_file

修改Kubernetes dashboard配置文件yaml，在args添加配置

  1.   - --authentication-mode=basic

绑定用户权限

  1. kubectl create clusterrolebinding  login-on-dashboard-with-cluster-admin  --clusterrole=cluster-admin --user=admin

重新加载Dashboard

  1. kubectl apply -f kubernetes-dashboard.yaml

Kuboard Dashboard

Kuboard是一款开源的k8s图形化管理软件，提供了一键安装的方式。

  1. kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml
  2. # 您也可以使用下面的指令，唯一的区别是，该指令使用华为云的镜像仓库替代 docker hub 分发 Kuboard 所需要的镜像
  3. # kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3-swr.yaml

执行命令监控安装进度

  1. watch kubectl get pods -n kuboard
  3. Every 2.0s: kubectl get pods -n kuboard                                                                       master: Wed Mar 30 22:11:22 2022
  4. NAME                               READY   STATUS    RESTARTS        AGE
  5. kuboard-agent-2-584456bfcb-rrwwf   1/1     Running   1 (2m23s ago)   2m43s
  6. kuboard-agent-64dc57c9b-2d84d      1/1     Running   1 (2m26s ago)   2m43s
  7. kuboard-etcd-69bcb                 1/1     Running   0               4m6s
  8. kuboard-questdb-7bcf54c98c-s825d   1/1     Running   0               2m42s
  9. kuboard-v3-56b4b954c9-745zt        1/1     Running   0               4m6s

容器全部进入running状态后，通过浏览器访问

image.png
具体安装细节参考官方网站：https://www.kuboard.cn/install/v3/install-in-k8s.html

摘自Kuboard官网：Kuboard 的定位和 Dashboard 是相似的，主要的区别 在于：

  • Kuboard 关注微服务参考架构的视角对界面进行组织，参考 Kuboard 简介
  • Kuboard 中，不需要手工编写 YAML 文件，进一步降低 K8S 使用难度，提高便捷性
  • Kuboard 可以导出整个微服务架构的部署信息，并在新的名称空间/集群导入配置信息
  • Kuboard 的一个发展方向是，提供内建的 监控套件（目前的全局监控套件成熟度比较高）

参考文档：