实验机器清单
环境准备

以下操作,如果没有特殊说明, 指在对应的服务器上执行

一、设置yum源

所有服务器都执行
因为我使用的环境不能科学上网,所以配置阿里云的yum源

  1. [root@k8s-5-146 ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
  2. > [kubernetes]
  3. > name=Kubernetes
  4. > baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
  5. > enabled=1
  6. > gpgcheck=0
  7. > repo_gpgcheck=0
  8. > gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
  9. >         http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
  10. > EOF

二、安装 kubeadm、kubelet、kubectl

所有服务器都执行

由于我的环境中docker版本是 20.10.6 

  1. # --disableexcludes 禁掉除了kubernetes之外的别的仓库
  2. [root@k8s-5-146 ~]# yum install -y kubelet-1.19.9 kubeadm-1.19.9 kubectl-1.19.9 --disableexcludes=kubernetes
  4. [root@k8s-5-147 ~]# kubeadm version
  5. kubeadm version: &version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.9", GitCommit:"9dd794e454ac32d97cde41ae10be801ae98f75df", GitTreeState:"clean", BuildDate:"2021-03-18T01:07:09Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
  7. ## 设置开机启动
  8. [root@k8s-5-146 ~]# systemctl enable --now kubelet

三、初始化集群

  1. #导出默认的初始化配置文件
  2. [root@k8s-5-146 ~]# mkdir -p /opt/k8s
  3. [root@k8s-5-146 ~]# cd /opt/k8s
  4. [root@k8s-5-146 k8s]#  kubeadm config print init-defaults > kubeadm.yaml
  6. [root@k8s-5-146 k8s]# vim kubeadm.yaml
  7. apiVersion: kubeadm.k8s.io/v1beta2
  8. bootstrapTokens:
  9. - groups:
  10.   - system:bootstrappers:kubeadm:default-node-token
  11.   token: abcdef.0123456789abcdef
  12.   ttl: 24h0m0s
  13.   usages:
  14.   - signing
  15.   - authentication
  16. kind: InitConfiguration
  17. localAPIEndpoint:
  18.   advertiseAddress: 192.168.5.146
  19.   bindPort: 6443
  20. nodeRegistration:
  21.   criSocket: /var/run/dockershim.sock
  22.   name: k8s-5-146.k8s.host
  23.   taints:
  24.   - effect: NoSchedule
  25.     key: node-role.kubernetes.io/master
  26. ---
  27. apiServer:
  28.   timeoutForControlPlane: 4m0s
  29. apiVersion: kubeadm.k8s.io/v1beta2
  30. certificatesDir: /etc/kubernetes/pki
  31. clusterName: kubernetes
  32. controllerManager: {}
  33. dns:
  34.   type: CoreDNS
  35. etcd:
  36.   local:
  37.     dataDir: /var/lib/etcd
  38. imageRepository: registry.aliyuncs.com/google_containers #将原来默认的镜像仓库修改为阿里云的
  39. kind: ClusterConfiguration
  40. kubernetesVersion: v1.19.0
  41. networking:
  42.   dnsDomain: cluster.local
  43.   serviceSubnet: 10.96.0.0/12
  44. scheduler: {}
  46. ---
  47. apiVersion: kubeproxy.config.k8s.io/v1alpha1
  48. kind: KubeProxyConfiguration
  49. mode: ipvs  # kube-proxy 模式
  52. [root@k8s-5-146 k8s]# kubeadm init --config kubeadm.yaml
  53. W0416 15:09:49.726813    6094 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
  54. [init] Using Kubernetes version: v1.19.0
  55. [preflight] Running pre-flight checks
  56.     [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
  57.     [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.6. Latest validated version: 19.03
  58. [preflight] Pulling images required for setting up a Kubernetes cluster
  59. [preflight] This might take a minute or two, depending on the speed of your internet connection
  60. [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
  61. [certs] Using certificateDir folder "/etc/kubernetes/pki"
  62. [certs] Generating "ca" certificate and key
  63. [certs] Generating "apiserver" certificate and key
  64. [certs] apiserver serving cert is signed for DNS names [k8s-5-146.k8s.host kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.5.146]
  65. [certs] Generating "apiserver-kubelet-client" certificate and key
  66. [certs] Generating "front-proxy-ca" certificate and key
  67. [certs] Generating "front-proxy-client" certificate and key
  68. [certs] Generating "etcd/ca" certificate and key
  69. [certs] Generating "etcd/server" certificate and key
  70. [certs] etcd/server serving cert is signed for DNS names [k8s-5-146.k8s.host localhost] and IPs [192.168.5.146 127.0.0.1 ::1]
  71. [certs] Generating "etcd/peer" certificate and key
  72. [certs] etcd/peer serving cert is signed for DNS names [k8s-5-146.k8s.host localhost] and IPs [192.168.5.146 127.0.0.1 ::1]
  73. [certs] Generating "etcd/healthcheck-client" certificate and key
  74. [certs] Generating "apiserver-etcd-client" certificate and key
  75. [certs] Generating "sa" key and public key
  76. [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
  77. [kubeconfig] Writing "admin.conf" kubeconfig file
  78. [kubeconfig] Writing "kubelet.conf" kubeconfig file
  79. [kubeconfig] Writing "controller-manager.conf" kubeconfig file
  80. [kubeconfig] Writing "scheduler.conf" kubeconfig file
  81. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
  82. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
  83. [kubelet-start] Starting the kubelet
  84. [control-plane] Using manifest folder "/etc/kubernetes/manifests"
  85. [control-plane] Creating static Pod manifest for "kube-apiserver"
  86. [control-plane] Creating static Pod manifest for "kube-controller-manager"
  87. [control-plane] Creating static Pod manifest for "kube-scheduler"
  88. [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
  89. [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
  90. [kubelet-check] Initial timeout of 40s passed.
  91. [apiclient] All control plane components are healthy after 59.002281 seconds
  92. [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
  93. [kubelet] Creating a ConfigMap "kubelet-config-1.19" in namespace kube-system with the configuration for the kubelets in the cluster
  94. [upload-certs] Skipping phase. Please see --upload-certs
  95. [mark-control-plane] Marking the node k8s-5-146.k8s.host as control-plane by adding the label "node-role.kubernetes.io/master=''"
  96. [mark-control-plane] Marking the node k8s-5-146.k8s.host as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
  97. [bootstrap-token] Using token: abcdef.0123456789abcdef
  98. [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
  99. [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
  100. [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
  101. [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
  102. [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
  103. [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
  104. [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
  105. [addons] Applied essential addon: CoreDNS
  106. [addons] Applied essential addon: kube-proxy
  108. Your Kubernetes control-plane has initialized successfully!
  110. To start using your cluster, you need to run the following as a regular user:
  112.   mkdir -p $HOME/.kube
  113.   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  114.   sudo chown $(id -u):$(id -g) $HOME/.kube/config
  116. You should now deploy a pod network to the cluster.
  117. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  118.   https://kubernetes.io/docs/concepts/cluster-administration/addons/
  120. Then you can join any number of worker nodes by running the following on each as root:
  122. kubeadm join 192.168.5.146:6443 --token abcdef.0123456789abcdef \
  123.     --discovery-token-ca-cert-hash sha256:a025044360496164420da8350ba17f5acf9ab6444398707f8cef568b8920059a

四、拷贝 kubeconfig 文件

  1. [root@k8s-5-146 ~]# mkdir -p $HOME/.kube
  2. [root@k8s-5-146 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  3. [root@k8s-5-146 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

五、添加节点

记住初始化集群上面的配置和操作要提前做好，将 master 节点上面的 $HOME/.kube/config 文件拷贝到 node 节点对应的文件中，安装 kubeadm、kubelet、kubectl（可选），然后执行上面初始化完成后提示的 join 命令即可：

  1. # 分别在 k8s-5-147 和 k8s-5-148 上执行
  2. [root@k8s-5-147 ~]# mkdir -p $HOME/.kube
  3. [root@k8s-5-147 ~]# scp k8s-5-146.k8s.host:/etc/kubernetes/admin.conf $HOME/.kube/config
  4. [root@k8s-5-147 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
  6. # 加入集群
  7. [root@k8s-5-147 ~]# kubeadm join 192.168.5.146:6443 --token abcdef.0123456789abcdef \
  8.     --discovery-token-ca-cert-hash sha256:a025044360496164420da8350ba17f5acf9ab6444398707f8cef568b8920059a
  10.  [preflight] Running pre-flight checks
  11.     [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.6. Latest validated version: 19.03
  12. [preflight] Reading configuration from the cluster...
  13. [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
  14. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
  15. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
  16. [kubelet-start] Starting the kubelet
  17. [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
  19. This node has joined the cluster:
  20. * Certificate signing request was sent to apiserver and a response was received.
  21. * The Kubelet was informed of the new secure connection details.
  23. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

join 命令 如果忘记了上面的 join 命令可以使用命令 kubeadm token create --print-join-command 重新获取。

查看节点

  1. [root@k8s-5-146 ~]# kubectl get nodes
  2. NAME                 STATUS     ROLES    AGE   VERSION
  3. k8s-5-146.k8s.host   NotReady   master   17m   v1.19.9
  4. k8s-5-147.k8s.host   NotReady   <none>   16s   v1.19.9
  5. k8s-5-148.k8s.host   NotReady   <none>   4s    v1.19.9

可以看到是 NotReady 状态，这是因为还没有安装网络插件，接下来安装网络插件，可以在文档 https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/ 中选择我们自己的网络插件，这里我们安装 flannel:

六、安装网络插件 flannel

  1. [root@k8s-5-146 ~]# cd /opt/k8s
  3. [root@k8s-5-146 k8s]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
  5. [root@k8s-5-146 k8s]# kubectl apply -f kube-flannel.yml 
  6. podsecuritypolicy.policy/psp.flannel.unprivileged created
  7. clusterrole.rbac.authorization.k8s.io/flannel created
  8. clusterrolebinding.rbac.authorization.k8s.io/flannel created
  9. serviceaccount/flannel created
  10. configmap/kube-flannel-cfg created
  11. daemonset.apps/kube-flannel-ds created
  13. [root@k8s-5-146 k8s]# kubectl get nodes
  14. NAME                 STATUS   ROLES    AGE   VERSION
  15. k8s-5-146.k8s.host   Ready    master   46m   v1.19.9
  16. k8s-5-147.k8s.host   Ready    <none>   29m   v1.19.9
  17. k8s-5-148.k8s.host   Ready    <none>   29m   v1.19.9
  19. # 现在全部都是Ready状态啦