1. 安装docker

      1. curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
      2. #docker 限制日志长度
      3. #修改docker 配置 vim /etc/docker/daemon.json
      4. {
      5. "log-driver":"json-file",
      6. "log-opts": {"max-size":"500m", "max-file":"3"},
      7. "registry-mirrors": ["https://4h27092x.mirror.aliyuncs.com"]
      8. }
      9. #使用阿里云加速器

    2. 安装kubeadm

      1. #使用国内源
      2. vi /etc/apt/sources.list.d/kubernetes.list
      3. #复制 deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main 进去
      4. apt-get update&& apt-get install kubelet=1.17.4-00 kubeadm=1.17.4-00 kubectl=1.17.4-00
      5. #这里注意版本都要求这几个组件的版本一致


    4. ssh到master2执行下面创建token ```bash root@master2:~# kubeadm token create W1112 15:36:36.570723 18502 validation.go:28] Cannot validate kube-proxy config - no validator is available W1112 15:36:36.570916 18502 validation.go:28] Cannot validate kubelet config - no validator is available jz4hwt.6ytjurcs5d34udgd
    2. 2. Kubernetes认证的SHA256加密字符串
    3. ```bash
    4. root@master2:/etc/kubernetes/pki# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
    5. 105f781fb35cf703ad4061d21aaf7a26b909534bfb5b61a5a7cc92dc9451e10c
    1. 执行join ```bash root@slave1:~# kubeadm join k8s-loadbalance:6443 —token jz4hwt.6ytjurcs5d34udgd —discovery-token-ca-cert-hash sha256:105f781fb35cf703ad4061d21aaf7a26b909534bfb5b61a5a7cc92dc9451e10c W1112 16:23:44.144412 29191 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Reading configuration from the cluster… [preflight] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -oyaml’ [kubelet-start] Downloading configuration for the kubelet from the “kubelet-config-1.17” ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file “/var/lib/kubelet/config.yaml” [kubelet-start] Writing kubelet environment file with flags to file “/var/lib/kubelet/kubeadm-flags.env” [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap…

    This node has joined the cluster:

    • Certificate signing request was sent to apiserver and a response was received.
    • The Kubelet was informed of the new secure connection details.

    Run ‘kubectl get nodes’ on the control-plane to see this node join the cluster.

    2. 4. 这里执行后
    3. ```bash
    4. #执行tail -f -n /var/logs/syslog 打印如下错误，说明下载镜像失败，需要翻墙，也可以从现有的镜像导出
    5. rror: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    6. Nov 12 16:26:16 slave1 kubelet[29355]: E1112 16:26:16.092602   29355 pod_workers.go:191] Error syncing pod e2bbb1b4-56e5-46dd-89e1-c7231cce9f22 ("cattle-node-agent-972hl_cattle-system(e2bbb1b4-56e5-46dd-89e1-c7231cce9f22)"), skipping: failed to "CreatePodSandbox" for "cattle-node-agent-972hl_cattle-system(e2bbb1b4-56e5-46dd-89e1-c7231cce9f22)" with CreatePodSandboxError: "CreatePodSandbox for pod \"cattle-node-agent-972hl_cattle-system(e2bbb1b4-56e5-46dd-89e1-c7231cce9f22)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    7. Nov 12 16:26:18 slave1 dockerd[24916]: time="2020-11-12T16:26:18.023829581+08:00" level=warning msg="Error getting v2 registry: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    8. Nov 12 16:26:18 slave1 dockerd[24916]: time="2020-11-12T16:26:18.023872889+08:00" level=info msg="Attempting next endpoint for pull after error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    9. Nov 12 16:26:18 slave1 dockerd[24916]: time="2020-11-12T16:26:18.023910274+08:00" level=error msg="Handler for POST /images/create returned error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    10. Nov 12 16:26:18 slave1 kubelet[29355]: E1112 16:26:18.024335   29355 remote_runtime.go:105] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    11. Nov 12 16:26:18 slave1 kubelet[29355]: E1112 16:26:18.024402   29355 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "weave-net-bkthv_kube-system(4ee98980-feba-4864-b090-7fd641644ee2)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    12. Nov 12 16:26:18 slave1 kubelet[29355]: E1112 16:26:18.024420   29355 kuberuntime_manager.go:729] createPodSandbox for pod "weave-net-bkthv_kube-system(4ee98980-feba-4864-b090-7fd641644ee2)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    13. Nov 12 16:26:18 slave1 kubelet[29355]: E1112 16:26:18.024482   29355 pod_workers.go:191] Error syncing pod 4ee98980-feba-4864-b090-7fd641644ee2 ("weave-net-bkthv_kube-system(4ee98980-feba-4864-b090-7fd641644ee2)"), skipping: failed to "CreatePodSandbox" for "weave-net-bkthv_kube-system(4ee98980-feba-4864-b090-7fd641644ee2)" with CreatePodSandboxError: "CreatePodSandbox for pod \"weave-net-bkthv_kube-system(4ee98980-feba-4864-b090-7fd641644ee2)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    14. Nov 12 16:26:20 slave1 dockerd[24916]: time="2020-11-12T16:26:20.045576254+08:00" level=warning msg="Error getting v2 registry: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    15. Nov 12 16:26:20 slave1 dockerd[24916]: time="2020-11-12T16:26:20.045609976+08:00" level=info msg="Attempting next endpoint for pull after error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    16. Nov 12 16:26:20 slave1 dockerd[24916]: time="2020-11-12T16:26:20.045645868+08:00" level=error msg="Handler for POST /images/create returned error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    17. Nov 12 16:26:20 slave1 kubelet[29355]: E1112 16:26:20.046273   29355 remote_runtime.go:105] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    18. Nov 12 16:26:20 slave1 kubelet[29355]: E1112 16:26:20.046327   29355 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "kube-proxy-kjghs_kube-system(3724f364-f4df-444c-83c2-c8985e39c181)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    19. Nov 12 16:26:20 slave1 kubelet[29355]: E1112 16:26:20.046344   29355 kuberuntime_manager.go:729] createPodSandbox for pod "kube-proxy-kjghs_kube-system(3724f364-f4df-444c-83c2-c8985e39c181)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    20. Nov 12 16:26:20 slave1 kubelet[29355]: E1112 16:26:20.046399   29355 pod_workers.go:191] Error syncing pod 3724f364-f4df-444c-83c2-c8985e39c181 ("kube-proxy-kjghs_kube-system(3724f364-f4df-444c-83c2-c8985e39c181)"), skipping: failed to "CreatePodSandbox" for "kube-proxy-kjghs_kube-system(3724f364-f4df-444c-83c2-c8985e39c181)" with CreatePodSandboxError: "CreatePodSandbox for pod \"kube-proxy-kjghs_kube-system(3724f364-f4df-444c-83c2-c8985e39c181)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    21. Nov 12 16:26:20 slave1 kubelet[29355]: W1112 16:26:20.383835   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    22. Nov 12 16:26:20 slave1 kubelet[29355]: E1112 16:26:20.992088   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    23. Nov 12 16:26:25 slave1 kubelet[29355]: W1112 16:26:25.383955   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    24. Nov 12 16:26:25 slave1 kubelet[29355]: E1112 16:26:25.998646   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    25. Nov 12 16:26:30 slave1 kubelet[29355]: W1112 16:26:30.384093   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    26. Nov 12 16:26:31 slave1 kubelet[29355]: E1112 16:26:31.005436   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    27. Nov 12 16:26:33 slave1 kubelet[29355]: E1112 16:26:33.031748   29355 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
    28. Nov 12 16:26:33 slave1 kubelet[29355]: #011For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    29. Nov 12 16:26:35 slave1 kubelet[29355]: W1112 16:26:35.384210   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    30. Nov 12 16:26:36 slave1 kubelet[29355]: E1112 16:26:36.012148   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    31. Nov 12 16:26:40 slave1 kubelet[29355]: W1112 16:26:40.384324   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    32. Nov 12 16:26:41 slave1 kubelet[29355]: E1112 16:26:41.019090   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    33. Nov 12 16:26:45 slave1 kubelet[29355]: W1112 16:26:45.384430   29355 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
    34. Nov 12 16:26:46 slave1 kubelet[29355]: E1112 16:26:46.025828   29355 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
    35. Nov 12 16:26:47 slave1 kubelet[29355]: E1112 16:26:47.070648   29355 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
    36. Nov 12 16:26:47 slave1 kubelet[29355]: #011For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    37. Nov 12 16:26:48 slave1 dockerd[24916]: time="2020-11-12T16:26:48.034085147+08:00" level=warning msg="Error getting v2 registry: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    38. Nov 12 16:26:48 slave1 dockerd[24916]: time="2020-11-12T16:26:48.034122483+08:00" level=info msg="Attempting next endpoint for pull after error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    39. Nov 12 16:26:48 slave1 dockerd[24916]: time="2020-11-12T16:26:48.034159015+08:00" level=error msg="Handler for POST /images/create returned error: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    40. Nov 12 16:26:48 slave1 kubelet[29355]: E1112 16:26:48.034486   29355 remote_runtime.go:105] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    41. Nov 12 16:26:48 slave1 kubelet[29355]: E1112 16:26:48.034532   29355 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "exporter-node-cluster-monitoring-9j7bg_cattle-prometheus(826eaaed-6e25-421a-895b-d6055091d2fe)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    42. Nov 12 16:26:48 slave1 kubelet[29355]: E1112 16:26:48.034549   29355 kuberuntime_manager.go:729] createPodSandbox for pod "exporter-node-cluster-monitoring-9j7bg_cattle-prometheus(826eaaed-6e25-421a-895b-d6055091d2fe)" failed: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    43. Nov 12 16:26:48 slave1 kubelet[29355]: E1112 16:26:48.034603   29355 pod_workers.go:191] Error syncing pod 826eaaed-6e25-421a-895b-d6055091d2fe ("exporter-node-cluster-monitoring-9j7bg_cattle-prometheus(826eaaed-6e25-421a-895b-d6055091d2fe)"), skipping: failed to "CreatePodSandbox" for "exporter-node-cluster-monitoring-9j7bg_cattle-prometheus(826eaaed-6e25-421a-895b-d6055091d2fe)" with CreatePodSandboxError: "CreatePodSandbox for pod \"exporter-node-cluster-monitoring-9j7bg_cattle-prometheus(826eaaed-6e25-421a-895b-d6055091d2fe)\" failed: rpc error: code = Unknown desc = failed pulling image \"k8s.gcr.io/pause:3.1\": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"

    1. 从有镜像的主机导出镜像到新的主机上

      1. mkdir tmp
      2. cd tmp
      3. docker save -o weave-npc-2.6.2 weaveworks/weave-npc:2.6.2
      4. docker save -o weave-kube-2.6.2 weaveworks/weave-kube:2.6.2
      5. docker save -o kube-proxy-v1.17.4 k8s.gcr.io/kube-proxy:v1.17.4
      6. docker save -o kube-controller-manager-v1.17.4 k8s.gcr.io/kube-controller-manager:v1.17.4
      7. docker save -o kube-apiserver-v1.17.4 k8s.gcr.io/kube-apiserver:v1.17.4
      8. docker save -o kube-scheduler-v1.17.4 k8s.gcr.io/kube-scheduler:v1.17.4
      9. docker save -o etcd-3.4.3-0 k8s.gcr.io/etcd:3.4.3-0
      10. docker save -o pause-3.1 k8s.gcr.io/pause:3.1
      11. cd ..
      12. scp -r tmp slave1:~/
      13. #切换到slave1 进入对应的目录  cd ~/tmp
      14. #因为后缀不是.tar所以要修改成.tar的后缀
      15. for file in `ls`;do mv $file $file.tar;done
      16. #导入docker image
      17. for file in `ls`;do docker load -i $file;done
      18. 显示如下：
      19. fe9a8b4f1dcc: Loading layer  43.87MB/43.87MB
      20. ce04b89b7def: Loading layer  224.9MB/224.9MB
      21. 1b2bc745b46f: Loading layer  21.22MB/21.22MB
      22. Loaded image: k8s.gcr.io/etcd:3.4.3-0
      23. fc4976bd934b: Loading layer  53.88MB/53.88MB
      24. 9daac3fed755: Loading layer  118.7MB/118.7MB
      25. Loaded image: k8s.gcr.io/kube-apiserver:v1.17.4
      26. 99df54617e88: Loading layer  108.6MB/108.6MB
      27. Loaded image: k8s.gcr.io/kube-controller-manager:v1.17.4
      28. 682fbb19de80: Loading layer  21.06MB/21.06MB
      29. 2dc2f2423ad1: Loading layer  5.168MB/5.168MB
      30. ad9fb2411669: Loading layer  4.608kB/4.608kB
      31. 597151d24476: Loading layer  8.192kB/8.192kB
      32. 0d8d54147a3a: Loading layer  8.704kB/8.704kB
      33. 960d0ce862e2: Loading layer  37.81MB/37.81MB
      34. Loaded image: k8s.gcr.io/kube-proxy:v1.17.4
      35. f6953727aaba: Loading layer   42.1MB/42.1MB
      36. Loaded image: k8s.gcr.io/kube-scheduler:v1.17.4
      37. e17133b79956: Loading layer  744.4kB/744.4kB
      38. Loaded image: k8s.gcr.io/pause:3.1
      39. 531743b7098c: Loading layer  5.819MB/5.819MB
      40. c8653ce70b13: Loading layer  25.98MB/25.98MB
      41. cde21d954e2b: Loading layer  40.26MB/40.26MB
      42. dc0c791a17c3: Loading layer  27.55MB/27.55MB
      43. 1b40ebf118d7: Loading layer   2.56kB/2.56kB
      44. f94d456056a5: Loading layer  24.58MB/24.58MB
      45. Loaded image: weaveworks/weave-kube:2.6.2
      46. ec24b5006700: Loading layer  3.387MB/3.387MB
      47. da124db19ffe: Loading layer  27.95MB/27.95MB
      48. 29606d2aa1b9: Loading layer   2.56kB/2.56kB
      49. 46b621817e1a: Loading layer  4.096kB/4.096kB
      50. Loaded image: weaveworks/weave-npc:2.6.2

    2. 去master2

      1. root@master2:~# kubectl get nodes
      2. NAME      STATUS   ROLES    AGE    VERSION
      3. master1   Ready    master   703d   v1.17.4
      4. master2   Ready    master   703d   v1.17.4
      5. master3   Ready    master   703d   v1.17.4
      6. slave1    Ready    <none>   34m    v1.17.4
      7. test1     Ready    <none>   696d   v1.17.4
      8. #说明添加节点成功