安装docker

语雀内容

安装kubernetes

  • 基本配置
  1. #关闭防火墙和selinux
  2. systemctl stop firewalld && systemctl disable firewalld
  3. sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config && setenforce 0
  5. #关闭swap
  6. swapoff -a
  7. yes | cp /etc/fstab /etc/fstab_bak
  8. cat /etc/fstab_bak |grep -v swap > /etc/fstab
  • 加载ipvs模块
  1. cat > /etc/sysconfig/modules/ipvs.modules <<EOF
  2. #!/bin/bash
  3. modprobe -- ip_vs
  4. modprobe -- ip_vs_rr
  5. modprobe -- ip_vs_wrr
  6. modprobe -- ip_vs_sh
  7. modprobe -- nf_conntrack_ipv4
  8. EOF
  10. #执行脚本
  11. chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

上面脚本创建了/etc/sysconfig/modules/ipvs.modules文件，保证在节点重启后能自动加载所需模块。 使用lsmod | grep -e ip_vs -e nf_conntrack_ipv4命令查看是否已经正确加载所需的内核模块。

安装管理工具ipvsadm

  1. yum install ipset ipvsadm -y
  • 配置 kubectl kubeadm kubelet 三个软件的镜像源(使用aliyun镜像)
  1. cat <<EOF > /etc/yum.repos.d/kubernetes.repo
  2. [kubernetes]
  3. name=Kubernetes
  4. baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
  5. enabled=1
  6. gpgcheck=1
  7. repo_gpgcheck=1
  8. gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
  9. EOF
  • 开始安装，并启动 kubelet
  1. yum install -y kubectl kubeadm kubelet
  2. systemctl enable kubelet
  3. systemctl start kubelet

kubectl 是命令行工具，所以不用启动 kubeadm 是集群搭建工具，也不用启动 kubelet 是节点管理工具，需要在所有节点启动 以上所有教程，都需要在所有机器上操作，包括安装 kubectl kubeadm kubelet，启动 kubelet

初始化master节点

  • 初始化
  1. kubeadm init \
  2.     --apiserver-advertise-address=[master public ip] \
  3.     --image-repository registry.aliyuncs.com/google_containers \
  4.     --kubernetes-version v1.18.0 \
  5.     --pod-network-cidr=33.244.0.0/16 \
  6.     --apiserver-cert-extra-sans [master inner ip] \
  7.     --ignore-preflight-errors=NumCPU

—apiserver-advertise-address master的ip —image-repository 镜像仓库设置,使用aliyun仓库 —kubernetes-version 指定kubernetes版本 —pod-network-cidr 指定pod子网cidr,默认是10.244.0.0/16 —apiserver-cert-extra-sans 指定额外的签名地址 —ignore-preflight-errors=NumCPU 忽略单核运行警告

  • 如果—apiserver-advertise-address指定的ip没有网卡配置的话(如阿里云的公网ip),会导致etcd无法绑定到ip端口,则需要手动修改etcd的绑定地址

  1. 此时会卡在

    1. [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

  2. 修改etcd.yaml

  1. vim /etc/kubernetes/manifests/etcd.yaml
  1. spec:
  2.   containers:
  3.   - command:
  4.     - etcd
  5.     - --listen-client-urls=https://127.0.0.1:2379,https://[public ip]:2379
  6.     - --listen-peer-urls=https://[public ip]:2380

修改—listen-client-urls和—listen-peer-urls中的public ip为127.0.0.1保存即可

  • 配置kubectl
  1. mkdir -p $HOME/.kube
  2. sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  3. sudo chown $(id -u):$(id -g) $HOME/.kube/config
  • 安装pod附加网络组件flannel
  1. 新建文件kube-flannel.yml
  1. #注意修改net-conf.json子网段
  2. ---
  3. apiVersion: policy/v1beta1
  4. kind: PodSecurityPolicy
  5. metadata:
  6.   name: psp.flannel.unprivileged
  7.   annotations:
  8.     seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
  9.     seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
  10.     apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
  11.     apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
  12. spec:
  13.   privileged: false
  14.   volumes:
  15.     - configMap
  16.     - secret
  17.     - emptyDir
  18.     - hostPath
  19.   allowedHostPaths:
  20.     - pathPrefix: "/etc/cni/net.d"
  21.     - pathPrefix: "/etc/kube-flannel"
  22.     - pathPrefix: "/run/flannel"
  23.   readOnlyRootFilesystem: false
  24.   # Users and groups
  25.   runAsUser:
  26.     rule: RunAsAny
  27.   supplementalGroups:
  28.     rule: RunAsAny
  29.   fsGroup:
  30.     rule: RunAsAny
  31.   # Privilege Escalation
  32.   allowPrivilegeEscalation: false
  33.   defaultAllowPrivilegeEscalation: false
  34.   # Capabilities
  35.   allowedCapabilities: ['NET_ADMIN']
  36.   defaultAddCapabilities: []
  37.   requiredDropCapabilities: []
  38.   # Host namespaces
  39.   hostPID: false
  40.   hostIPC: false
  41.   hostNetwork: true
  42.   hostPorts:
  43.     - min: 0
  44.       max: 65535
  45.   # SELinux
  46.   seLinux:
  47.     # SELinux is unused in CaaSP
  48.     rule: 'RunAsAny'
  49. ---
  50. kind: ClusterRole
  51. apiVersion: rbac.authorization.k8s.io/v1beta1
  52. metadata:
  53.   name: flannel
  54. rules:
  55.   - apiGroups: ['extensions']
  56.     resources: ['podsecuritypolicies']
  57.     verbs: ['use']
  58.     resourceNames: ['psp.flannel.unprivileged']
  59.   - apiGroups:
  60.       - ""
  61.     resources:
  62.       - pods
  63.     verbs:
  64.       - get
  65.   - apiGroups:
  66.       - ""
  67.     resources:
  68.       - nodes
  69.     verbs:
  70.       - list
  71.       - watch
  72.   - apiGroups:
  73.       - ""
  74.     resources:
  75.       - nodes/status
  76.     verbs:
  77.       - patch
  78. ---
  79. kind: ClusterRoleBinding
  80. apiVersion: rbac.authorization.k8s.io/v1beta1
  81. metadata:
  82.   name: flannel
  83. roleRef:
  84.   apiGroup: rbac.authorization.k8s.io
  85.   kind: ClusterRole
  86.   name: flannel
  87. subjects:
  88.   - kind: ServiceAccount
  89.     name: flannel
  90.     namespace: kube-system
  91. ---
  92. apiVersion: v1
  93. kind: ServiceAccount
  94. metadata:
  95.   name: flannel
  96.   namespace: kube-system
  97. ---
  98. kind: ConfigMap
  99. apiVersion: v1
  100. metadata:
  101.   name: kube-flannel-cfg
  102.   namespace: kube-system
  103.   labels:
  104.     tier: node
  105.     app: flannel
  106. data:
  107.   cni-conf.json: |
  108.     {
  109.       "name": "cbr0",
  110.       "cniVersion": "0.3.1",
  111.       "plugins": [
  112.         {
  113.           "type": "flannel",
  114.           "delegate": {
  115.             "hairpinMode": true,
  116.             "isDefaultGateway": true
  117.           }
  118.         },
  119.         {
  120.           "type": "portmap",
  121.           "capabilities": {
  122.             "portMappings": true
  123.           }
  124.         }
  125.       ]
  126.     }
  127.   net-conf.json: |
  128.     {
  129.       "Network": "33.244.0.0/16",
  130.       "Backend": {
  131.         "Type": "vxlan"
  132.       }
  133.     }
  134. ---
  135. apiVersion: apps/v1
  136. kind: DaemonSet
  137. metadata:
  138.   name: kube-flannel-ds-amd64
  139.   namespace: kube-system
  140.   labels:
  141.     tier: node
  142.     app: flannel
  143. spec:
  144.   selector:
  145.     matchLabels:
  146.       app: flannel
  147.   template:
  148.     metadata:
  149.       labels:
  150.         tier: node
  151.         app: flannel
  152.     spec:
  153.       affinity:
  154.         nodeAffinity:
  155.           requiredDuringSchedulingIgnoredDuringExecution:
  156.             nodeSelectorTerms:
  157.               - matchExpressions:
  158.                   - key: beta.kubernetes.io/os
  159.                     operator: In
  160.                     values:
  161.                       - linux
  162.                   - key: beta.kubernetes.io/arch
  163.                     operator: In
  164.                     values:
  165.                       - amd64
  166.       hostNetwork: true
  167.       tolerations:
  168.         - operator: Exists
  169.           effect: NoSchedule
  170.       serviceAccountName: flannel
  171.       initContainers:
  172.         - name: install-cni
  173.           image: quay.io/coreos/flannel:v0.12.0-amd64
  174.           command:
  175.             - cp
  176.           args:
  177.             - -f
  178.             - /etc/kube-flannel/cni-conf.json
  179.             - /etc/cni/net.d/10-flannel.conflist
  180.           volumeMounts:
  181.             - name: cni
  182.               mountPath: /etc/cni/net.d
  183.             - name: flannel-cfg
  184.               mountPath: /etc/kube-flannel/
  185.       containers:
  186.         - name: kube-flannel
  187.           image: quay.io/coreos/flannel:v0.12.0-amd64
  188.           command:
  189.             - /opt/bin/flanneld
  190.           args:
  191.             - --ip-masq
  192.             - --kube-subnet-mgr
  193.           resources:
  194.             requests:
  195.               cpu: "100m"
  196.               memory: "50Mi"
  197.             limits:
  198.               cpu: "100m"
  199.               memory: "50Mi"
  200.           securityContext:
  201.             privileged: false
  202.             capabilities:
  203.               add: ["NET_ADMIN"]
  204.           env:
  205.             - name: POD_NAME
  206.               valueFrom:
  207.                 fieldRef:
  208.                   fieldPath: metadata.name
  209.             - name: POD_NAMESPACE
  210.               valueFrom:
  211.                 fieldRef:
  212.                   fieldPath: metadata.namespace
  213.           volumeMounts:
  214.             - name: run
  215.               mountPath: /run/flannel
  216.             - name: flannel-cfg
  217.               mountPath: /etc/kube-flannel/
  218.       volumes:
  219.         - name: run
  220.           hostPath:
  221.             path: /run/flannel
  222.         - name: cni
  223.           hostPath:
  224.             path: /etc/cni/net.d
  225.         - name: flannel-cfg
  226.           configMap:
  227.             name: kube-flannel-cfg
  228. ---
  229. apiVersion: apps/v1
  230. kind: DaemonSet
  231. metadata:
  232.   name: kube-flannel-ds-arm64
  233.   namespace: kube-system
  234.   labels:
  235.     tier: node
  236.     app: flannel
  237. spec:
  238.   selector:
  239.     matchLabels:
  240.       app: flannel
  241.   template:
  242.     metadata:
  243.       labels:
  244.         tier: node
  245.         app: flannel
  246.     spec:
  247.       affinity:
  248.         nodeAffinity:
  249.           requiredDuringSchedulingIgnoredDuringExecution:
  250.             nodeSelectorTerms:
  251.               - matchExpressions:
  252.                   - key: beta.kubernetes.io/os
  253.                     operator: In
  254.                     values:
  255.                       - linux
  256.                   - key: beta.kubernetes.io/arch
  257.                     operator: In
  258.                     values:
  259.                       - arm64
  260.       hostNetwork: true
  261.       tolerations:
  262.         - operator: Exists
  263.           effect: NoSchedule
  264.       serviceAccountName: flannel
  265.       initContainers:
  266.         - name: install-cni
  267.           image: quay.io/coreos/flannel:v0.12.0-arm64
  268.           command:
  269.             - cp
  270.           args:
  271.             - -f
  272.             - /etc/kube-flannel/cni-conf.json
  273.             - /etc/cni/net.d/10-flannel.conflist
  274.           volumeMounts:
  275.             - name: cni
  276.               mountPath: /etc/cni/net.d
  277.             - name: flannel-cfg
  278.               mountPath: /etc/kube-flannel/
  279.       containers:
  280.         - name: kube-flannel
  281.           image: quay.io/coreos/flannel:v0.12.0-arm64
  282.           command:
  283.             - /opt/bin/flanneld
  284.           args:
  285.             - --ip-masq
  286.             - --kube-subnet-mgr
  287.           resources:
  288.             requests:
  289.               cpu: "100m"
  290.               memory: "50Mi"
  291.             limits:
  292.               cpu: "100m"
  293.               memory: "50Mi"
  294.           securityContext:
  295.             privileged: false
  296.             capabilities:
  297.               add: ["NET_ADMIN"]
  298.           env:
  299.             - name: POD_NAME
  300.               valueFrom:
  301.                 fieldRef:
  302.                   fieldPath: metadata.name
  303.             - name: POD_NAMESPACE
  304.               valueFrom:
  305.                 fieldRef:
  306.                   fieldPath: metadata.namespace
  307.           volumeMounts:
  308.             - name: run
  309.               mountPath: /run/flannel
  310.             - name: flannel-cfg
  311.               mountPath: /etc/kube-flannel/
  312.       volumes:
  313.         - name: run
  314.           hostPath:
  315.             path: /run/flannel
  316.         - name: cni
  317.           hostPath:
  318.             path: /etc/cni/net.d
  319.         - name: flannel-cfg
  320.           configMap:
  321.             name: kube-flannel-cfg
  322. ---
  323. apiVersion: apps/v1
  324. kind: DaemonSet
  325. metadata:
  326.   name: kube-flannel-ds-arm
  327.   namespace: kube-system
  328.   labels:
  329.     tier: node
  330.     app: flannel
  331. spec:
  332.   selector:
  333.     matchLabels:
  334.       app: flannel
  335.   template:
  336.     metadata:
  337.       labels:
  338.         tier: node
  339.         app: flannel
  340.     spec:
  341.       affinity:
  342.         nodeAffinity:
  343.           requiredDuringSchedulingIgnoredDuringExecution:
  344.             nodeSelectorTerms:
  345.               - matchExpressions:
  346.                   - key: beta.kubernetes.io/os
  347.                     operator: In
  348.                     values:
  349.                       - linux
  350.                   - key: beta.kubernetes.io/arch
  351.                     operator: In
  352.                     values:
  353.                       - arm
  354.       hostNetwork: true
  355.       tolerations:
  356.         - operator: Exists
  357.           effect: NoSchedule
  358.       serviceAccountName: flannel
  359.       initContainers:
  360.         - name: install-cni
  361.           image: quay.io/coreos/flannel:v0.12.0-arm
  362.           command:
  363.             - cp
  364.           args:
  365.             - -f
  366.             - /etc/kube-flannel/cni-conf.json
  367.             - /etc/cni/net.d/10-flannel.conflist
  368.           volumeMounts:
  369.             - name: cni
  370.               mountPath: /etc/cni/net.d
  371.             - name: flannel-cfg
  372.               mountPath: /etc/kube-flannel/
  373.       containers:
  374.         - name: kube-flannel
  375.           image: quay.io/coreos/flannel:v0.12.0-arm
  376.           command:
  377.             - /opt/bin/flanneld
  378.           args:
  379.             - --ip-masq
  380.             - --kube-subnet-mgr
  381.           resources:
  382.             requests:
  383.               cpu: "100m"
  384.               memory: "50Mi"
  385.             limits:
  386.               cpu: "100m"
  387.               memory: "50Mi"
  388.           securityContext:
  389.             privileged: false
  390.             capabilities:
  391.               add: ["NET_ADMIN"]
  392.           env:
  393.             - name: POD_NAME
  394.               valueFrom:
  395.                 fieldRef:
  396.                   fieldPath: metadata.name
  397.             - name: POD_NAMESPACE
  398.               valueFrom:
  399.                 fieldRef:
  400.                   fieldPath: metadata.namespace
  401.           volumeMounts:
  402.             - name: run
  403.               mountPath: /run/flannel
  404.             - name: flannel-cfg
  405.               mountPath: /etc/kube-flannel/
  406.       volumes:
  407.         - name: run
  408.           hostPath:
  409.             path: /run/flannel
  410.         - name: cni
  411.           hostPath:
  412.             path: /etc/cni/net.d
  413.         - name: flannel-cfg
  414.           configMap:
  415.             name: kube-flannel-cfg
  416. ---
  417. apiVersion: apps/v1
  418. kind: DaemonSet
  419. metadata:
  420.   name: kube-flannel-ds-ppc64le
  421.   namespace: kube-system
  422.   labels:
  423.     tier: node
  424.     app: flannel
  425. spec:
  426.   selector:
  427.     matchLabels:
  428.       app: flannel
  429.   template:
  430.     metadata:
  431.       labels:
  432.         tier: node
  433.         app: flannel
  434.     spec:
  435.       affinity:
  436.         nodeAffinity:
  437.           requiredDuringSchedulingIgnoredDuringExecution:
  438.             nodeSelectorTerms:
  439.               - matchExpressions:
  440.                   - key: beta.kubernetes.io/os
  441.                     operator: In
  442.                     values:
  443.                       - linux
  444.                   - key: beta.kubernetes.io/arch
  445.                     operator: In
  446.                     values:
  447.                       - ppc64le
  448.       hostNetwork: true
  449.       tolerations:
  450.         - operator: Exists
  451.           effect: NoSchedule
  452.       serviceAccountName: flannel
  453.       initContainers:
  454.         - name: install-cni
  455.           image: quay.io/coreos/flannel:v0.12.0-ppc64le
  456.           command:
  457.             - cp
  458.           args:
  459.             - -f
  460.             - /etc/kube-flannel/cni-conf.json
  461.             - /etc/cni/net.d/10-flannel.conflist
  462.           volumeMounts:
  463.             - name: cni
  464.               mountPath: /etc/cni/net.d
  465.             - name: flannel-cfg
  466.               mountPath: /etc/kube-flannel/
  467.       containers:
  468.         - name: kube-flannel
  469.           image: quay.io/coreos/flannel:v0.12.0-ppc64le
  470.           command:
  471.             - /opt/bin/flanneld
  472.           args:
  473.             - --ip-masq
  474.             - --kube-subnet-mgr
  475.           resources:
  476.             requests:
  477.               cpu: "100m"
  478.               memory: "50Mi"
  479.             limits:
  480.               cpu: "100m"
  481.               memory: "50Mi"
  482.           securityContext:
  483.             privileged: false
  484.             capabilities:
  485.               add: ["NET_ADMIN"]
  486.           env:
  487.             - name: POD_NAME
  488.               valueFrom:
  489.                 fieldRef:
  490.                   fieldPath: metadata.name
  491.             - name: POD_NAMESPACE
  492.               valueFrom:
  493.                 fieldRef:
  494.                   fieldPath: metadata.namespace
  495.           volumeMounts:
  496.             - name: run
  497.               mountPath: /run/flannel
  498.             - name: flannel-cfg
  499.               mountPath: /etc/kube-flannel/
  500.       volumes:
  501.         - name: run
  502.           hostPath:
  503.             path: /run/flannel
  504.         - name: cni
  505.           hostPath:
  506.             path: /etc/cni/net.d
  507.         - name: flannel-cfg
  508.           configMap:
  509.             name: kube-flannel-cfg
  510. ---
  511. apiVersion: apps/v1
  512. kind: DaemonSet
  513. metadata:
  514.   name: kube-flannel-ds-s390x
  515.   namespace: kube-system
  516.   labels:
  517.     tier: node
  518.     app: flannel
  519. spec:
  520.   selector:
  521.     matchLabels:
  522.       app: flannel
  523.   template:
  524.     metadata:
  525.       labels:
  526.         tier: node
  527.         app: flannel
  528.     spec:
  529.       affinity:
  530.         nodeAffinity:
  531.           requiredDuringSchedulingIgnoredDuringExecution:
  532.             nodeSelectorTerms:
  533.               - matchExpressions:
  534.                   - key: beta.kubernetes.io/os
  535.                     operator: In
  536.                     values:
  537.                       - linux
  538.                   - key: beta.kubernetes.io/arch
  539.                     operator: In
  540.                     values:
  541.                       - s390x
  542.       hostNetwork: true
  543.       tolerations:
  544.         - operator: Exists
  545.           effect: NoSchedule
  546.       serviceAccountName: flannel
  547.       initContainers:
  548.         - name: install-cni
  549.           image: quay.io/coreos/flannel:v0.12.0-s390x
  550.           command:
  551.             - cp
  552.           args:
  553.             - -f
  554.             - /etc/kube-flannel/cni-conf.json
  555.             - /etc/cni/net.d/10-flannel.conflist
  556.           volumeMounts:
  557.             - name: cni
  558.               mountPath: /etc/cni/net.d
  559.             - name: flannel-cfg
  560.               mountPath: /etc/kube-flannel/
  561.       containers:
  562.         - name: kube-flannel
  563.           image: quay.io/coreos/flannel:v0.12.0-s390x
  564.           command:
  565.             - /opt/bin/flanneld
  566.           args:
  567.             - --ip-masq
  568.             - --kube-subnet-mgr
  569.           resources:
  570.             requests:
  571.               cpu: "100m"
  572.               memory: "50Mi"
  573.             limits:
  574.               cpu: "100m"
  575.               memory: "50Mi"
  576.           securityContext:
  577.             privileged: false
  578.             capabilities:
  579.               add: ["NET_ADMIN"]
  580.           env:
  581.             - name: POD_NAME
  582.               valueFrom:
  583.                 fieldRef:
  584.                   fieldPath: metadata.name
  585.             - name: POD_NAMESPACE
  586.               valueFrom:
  587.                 fieldRef:
  588.                   fieldPath: metadata.namespace
  589.           volumeMounts:
  590.             - name: run
  591.               mountPath: /run/flannel
  592.             - name: flannel-cfg
  593.               mountPath: /etc/kube-flannel/
  594.       volumes:
  595.         - name: run
  596.           hostPath:
  597.             path: /run/flannel
  598.         - name: cni
  599.           hostPath:
  600.             path: /etc/cni/net.d
  601.         - name: flannel-cfg
  602.           configMap:
  603.             name: kube-flannel-cfg

注意net-conf.json的Network网段要和kubeadm init命令的—pod-network-cidr的网段一致

  1. 应用flannel
  1. kubectl apply -f kube-flannel.yml
  1. 应用flannel-node-annotator组件用来修改节点的public ip(重要)

编辑flannel-node-annotator.yml

  1. # flannel.alpha.coreos.com/public-ip-overwrite
  2. apiVersion: v1
  3. kind: ServiceAccount
  4. metadata:
  5.   name: flannel-node-annotator
  6.   namespace: kube-system
  7. ---
  8. apiVersion: rbac.authorization.k8s.io/v1beta1
  9. kind: ClusterRole
  10. metadata:
  11.   name: flannel-node-annotator
  12. rules:
  13.   - apiGroups:
  14.       - ""
  15.     resources:
  16.       - "nodes"
  17.     verbs:
  18.       - list
  19.       - get
  20.       - watch
  21.       - update
  22. ---
  23. apiVersion: rbac.authorization.k8s.io/v1beta1
  24. kind: ClusterRoleBinding
  25. metadata:
  26.   name: flannel-node-annotator
  27. roleRef:
  28.   apiGroup: rbac.authorization.k8s.io
  29.   kind: ClusterRole
  30.   name: flannel-node-annotator
  31. subjects:
  32.   - kind: ServiceAccount
  33.     name: flannel-node-annotator
  34.     namespace: kube-system
  35. ---
  36. apiVersion: apps/v1
  37. kind: Deployment
  38. metadata:
  39.   name: flannel-node-annotator
  40.   namespace: kube-system
  41. spec:
  42.   replicas: 1
  43.   selector:
  44.     matchLabels:
  45.       app: flannel-node-annotator
  46.   template:
  47.     metadata:
  48.       labels:
  49.         app: flannel-node-annotator
  50.     spec:
  51.       serviceAccountName: flannel-node-annotator
  52.       containers:
  53.         - image: alvaroaleman/flannel-node-annotator
  54.           imagePullPolicy: IfNotPresent
  55.           name: flannel-node-annotator

应用

  1. kubectl apply -f flannel-node-annotator.yml

修改节点的public ip

  1. kubectl edit nodes [node name]
  1. metadata:
  2.   annotations:
  3.     flannel.alpha.coreos.com/public-ip-overwrite: [node public ip]

重启flannel

  1. kubectl get pod -n kube-system | grep kube-flannel | awk '{system("kubectl delete pod "$1" -n kube-system")}'

初始化其他节点

  • 初始化
  1. kubeadm join [master public ip]:6443 --token [token] \
  2.     --discovery-token-ca-cert-hash [hash]

关于ipvs

kubernetes ipvs模式下可能存在bug,不建议使用

kubelet启动失败原因查看

  1. systemctl status kubelet

查看kubelet运行状态

  1. journalctl -xefu kubelet

查看失败日志

重置kubernetes


  1. ```bash sudo kubectl drain [node name]

sudo kubectl delete nodes [node name]

sudo kubeadm reset

  2. 2. 重置iptables
  4. ```bash
  5. iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
  6. sysctl net.bridge.bridge-nf-call-iptables=1
  1. 清除ipvs
  1. ipvsadm --clear
  1. 清除flannel
  1. rm -rf /var/lib/cni/flannel/* && rm -rf /var/lib/cni/networks/cbr0/* && ip link delete cni0
  2. rm -rf /var/lib/cni/networks/cni0/*
  1. 手动执行以下命令来清楚对应的残余网卡信息
  1. sudo ip link del cni0
  2. sudo ip link del flannel.1

其他多余网卡全部手动清除

kubernetes默认证书1年有效期修改

修改有效期

解除端口限制 nodePort: Invalid value valid ports is 30000-32767

kubelet证书轮换(需要版本v1.19.0或以上)

  1. vim /var/lib/kubelet/config.yaml

增加 rotateCertificates: true 参数
重启kubelet

  1. systemctl daemon-reload
  2. service kubelet restart