KubeKey 一键安装 K8S 集群 或者 KubeSphere 集群。

01 安装依赖

环境：Ubuntu 20.04

  • sudo apt install socat
  • sudo apt install conntrack
  • sudo apt install ebtables
  • sudo apt install ipset

02 安装

执行：export KKZONE=cn

需要所有节点都执行这个命令？初步判定不需要，这个主要是解决国内下载 Package 的问题。

02-1 快速安装

单节点安装所有服务，且会安装 kubesphere。

./kk create cluster [--with-kubernetes version] [--with-kubesphere version]

02-2 高级安装

  • 创建示例配置文件

    ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path]

  • 创建示例配置文件 - 不安装 kubesphere

    ./kk create config [-f ~/myfolder/abc.yaml]，示例内容如下：

    1. apiVersion: kubekey.kubesphere.io/v1alpha2
    2. kind: Cluster
    3. metadata:
    4. name: sample
    5. spec:
    6. hosts:
    7. - {name: node1, address: 172.16.0.2, internalAddress: 172.16.0.2, user: ubuntu, password: Qcloud@123}
    8. - {name: node2, address: 172.16.0.3, internalAddress: 172.16.0.3, user: ubuntu, password: Qcloud@123}
    9. roleGroups:
    10.   etcd:
    11.   - node1
    12.   master:
    13.   - node1
    14.   worker:
    15.   - node1
    16.   - node2
    17. controlPlaneEndpoint:
    18.   ##Internal loadbalancer for apiservers
    19.   #internalLoadbalancer: haproxy
    20.   domain: lb.kubesphere.local
    21.   address: ""
    22.   port: 6443
    23. kubernetes:
    24.   version: v1.21.5
    25.   clusterName: cluster.local
    26. network:
    27.   plugin: calico
    28.   kubePodsCIDR: 10.233.64.0/18
    29.   kubeServiceCIDR: 10.233.0.0/18
    30.   # multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    31.   enableMultusCNI: false
    32. registry:
    33.   registryMirrors: []
    34.   insecureRegistries: []
    35. addons: []

  • 创建安装 kubeshpere 的 示例配置文件

./kk create config --with-kubesphere [-f ~/myfolder/abc.yaml]，内容如下：

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: node1, address: 172.16.0.2, internalAddress: 172.16.0.2, user: ubuntu, password: Qcloud@123}
  - {name: node2, address: 172.16.0.3, internalAddress: 172.16.0.3, user: ubuntu, password: Qcloud@123}
  roleGroups:
    etcd:
    - node1
    master:
    - node1
    worker:
    - node1
    - node2
  controlPlaneEndpoint:
    ##Internal loadbalancer for apiservers
    #internalLoadbalancer: haproxy
    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.21.5
    clusterName: cluster.local
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    # multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    enableMultusCNI: false
  registry:
    registryMirrors: []
    insecureRegistries: []
  addons: []

  ---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
  name: ks-installer
  namespace: kubesphere-system
  labels:
    version: v3.2.1
spec:
  persistence:
    storageClass: ""
  authentication:
    jwtSecret: ""
  local_registry: ""
  # dev_tag: ""
  etcd:
    monitoring: false
    endpointIps: localhost
    port: 2379
    tlsEnable: true
  common:
    core:
      console:
        enableMultiLogin: true
        port: 30880
        type: NodePort
    # apiserver:
    #  resources: {}
    # controllerManager:
    #  resources: {}
    redis:
      enabled: false
      volumeSize: 2Gi
    openldap:
      enabled: false
      volumeSize: 2Gi
    minio:
        volumeSize: 20Gi
    monitoring:
      # type: external
      endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
      GPUMonitoring:
        enabled: false
    gpu:
      kinds:
      - resourceName: "nvidia.com/gpu"
        resourceType: "GPU"
        default: true
    es:
      # master:
      #   volumeSize: 4Gi
      #   replicas: 1
      #   resources: {}
      # data:
      #   volumeSize: 20Gi
      #   replicas: 1
      #   resources: {}
      logMaxAge: 7
      elkPrefix: logstash
      basicAuth:
        enabled: false
        username: ""
        password: ""
      externalElasticsearchHost: ""
      externalElasticsearchPort: ""
  alerting:
    enabled: false
    # thanosruler:
    #   replicas: 1
    #   resources: {}
  auditing:
    enabled: false
    # operator:
    #   resources: {}
    # webhook:
    #   resources: {}
  devops:
    enabled: false
    jenkinsMemoryLim: 2Gi
    jenkinsMemoryReq: 1500Mi
    jenkinsVolumeSize: 8Gi
    jenkinsJavaOpts_Xms: 512m
    jenkinsJavaOpts_Xmx: 512m
    jenkinsJavaOpts_MaxRAM: 2g
  events:
    enabled: false
    # operator:
    #   resources: {}
    # exporter:
    #   resources: {}
    # ruler:
    #   enabled: true
    #   replicas: 2
    #   resources: {}
  logging:
      enabled: false
    containerruntime: docker
    logsidecar:
      enabled: true
      replicas: 2
      # resources: {}
  metrics_server:
    enabled: false
  monitoring:
    storageClass: ""
    # kube_rbac_proxy:
    #   resources: {}
    # kube_state_metrics:
    #   resources: {}
    # prometheus:
    #   replicas: 1
    #   volumeSize: 20Gi
    #   resources: {}
    #   operator:
    #     resources: {}
    #   adapter:
    #     resources: {}
    # node_exporter:
    #   resources: {}
    # alertmanager:
    #   replicas: 1
    #   resources: {}
    # notification_manager:
    #   resources: {}
    #   operator:
    #     resources: {}
    #   proxy:
    #     resources: {}
    gpu:
      nvidia_dcgm_exporter:
        enabled: false
        # resources: {}
  multicluster:
    clusterRole: none
  network:
    networkpolicy:
      enabled: false
    ippool:
      type: none
    topology:
      type: none
  openpitrix:
    store:
      enabled: false
    servicemesh:
    enabled: false
  kubeedge:
    enabled: false
    cloudCore:
      nodeSelector: {"node-role.kubernetes.io/worker": ""}
      tolerations: []
      cloudhubPort: "10000"
      cloudhubQuicPort: "10001"
      cloudhubHttpsPort: "10002"
      cloudstreamPort: "10003"
      tunnelPort: "10004"
      cloudHub:
        advertiseAddress:
          - ""
        nodeLimit: "100"
      service:
        cloudhubNodePort: "30000"
        cloudhubQuicNodePort: "30001"
        cloudhubHttpsNodePort: "30002"
        cloudstreamNodePort: "30003"
        tunnelNodePort: "30004"
    edgeWatcher:
      nodeSelector: {"node-role.kubernetes.io/worker": ""}
      tolerations: []
      edgeWatcherAgent:
        nodeSelector: {"node-role.kubernetes.io/worker": ""}
        tolerations: []

02-3 开始安装

修改配置并安装：./kk create cluster -f config.yaml

03 日志

  • 错误一

    I0106 15:09:26.528643    8281 version.go:254] remote version is much newer: v1.23.1; falling back to: stable-1.21

    04 需要注意的问题

  • 需要确保节点之间的相关端口是通的，而不仅仅是可以 ping 通。

    需要注意的是：青云公有云每个云服务器都有自己的安全组，可以直接选择使用 VPC 网络的安全组，但是安全组的规则设置为组内互信。

  • 安装完成之后，节点名变更为 config.yaml 里的节点配置 name 字段。

    因此在配置 config.yaml 时，注意配置节点配置的 name 字段。