- 角色组件设置
- 获取二进制运行文件
- 创建证书
- 创建token文件
- 第一列：随机字符串，自己可生成
- 第二列：用户名
- 第三列：UID
- 第四列：用户组
- 运行kube-scheduler
- 运行 kube-controller-manager
- node 节点部署组件
- ———————————————————————————————————————-
- 创建kubelet 配置文件
- 设置集群参数
- 设置客户端认证参数
- 设置上下文参数
- 设置默认上下文
- ————————————————————————————————————————-
- 创建 kube-proxy 配置文件
- 设置集群参数
- 设置客户端认证参数
- 设置上下文参数
- 设置默认上下文
- config 文件内容
- cat /data/server/k8s/cfg/kubelet-config.yml
- kubectl get csr
- kubectl certificate approve node-csr-Tju4S-Nabh5S-rinQVhIylvAv0_6eIJOtXalWgGO4_A
- kubectl get csr
- 上面这一步如果CONDITION 显示为Approved，说明你通过了node 节点的加入申请，但是证书还没有下发到node 节点，稍等一会，再次查看，如果为Approved,Issued 则为正确，如果还是Approved，说明controller 或者scheduler 有问题，需要查看日志进行排查
- kubectl get node
下载地址：
获取方式（gtihub,改变历史的md文件）
https://dl.k8s.io/v1.16.9/kubernetes-server-linux-amd64.tar.gz
下载方式二：（单个下载）
https://www.downloadkubernetes.com/
角色组件设置
| master | kube-apiserver，kube-controller-manager，kube-scheduler，docker |
|---|---|
| node | kubelet，kube-proxy，docker，flannel |
获取二进制运行文件
mkdir -p /opt/k8s-package && cd /opt/k8s-packagewget https://dl.k8s.io/v1.16.9/kubernetes-server-linux-amd64.tar.gztar xf kubernetes-server-linux-amd64.tar.gzcd kubernetes/server/bin/ && cp kube-apiserver kube-controller-manager kubectl kube-proxy kube-scheduler /usr/local/k8scp kubectl /usr/local/bin
创建证书
创建证书目录
mkdir -p /usr/local/ssl/k8s cd /usr/local/ssl/k8s创建ca证书 ```bash cat <
ca-config.json { “signing”: { “default”: { "expiry": "876000h"}, “profiles”: {
"kubernetes": { "expiry": "876000h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] }} } } EOF
cat <
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
- 生成apiserver证书 证书里要写上master节点的ip,要不然加入不进去
```bash
cat <<EOF > server-csr.json
{
"CN": "kubernetes",
"hosts": [
"10.0.0.1",
"127.0.0.1",
"192.168.13.60",
"192.168.13.61",
"192.168.0.0/17",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
- 生成kube-proxy 证书
```bash
cat <
kube-proxy-csr.json { “CN”: “system:kube-proxy”, “hosts”: [], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ {
} ] } EOF"C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System"
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
- 结果
```bash
ls |grep -v json
ca.csr
ca-key.pem
ca.pem
kube-proxy.csr
kube-proxy-key.pem
kube-proxy.pem
server.csr
server-key.pem
server.pem
- 复制证书
mkdir -p /usr/local/k8s/ssl cp /usr/local/ssl/k8s/*.pem /usr/local/k8s/ssl/创建token文件
```bash cat /usr/local/k8s/token.csv 68b329da9893e34099c7d8ad5cb9c940,kubelet-bootstrap,10001,”system:node-bootstrapper”
第一列：随机字符串，自己可生成
第二列：用户名
第三列：UID
第四列：用户组
<a name="JbHKk"></a>
### 运行kube-apiserver
```bash
mkdir /var/log/k8s
/usr/local/k8s/kube-apiserver --logtostderr=false \
--v=2 \
--log-dir=/var/log/k8s/apiserver.log \
--etcd-servers=https://192.168.13.60:2379 \
--bind-address=192.168.13.60 \
--secure-port=6443 \
--advertise-address=192.168.13.60 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/16 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth=true \
--token-auth-file=/usr/local/k8s/token.csv \
--service-node-port-range=30000-32767 \
--kubelet-client-certificate=/usr/local/k8s/ssl/server.pem \
--kubelet-client-key=/usr/local/k8s/ssl/server-key.pem \
--tls-cert-file=/usr/local/k8s/ssl/server.pem \
--tls-private-key-file=/usr/local/k8s/ssl/server-key.pem \
--client-ca-file=/usr/local/k8s/ssl/ca.pem \
--service-account-key-file=/usr/local/k8s/ssl/ca-key.pem \
--etcd-cafile=/usr/local/etcd/ssl/ca.pem --etcd-certfile=/usr/local/etcd/ssl/server.pem \
--etcd-keyfile=/usr/local/etcd/ssl/server-key.pem \
--audit-log-maxage=30 \
--audit-log-maxbackup=3 \
--audit-log-maxsize=100 \
--audit-log-path=/var/log/k8s/apiserver-audit.log
参数说明： —logtostderr 关闭日志 —v 日志等级 —etcd-servers etcd集群地址 —bind-address 监听地址 —secure-port 监听端口 —advertise-address 集群通告地址 —allow-privileged 启用授权 —service-cluster-ip-range service IP虚拟地址段 —enable-admission-plugins 启用插件 —authorization-mode 认证授权，启用RBAC授权和节和节点自管理 —enable-bootstrap-token-auth 启用 TLS bootstrap 功能 —token-auth-file TLS bootstrap 功能的 token 文件 —service-node-port-range service node 类型默认端口分配范围
剩下的就是证书 和 审计日志的配置，审计日志可以不配置，也就是以audit-log 开头的都可以删除掉
supervisor管理 apiserver
cat <<EOF > /etc/supervisor/conf.d/apiserver.conf
[program:apiserver]
user = root
command=/usr/local/k8s/kube-apiserver --logtostderr=false
--v=2
--log-dir=/var/log/k8s/apiserver.log
--etcd-servers=https://192.168.13.60:2379
--bind-address=192.168.13.60
--secure-port=6443
--advertise-address=192.168.13.60
--allow-privileged=true
--service-cluster-ip-range=10.0.0.0/16
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction
--authorization-mode=RBAC,Node
--enable-bootstrap-token-auth=true
--token-auth-file=/usr/local/k8s/token.csv
--service-node-port-range=30000-32767 --kubelet-client-certificate=/usr/local/k8s/ssl/server.pem
--kubelet-client-key=/usr/local/k8s/ssl/server-key.pem
--tls-cert-file=/usr/local/k8s/ssl/server.pem
--tls-private-key-file=/usr/local/k8s/ssl/server-key.pem
--client-ca-file=/usr/local/k8s/ssl/ca.pem
--service-account-key-file=/usr/local/k8s/ssl/ca-key.pem
--etcd-cafile=/usr/local/etcd/ssl/ca.pem --etcd-certfile=/usr/local/etcd/ssl/server.pem
--etcd-keyfile=/usr/local/etcd/ssl/server-key.pem
--audit-log-maxage=30
--audit-log-maxbackup=3
--audit-log-maxsize=100
--audit-log-path=/var/log/k8s/apiserver-audit.log
stderr_logfile = /var/log/supervisor/apiserver_err.log
stdout_logfile = /var/log/supervisor/apiserver_stdout.log
autostart=true
autorestart=true
startsecs=3
EOF
supervisorctl update
运行kube-scheduler
/usr/local/k8s/kube-scheduler \
--logtostderr=false \
--v=2 \
--log-dir=/var/log/k8s/scheduler.log \
--leader-elect \
--master=127.0.0.1:8080 \
--bind-address=127.0.0.1
参数说明： —logtostderr 日志是否打印 —v 日志等级 —log-dir 日志文件 —leader-elect 当该组件启动多个时，自动选举（HA） —master 连接本地的 apiserver —bind-address 监听地址
supervisor管理 scheduler
cat <<EOF > /etc/supervisor/conf.d/scheduler.conf
[program:scheduler]
user = root
command=/usr/local/k8s/kube-scheduler
--logtostderr=false
--v=2
--log-dir=/var/log/k8s/scheduler.log
--leader-elect
--master=127.0.0.1:8080
--bind-address=127.0.0.1
stderr_logfile = /var/log/supervisor/scheduler_err.log
stdout_logfile = /var/log/supervisor/scheduler_stdout.log
autostart=true
autorestart=true
startsecs=3
EOF
supervisorctl update
运行 kube-controller-manager
/usr/local/k8s/kube-controller-manager \
--logtostderr=false \
--v=2 \
--log-dir=/var/log/k8s/controller.log \
--leader-elect=true \
--master=127.0.0.1:8080 \
--bind-address=127.0.0.1 \
--allocate-node-cidrs=true \
--cluster-cidr=10.244.0.0/16 \
--service-cluster-ip-range=10.0.0.0/16 \
--cluster-signing-cert-file=/usr/local/k8s/ssl/ca.pem \
--cluster-signing-key-file=/usr/local/k8s/ssl/ca-key.pem \
--root-ca-file=/usr/local/k8s/ssl/ca.pem \
--service-account-private-key-file=/usr/local/k8s/ssl/ca-key.pem
参数说明： —logtostderr 日志是否开启 —v 日志等级 —log-dir 日志文件 —leader-elect 当该组件启动多个时，自动选举（HA） —master 连接本地 apiserver
—bind-address 监听地址 —allocate-node-cidrs 基于云驱动来为 Pod 分配和设置子网掩码 —cluster-cidr 集群中 Pods 的 CIDR 范围。要求 —allocate-node-cidrs 标志为 true —service-cluster-ip-range 集群中 Service 对象的 CIDR 范围。要求 —allocate-node-cidrs 标志为 true 余下为证书配置
supervisor管理 controller
cat <<EOF > /etc/supervisor/conf.d/controller.conf
[program:controller]
user = root
command=/usr/local/k8s/kube-controller-manager
--logtostderr=false
--v=2
--log-dir=/var/log/k8s/controller.log
--leader-elect=true
--master=127.0.0.1:8080
--bind-address=127.0.0.1
--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16
--service-cluster-ip-range=10.0.0.0/24
--cluster-signing-cert-file=/usr/local/k8s/ssl/ca.pem
--cluster-signing-key-file=/usr/local/k8s/ssl/ca-key.pem
--root-ca-file=/usr/local/k8s/ssl/ca.pem
--service-account-private-key-file=/usr/local/k8s/ssl/ca-key.pem
stderr_logfile = /var/log/supervisor/controller_err.log
stdout_logfile = /var/log/supervisor/controller_stdout.log
autostart=true
autorestart=true
startsecs=3
EOF
supervisorctl update
说明
各个组件的具体配置解释可以去参考官网解释：组件配置详解
验证当前组件状态
- 如下即为正常
NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {"health":"true","reason":""}node 节点部署组件
以下操作还在master 节点操作将 kubelet-bootstrap用户绑定到系统集群角色
kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap生成 node 节点所需的配置文件
```bash cd /usr/local/k8s———————————————————————————————————————-
创建kubelet 配置文件
设置集群参数
kubectl config set-cluster kubernetes —certificate-authority=ssl/ca.pem —embed-certs=true —server=https://192.168.13.60:6443 —kubeconfig=bootstrap.kubeconfig设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap —token=68b329da9893e34099c7d8ad5cb9c940 —kubeconfig=bootstrap.kubeconfig设置上下文参数
kubectl config set-context default —cluster=kubernetes —user=kubelet-bootstrap —kubeconfig=bootstrap.kubeconfig设置默认上下文
kubectl config use-context default —kubeconfig=bootstrap.kubeconfig
————————————————————————————————————————-
创建 kube-proxy 配置文件
设置集群参数
kubectl config set-cluster kubernetes \ —certificate-authority=ssl/ca.pem \ —embed-certs=true \ —server=https://192.168.13.60:6443 \ —kubeconfig=kube-proxy.kubeconfig
设置客户端认证参数
kubectl config set-credentials kube-proxy \ —client-certificate=ssl/kube-proxy.pem \ —client-key=ssl/kube-proxy-key.pem \ —embed-certs=true \ —kubeconfig=kube-proxy.kubeconfig
设置上下文参数
kubectl config set-context default \ —cluster=kubernetes \ —user=kube-proxy \ —kubeconfig=kube-proxy.kubeconfig
设置默认上下文
kubectl config use-context default —kubeconfig=kube-proxy.kubeconfig
以上操作会生成两个配置文件，`bootstrap.kubeconfig kube-proxy.kubeconfig` ，命令添加的信息也在里边，可以cat 查看一下，加深理解。
<a name="v3eN4"></a>
#### node 节点准备工作
- 创建工作文件夹
```bash
mkdir /usr/local/k8s/{bin,cfg,ssl} -p && cd /usr/local/k8s/
将master 节点的生成的配置拷贝过来
scp -r root@192.168.13.60:/opt/k8s-package/* ./bin/ scp -r root@192.168.13.60:/usr/local/k8s/bootstrap.kubeconfig ./cfg/ scp -r root@192.168.13.60:/usr/local/k8s/kube-proxy.kubeconfig ./cfg/部署 kubelet
直接命令行启动
需要修改的配置：hostname-override
# 创建日志文件 mkdir /var/log/k8s/ -p touch /var/log/k8s/kubelet.log/data/server/k8s/bin/kubelet \ --logtostderr=false \ --v=2 \ --network-plugin=cni \ --log-dir=/var/log/k8s/kubelet.log \ --hostname-override=192.168.13.60 \ --kubeconfig=/data/server/k8s/cfg/kubelet.kubeconfig \ --bootstrap-kubeconfig=/data/server/k8s/cfg/bootstrap.kubeconfig \ --config=/data/server/k8s/cfg/kubelet-config.yml \ --cert-dir=/data/server/k8s/ssl/ \ --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0参数说明：
—hostname-override 在集群中显示的主机名
—kubeconfig 指定kubeconfig 配置文件，会自动生成
—bootstrap-kubeconfig 指定刚才复制过来的配置文件
—cert-dir master 节点签发的证书，加入集群后会自动生成
—pod-infra-container-image 管理pod 的网络的镜像kubelet 的配置文件 ```bash
config 文件内容
cat /data/server/k8s/cfg/kubelet-config.yml
kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 address: 0.0.0.0 port: 10250 readOnlyPort: 10255 cgroupDriver: systemd clusterDNS:
10.0.0.2 clusterDomain: cluster.local failSwapOn: false authentication: anonymous: enabled: false webhook: cacheTTL: 2m0s enabled: true authorization: mode: Webhook webhook: cacheAuthorizedTTL: 5m0s cacheUnauthorizedTTL: 30s evictionHard: imagefs.available: 15% memory.available: 100Mi nodefs.available: 10% nodefs.inodesFree: 5% maxOpenFiles: 1000000 maxPods: 210 ```
同样 supervisor 启动kublete
# cat /etc/supervisor/conf.d/kubelet.conf [program:kubelet] user = root command=/data/server/k8s/bin/kubelet --logtostderr=false --v=2 --network-plugin=cni --log-dir=/var/log/k8s/kubelet.log --hostname-override=192.168.13.56 --kubeconfig=/data/server/k8s/cfg/kubelet.kubeconfig --bootstrap-kubeconfig=/data/server/k8s/cfg/bootstrap.kubeconfig --config=/data/server/k8s/cfg/kubelet-config.yml --cert-dir=/data/server/k8s/ssl/ --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0 stderr_logfile = /var/log/supervisor/kubelet_err.log stdout_logfile = /var/log/supervisor/kubelet_stdout.log directory = /data/server/k8s/ autostart=true autorestart=true startsecs=3master 节点审批node 节点进入集群
```bash
kubectl get csr
NAME AGE REQUESTOR CONDITION node-csr-Tju4S-Nabh5S-rinQVhIylvAv0_6eIJOtXalWgGO4_A 1m kubelet-bootstrap Pending
kubectl certificate approve node-csr-Tju4S-Nabh5S-rinQVhIylvAv0_6eIJOtXalWgGO4_A
kubectl get csr
NAME AGE REQUESTOR CONDITION node-csr-Tju4S-Nabh5S-rinQVhIylvAv0_6eIJOtXalWgGO4_A 1m kubelet-bootstrap Approved,Issued
上面这一步如果CONDITION 显示为Approved，说明你通过了node 节点的加入申请，但是证书还没有下发到node 节点，稍等一会，再次查看，如果为Approved,Issued 则为正确，如果还是Approved，说明controller 或者scheduler 有问题，需要查看日志进行排查
kubectl get node
**部署 kube-proxy**
- node 节点直接命令行启动
```bash
/data/server/k8s/bin/kube-proxy \
--logtostderr=false \
--v=2 \
--log-dir=/var/log/k8s/kube-proxy.log \
--hostname-override=192.168.13.56 \
--cluster-cidr=10.0.0.0/24 \
--kubeconfig=/data/server/k8s/cfg/kube-proxy.kubeconfig
测试运行实例
至此我们已经部署完成了整个kubernetes 集群，然我们来测试下，运行个nginx。
# 主机点运行
kubectl run nginx --image=nginx
# 查看pod
kubectl get pod
若有收获，就点个赞吧
0 人点赞
