1、准备证书

1、创建证书

1.1、cfssl签发证书

在hdss7-200
cd /opt/certs/
vi dashboard-csr.json
输入：
{
“CN”: “*.jack.com”,
“hosts”: [
],
“key”: {
“algo”: “rsa”,
“size”: 2048
},
“names”: [
{
“C”: “CN”,
“ST”: “zhejiang”,
“L”: “hangzhou”,
“O”: “jack”,
“OU”: “ops”
}
]
}

生成证书：cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server dashboard-csr.json |cfssl-json -bare dashboard

1.2、拷贝证书

拷贝到我们nginx的服务器上：10.4.7.11 10.4.7.12 都需要
cd /etc/nginx/
mkdir certs
cd certs
scp hdss7-200:/opt/cert/dash* ./

2、OpenSSL签发证书（本次使用）

在10.4.7.200上

1、准备私钥

执行：(umask 077; openssl genrsa -out dashboard.jack.com.key 2048)
输出：
image.png

2、创建证书签发请求文件

执行：openssl req -new -key dashboard.jack.com.key -out dashboard.jack.com.csr -subj “/CN=dashboard.jack.com/C=CN/ST=zhejiang/L=hangzhou/O=jack/OU=ops”

执行：ll
输出：
image.png

3、签发证书：

执行：openssl x509 -req -in dashboard.jack.com.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out dashboard.jack.com.crt -days 3650
输出：
image.png
image.png

执行：cfssl certinfo -cert dashboard.jack.com.crt
输出：
image.png

4、拷贝证书 10.4.7.11 10.4.7.12

scp hdss7-200:/opt/certs/dashboard.jack.com.crt ./
scp hdss7-200:/opt/certs/dashboard.jack.com.key ./

3、配置dashboard 10.4.7.11 10.4.7.12

cd /etc/nginx/conf.d/
vi dashboard.jack.com.conf
输入：
server {
listen 80;
server_name dashboard.jack.com;

  1. rewrite ^(.*)$ https://${server_name}$1 permanent;<br />}<br />server {<br />    listen       443 ssl;<br />    server_name  dashboard.jack.com;
  3. ssl_certificate "certs/dashboard.jack.com.crt";<br />    ssl_certificate_key "certs/dashboard.jack.com.key";<br />    ssl_session_cache shared:SSL:1m;<br />    ssl_session_timeout  10m;<br />    ssl_ciphers HIGH:!aNULL:!MD5;<br />    ssl_prefer_server_ciphers on;
  5. location / {<br />        proxy_pass http://default_backend_traefik;<br />        proxy_set_header Host       $http_host;<br />        proxy_set_header x-forwarded-for $proxy_add_x_forwarded_for;<br />    }<br />}

nginx -t
nginx -s reload

2、准备DashBoard镜像(10.4.7.200上)

https://github.com/kubernetes/dashboard

docker pull kubernetesui/dashboard:v1.10.1

docker tag 9a07b5b4bfac harbor.jack.com/public/dashboard:v1.10.1

docker push harbor.jack.com/public/dashboard:v1.10.1

3、准备资源配置清单（10.4.7.200上）

mkdir p /data/k8s-yaml/dashboard
cd /data/k8s-yaml/dashboard

1、rbac.yaml

vi rbac.yaml
输入：
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
addonmanager.kubernetes.io/mode: Reconcile
name: kubernetes-dashboard-admin
namespace: kube-system
—-
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard-admin
namespace: kube-system
labels:
k8s-app: kubernetes-dashboard
addonmanager.kubernetes.io/mode: Reconcile
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard-admin
namespace: kube-system

2、dp.yaml

vi dp.yaml
输入：
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubernetes-dashboard
namespace: kube-system
labels:
k8s-app: kubernetes-dashboard
kubernetes.io/cluster-service: “true”
addonmanager.kubernetes.io/mode: Reconcile
spec:
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ‘’
spec:
priorityClassName: system-cluster-critical
containers:
- name: kubernetes-dashboard
image: harbor.jack.com/public/dashboard:v1.10.1
resources:
limits:
cpu: 100m
memory: 300Mi
requests:
cpu: 50m
memory: 100Mi
ports:
- containerPort: 8443
protocol: TCP
env:
- name: ACCEPT_LANGUAGE
value: english
args:
# PLATFORM-SPECIFIC ARGS HERE
- —auto-generate-certificates
volumeMounts:
- name: tmp-volume
mountPath: /tmp
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard-admin
tolerations:
- key: “CriticalAddonsOnly”
operator: “Exists”

3、svc.yaml

vi svc.yaml
输入：
apiVersion: v1
kind: Service
metadata:
name: kubernetes-dashboard
namespace: kube-system
labels:
k8s-app: kubernetes-dashboard
kubernetes.io/cluster-service: “true”
addonmanager.kubernetes.io/mode: Reconcile
spec:
selector:
k8s-app: kubernetes-dashboard
ports:
- port: 443
targetPort: 8443

4、ingress.yaml

vi ingress.yaml
输入：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubernetes-dashboard
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: dashboard.jack.com
http:
paths:
- backend:
serviceName: kubernetes-dashboard
servicePort: 443

4、创建资源配置清单

创建资源：任意node 7.21或7.22
kubectl create -f http://k8s-yaml.jack.com/dashboard/rbac.yaml
kubectl create -f http://k8s-yaml.jack.com/dashboard/dp.yaml
kubectl create -f http://k8s-yaml.jack.com/dashboard/svc.yaml
kubectl create -f http://k8s-yaml.jack.com/dashboard/ingress.yaml

kubectl apply -f http://k8s-yaml.jack.com/dashboard/rbac.yaml
kubectl apply -f http://k8s-yaml.jack.com/dashboard/dp.yaml
kubectl apply -f http://k8s-yaml.jack.com/dashboard/svc.yaml
kubectl apply -f http://k8s-yaml.jack.com/dashboard/ingress.yaml

5、添加域名解析

10.4.7.11上

vi /var/named/jack.com.zone

输入：dashboard A 10.4.7.10

systemctl restart named

通过浏览器访问：
http://dashboard.jack.com

10.4.7.21或10.4.7.22上执行：kubectl get secret -n kube-system
输出：
image.png

查看具体secret： kubectl describe secret kubernetes-dashboard-admin-token-xvgbr -n kube-system
输出：
Name: kubernetes-dashboard-admin-token-xvgbr
Namespace: kube-system
Labels:
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard-admin
kubernetes.io/service-account.uid: 373fb5a9-cfa5-4174-aac8-315b5925af60

Type: kubernetes.io/service-account-token

Data
====
ca.crt: 1342 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC1hZG1pbi10b2tlbi14dmdiciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC1hZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjM3M2ZiNWE5LWNmYTUtNDE3NC1hYWM4LTMxNWI1OTI1YWY2MCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlcm5ldGVzLWRhc2hib2FyZC1hZG1pbiJ9.NPKxtjn10SED-a7xiGp2vL79xDMPpWTekJsNnU1XWcNU-gUMdIncNsPxuAC6kVN9j9MPX9LBeEjmR8lkXXOCD2PgZzEsJ8db-b1wBwvXsNWJa5vkzE5HA_2hGY2E-8MxukVkzn5HilZPHIO9d4uCw0-debYJLgZjpE240kaLTL4ARdbdn7KjT5_wYyaQOvTzk5wHaE9c1N8ZBuVlfFDUqF3eE-TwiIuyOIsaOxf1veDfy0-ozylmlFIRn5PV7ueV8SnuQdjgTIxC6M9UiGs-eZUVPBifAmBTebUeiuB-bON-8SwXQ3K34aE9qykzDVusb5ABJb6rE6y3SH2Gav25sQ

运维主机上配置最小权限示例：
cd /data/k8s-yaml/dashboard
vi rbac-minimal.yaml
输入：
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
addonmanager.kubernetes.io/mode: Reconcile
name: kubernetes-dashboard
namespace: kube-system
—-
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
addonmanager.kubernetes.io/mode: Reconcile
name: kubernetes-dashboard-minimal
namespace: kube-system
rules:
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [“”]
resources: [“secrets”]
resourceNames: [“kubernetes-dashboard-key-holder”, “kubernetes-dashboard-certs”]
verbs: [“get”, “update”, “delete”]
# Allow Dashboard to get and update ‘kubernetes-dashboard-settings’ config map.
- apiGroups: [“”]
resources: [“configmaps”]
resourceNames: [“kubernetes-dashboard-settings”]
verbs: [“get”, “update”]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [“”]
resources: [“services”]
resourceNames: [“heapster”]
verbs: [“proxy”]
- apiGroups: [“”]
resources: [“services/proxy”]
resourceNames: [“heapster”, “http:heapster:”, “https:heapster:”]
verbs: [“get”]
—-
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
labels:
k8s-app: kubernetes-dashboard
addonmanager.kubernetes.io/mode: Reconcile
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system

执行（7.21或7.22上）：kubectl apply -f http://k8s-yaml.jack.com/dashboard/rbac-minimal.yaml
输出：
image.png

在7.200上修改vi /data/k8s-yaml/dashboard/dp.yaml
serviceAccountName: kubernetes-dashboard-admin改为serviceAccountName: kubernetes-dashboard

执行（7.21或7.22上）:kubectl apply -f http://k8s-yaml.jack.com/dashboard/dp.yaml
image.png

再去访问dashboard.jack.com
image.png

重新获取token登录：
image.png
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi1oZHFuaCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImZjOTVkZmM0LTU1ZDMtNGQ3MS1iMjY0LTg0YjM3NzkwYzY5MiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.eNnv3C6NiaDjwdK9M6qmuDKRtW-z_n1Ib5zFS8b3UNQJzpxeIVtHP2LKJa6Ifl-VXZu8hY7lznRV7Hy8Hh4Dyhwbuy9coW0-gbDugn2hPbHkVOm6w8n7rlm3fMJ5Fu3lhbFCpWHETrknC-JhHHTdXv_IEhdPrWmnViaNo4Ebz1hply7rLITdPKn0X2EH71XSsWm7IRc_RTpz0sBl35hnZm_5CnMNzfAnJ_r-_wegEUZipSjDFXU9rnrIo6y7aVR85Rb4pd9HKcK-a9WuT6g3JYNzqRuEzgQmn1ZhZ1ceU9VCLWdhE9-5RYfsa4NQxLm-_zS59gdfmPqwSsWUsagwFA

会发现能访问的东西很少——>权限变小

image.png

6、其他

查看集群权限：kubectl get clusterrole
输出：
NAME AGE
admin 14d
cluster-admin 14d
edit 14d
system:aggregate-to-admin 14d
system:aggregate-to-edit 14d
system:aggregate-to-view 14d
system:auth-delegator 14d
system:basic-user 14d
system:certificates.k8s.io:certificatesigningrequests:nodeclient 14d
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 14d
system:controller:attachdetach-controller 14d
system:controller:certificate-controller 14d
system:controller:clusterrole-aggregation-controller 14d
system:controller:cronjob-controller 14d
system:controller:daemon-set-controller 14d
system:controller:deployment-controller 14d
system:controller:disruption-controller 14d
system:controller:endpoint-controller 14d
system:controller:expand-controller 14d
system:controller:generic-garbage-collector 14d
system:controller:horizontal-pod-autoscaler 14d
system:controller:job-controller 14d
system:controller:namespace-controller 14d
system:controller:node-controller 14d
system:controller:persistent-volume-binder 14d
system:controller:pod-garbage-collector 14d
system:controller:pv-protection-controller 14d
system:controller:pvc-protection-controller 14d
system:controller:replicaset-controller 14d
system:controller:replication-controller 14d
system:controller:resourcequota-controller 14d
system:controller:route-controller 14d
system:controller:service-account-controller 14d
system:controller:service-controller 14d
system:controller:statefulset-controller 14d
system:controller:ttl-controller 14d
system:coredns 12d
system:csi-external-attacher 14d
system:csi-external-provisioner 14d
system:discovery 14d
system:heapster 14d
system:kube-aggregator 14d
system:kube-controller-manager 14d
system:kube-dns 14d
system:kube-scheduler 14d
system:kubelet-api-admin 14d
system:node 14d
system:node-bootstrapper 14d
system:node-problem-detector 14d
system:node-proxier 14d
system:persistent-volume-provisioner 14d
system:public-info-viewer 14d
system:volume-scheduler 14d
traefik-ingress-controller 3d16h
view 14d

获取集群管理员用户描述文件：kubectl get clusterrole cluster-admin -o yaml
输出：
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: “true”
creationTimestamp: “2021-08-14T10:57:44Z”
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: cluster-admin
resourceVersion: “40”
selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-admin
uid: abab78a1-3fe2-4b17-acca-ac3a13ef625f
rules:
- apiGroups:
- ‘‘
resources:
- ‘‘
verbs:
- ‘‘
- nonResourceURLs:
- ‘‘
verbs:
- ‘*’

查看默认的集群角色描述问价：kubectl get clusterrole system:node -o yaml
输出：
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: “true”
creationTimestamp: “2021-08-14T10:57:44Z”
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:node
resourceVersion: “51”
selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/system%3Anode
uid: 150557cd-0f57-4169-8e07-1f18db06398a
rules:
- apiGroups:
- authentication.k8s.io
resources:
- tokenreviews
verbs:
- create
- apiGroups:
- authorization.k8s.io
resources:
- localsubjectaccessreviews
- subjectaccessreviews
verbs:
- create
- apiGroups:
- “”
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- “”
resources:
- nodes
verbs:
- create
- get
- list
- watch
- apiGroups:
- “”
resources:
- nodes/status
verbs:
- patch
- update
- apiGroups:
- “”
resources:
- nodes
verbs:
- patch
- update
- apiGroups:
- “”
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- “”
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- “”
resources:
- pods
verbs:
- create
- delete
- apiGroups:
- “”
resources:
- pods/status
verbs:
- patch
- update
- apiGroups:
- “”
resources:
- pods/eviction
verbs:
- create
- apiGroups:
- “”
resources:
- configmaps
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- “”
resources:
- persistentvolumeclaims
- persistentvolumes
verbs:
- get
- apiGroups:
- “”
resources:
- endpoints
verbs:
- get
- apiGroups:
- certificates.k8s.io
resources:
- certificatesigningrequests
verbs:
- create
- get
- list
- watch
- apiGroups:
- “”
resources:
- persistentvolumeclaims/status
verbs:
- get
- patch
- update
- apiGroups:
- “”
resources:
- serviceaccounts/token
verbs:
- create
- apiGroups:
- storage.k8s.io
resources:
- volumeattachments
verbs:
- get
- apiGroups:
- storage.k8s.io
resources:
- csidrivers
verbs:
- get
- list
- watch
- apiGroups:
- storage.k8s.io
resources:
- csinodes
verbs:
- create
- delete
- get
- patch
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- delete
- get
- patch
- update
- apiGroups:
- node.k8s.io
resources:
- runtimeclasses
verbs:
- get
- list
- watch

apiGroups：表示某个资源组
resources:：表示资源组的某个资源
verbs：哪些权限
表示在哪些资源组下的某些资源拥有什么样的权限

10.4.7.21上：cd /opt/kubernetes/server/bin/conf
image.png

官方：
image.png

image.png

7、彩蛋（dashboard的插件heapster）

在10.4.7.200上执行：

1、获取heapster镜像，并推送到自己的harbor仓库

1、cd /data/k8s-yaml/dashboard
2、mkdir heapster
3、cd heapster
4、docker pull quay.io/bitnami/heapster:1.5.4
image.png
5、docker tag c359b95ad38b harbor.jack.com/public/heapster:v1.5.4
6、docker push harbor.jack.com/public/heapster:v1.5.4
image.png

2、配置资源清单

1、vi rbac.yaml
输入：
apiVersion: v1
kind: ServiceAccount
metadata:
name: heapster
namespace: kube-system
—-
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: heapster
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:heapster
subjects:
- kind: ServiceAccount
name: heapster
namespace: kube-system

2、vi dp.yaml
输入：
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: heapster
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: heapster
spec:
serviceAccountName: heapster
containers:
- name: heapster
image: harbor.jack.com/public/heapster:v1.5.4
imagePullPolicy: IfNotPresent
command:
- /opt/bitnami/heapster/bin/heapster
- —source=kubernetes:https://kubernetes.default

3、vi svc.yaml
输入：
apiVersion: v1
kind: Service
metadata:
labels:
task: monitoring
# For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons))
# If you are NOT using this as an addon, you should comment out this line.
kubernetes.io/cluster-service: ‘true’
kubernetes.io/name: Heapster
name: heapster
namespace: kube-system
spec:
ports:
- port: 80
targetPort: 8082
selector:
k8s-app: heapster

3、应用 （10.4.7.21或10.4.7.21）

kubectl apply -f http://k8s-yaml.jack.com/dashboard/heapster/rbac.yaml
kubectl apply -f http://k8s-yaml.jack.com/dashboard/heapster/dp.yaml
kubectl apply -f http://k8s-yaml.jack.com/dashboard/heapster/svc.yaml
image.png