- apiServer的资源请求
客户端—>API server
API Request path路径格式
http://172.16.0.11:6443/apis/apps/v1/namespaces/kube-system/deployments/myapp-deploy
HTTP request verb: http动作
get,post,put,delete
API requests verb: 对应k8s的动作
get ,list,create,update,patch,watch,proxy,redirect,delete,deletecollection
当希望通过本地curl方式获取API信息时,我们可以使用以下方式kubectl proxy方式实现
[root@master ~]# curl https://127.0.0.1/api/v1/namespacescurl: (7) Failed connect to 127.0.0.1:443; Connection refused[root@master ~]# kubectl proxy --port=8080 &[1] 7760[root@master ~]# curl http://127.0.0.1:8080/api/v1/namespaces{"kind": "NamespaceList","apiVersion": "v1","metadata": {"resourceVersion": "323168"},.....
1 认证
1.1 k8s集群上的两种账户
k8s集群上面的用户按照使用的对象分类可以分为两种:
普通用户:即日常工程师登录k8s平台使用的账户,给人使用的
- ServiceAccount:给pod等资源类型使用的,如pod要访问apiServer,此时的用户就叫serviceaccount,在k8s集群上也是一种资源类型
1.2 ServiceAccount定义
我们可以通过命令创建相应的sa账号,当我们使用命令创建好sa账号之后,同时也会自动创建好对应的token信息,即secret存储卷[root@master ~]# kubectl create sa sa-xscserviceaccount/sa-xsc created[root@master ~]# kubectl describe sa sa-xscName: sa-xscNamespace: defaultLabels: <none>Annotations: <none>Image pull secrets: <none>Mountable secrets: sa-xsc-token-g7p8gTokens: sa-xsc-token-g7p8gEvents: <none>查看自动创建的secret[root@master ~]# kubectl get secretNAME TYPE DATA AGEdefault-token-wcbv6 kubernetes.io/service-account-token 3 60dmysql-root-password Opaque 1 7dnfs-client-provisioner-token-6vwzq kubernetes.io/service-account-token 3 58dsa-xsc-token-g7p8g kubernetes.io/service-account-token 3 8m32s
1.3 pod使用自定义的sa账号
在创建pod时,我们可以使用ServiceAccount字段指定pod使用的账户,当该pod以该账户启动之后,该pod也就具有了该账户所具有的权限
使用自定义的sa账号[root@master ~]# cat sa-pod-demo.yamlapiVersion: apps/v1kind: Deploymentmetadata:labels:app: sa-demoname: sa-demospec:replicas: 1selector:matchLabels:app: sa-demostrategy: {}template:metadata:labels:app: sa-demospec:containers:- image: nginxname: nginxresources: {}serviceAccount: sa-xsc查看pod使用的token信息[root@master ~]# kubectl describe pod sa-demo-d877998d-mr8lkName: sa-demo-d877998d-mr8lkNamespace: defaultPriority: 0Node: node02/10.0.0.12Start Time: Fri, 25 Jun 2021 16:50:35 +0800Labels: app=sa-demopod-template-hash=d877998dAnnotations: cni.projectcalico.org/podIP: 10.244.140.124/32cni.projectcalico.org/podIPs: 10.244.140.124/32Status: RunningIP: 10.244.140.124IPs:IP: 10.244.140.124Controlled By: ReplicaSet/sa-demo-d877998dContainers:nginx:Container ID: docker://499ea018a47990fc47c1eaf6efbf07caa1ead80dda826c5f84ed5f0c7e317ec7Image: nginxImage ID: docker-pullable://nginx@sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09dbPort: <none>Host Port: <none>State: RunningStarted: Fri, 25 Jun 2021 16:50:56 +0800Ready: TrueRestart Count: 0Environment: <none>Mounts:/var/run/secrets/kubernetes.io/serviceaccount from sa-xsc-token-g7p8g (ro)Conditions:Type StatusInitialized TrueReady TrueContainersReady TruePodScheduled TrueVolumes:sa-xsc-token-g7p8g:Type: Secret (a volume populated by a Secret)SecretName: sa-xsc-token-g7p8gOptional: falseQoS Class: BestEffortNode-Selectors: <none>Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300snode.kubernetes.io/unreachable:NoExecute op=Exists for 300sEvents:Type Reason Age From Message---- ------ ---- ---- -------Normal Scheduled 55s default-scheduler Successfully assigned default/sa-demo-d877998d-mr8lk to node02Normal Pulling 54s kubelet Pulling image "nginx"Normal Pulled 34s kubelet Successfully pulled image "nginx" in 19.693135511sNormal Created 34s kubelet Created container nginxNormal Started 34s kubelet Started container nginx
2 kubeconfig
```yaml [root@master ~]# kubectl config view apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://10.0.0.10:6443 name: kubernetes contexts:
- context: cluster: kubernetes user: kubernetes-admin name: kubernetes-admin@kubernetes current-context: kubernetes-admin@kubernetes kind: Config preferences: {} users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
```
2.1 创建自签证书
```yaml [root@master pki]# (umask 077; openssl genrsa -out scxiang.key 2048) Generating RSA private key, 2048 bit long modulus ….+++ …………………………………………………+++ e is 65537 (0x10001) [root@master pki]# openssl req -new -key scxiang.key -out scxiang.csr -subj “/CN=scxiang” You have new mail in /var/spool/mail/root [root@master pki]# openssl x509 -req -in scxiang.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out scxiang.crt -days 3650 Signature ok subject=/CN=scxiang Getting CA Private Key [root@master pki]# openssl x509 -in scxiang.crt -text -noout Certificate: Data:
Signature Algorithm: sha256WithRSAEncryptionVersion: 1 (0x0)Serial Number:f4:69:d5:a5:6f:5a:d1:6f
Signature Algorithm: sha256WithRSAEncryptionIssuer: CN=kubernetesValidityNot Before: Jun 26 09:47:45 2021 GMTNot After : Jun 24 09:47:45 2031 GMTSubject: CN=scxiangSubject Public Key Info:Public Key Algorithm: rsaEncryptionPublic-Key: (2048 bit)Modulus:00:bc:64:b7:c5:f5:c4:4c:53:b9:49:db:ab:17:37:77:51:3e:42:a6:5f:7c:30:d4:12:14:26:24:af:c1:fb:1b:0e:4a:da:54:19:e8:11:db:f3:4a:93:35:69:b3:cf:4e:13:1f:a2:94:da:39:e6:60:9d:19:c2:b4:ea:1c:76:04:74:94:bc:99:78:ed:8d:e3:30:59:b5:2e:70:3f:00:df:07:8f:43:81:fa:30:9f:4b:13:11:9a:de:c5:9d:fb:2f:b8:46:93:e1:40:a7:ba:b2:15:69:93:87:6b:79:16:44:22:e1:14:90:69:18:d0:10:01:ae:46:47:68:58:08:21:5a:01:89:b3:81:c0:cd:f5:99:34:94:a5:60:0b:8c:30:5d:de:8a:4f:c0:69:54:92:37:d3:2e:30:87:46:24:1f:1e:ec:78:1d:35:1e:19:80:7f:f1:f9:c5:99:08:ac:a0:f1:ab:89:9f:ea:c6:e1:4d:35:6a:de:b0:72:f5:98:f9:73:f6:f3:5b:49:d8:c4:65:13:89:66:a5:c0:90:74:04:7a:cc:17:a5:a2:70:2d:46:ac:e7:88:10:e2:70:bc:f9:bf:1d:94:a8:94:a1:bf:70:64:d7:e8:48:4c:fb:51:60:59:aa:3a:e8:d2:1c:b1:48:7d:be:f3:68:87:d6:c9:cd:5fExponent: 65537 (0x10001)
[root@master pki]# kubectl get pods Error from server (Forbidden): pods is forbidden: User “scxiang” cannot list resource “pods” in API group “” in the namespace “default” 自定义kubeconfig配置文件 [root@master ~]# kubectl config set-cluster my-cluster —kubeconfig=/tmp/test.conf —server=”https://10.0.0.10:6443“ —certificate-authority=/etc/kubernetes/pki/ca.crt —embed-certs=true Cluster “my-cluster” set. [root@master ~]# kubectl config view —kubeconfig=/tmp/test.conf apiVersion: v1 clusters:82:30:96:46:f4:52:7e:9d:28:23:cb:53:f3:ff:d7:d5:62:e5:db:f0:cb:14:02:c9:e0:d9:fa:3c:e3:cb:b9:5a:53:da:9a:41:2e:75:96:3d:e2:5f:1d:e9:ad:f9:0c:0d:71:4a:55:c8:cd:bd:82:95:eb:8a:8d:24:f6:dd:d4:4a:26:3e:fe:99:43:78:9f:69:30:ca:ed:5b:89:e0:9d:c4:bd:dc:72:0d:33:3f:7a:31:20:4a:20:de:8f:74:46:ce:c5:44:6b:e5:ae:6c:e7:99:39:81:2b:8d:08:5b:29:49:8f:28:e2:a3:d2:30:f2:53:3d:8b:3e:fb:37:b5:1b:2b:cb:50:6d:d0:9f:7c:64:a4:da:11:e0:e2:77:64:8c:8c:13:d2:54:0b:ea:16:d2:dc:ad:0e:65:bc:74:85:31:a0:0c:7f:52:f4:bf:b5:f7:2e:69:80:76:18:99:aa:e5:e5:60:cb:ee:ab:5c:f5:33:d0:8e:9c:ea:63:d2:6a:79:6a:94:3f:b8:d4:89:85:71:3b:a3:08:5c:20:4c:a4:7e:41:bb:c7:97:68:d9:bf:15:cd:f7:b9:7c:63:4d:b5:26:e1:c8:db:ee:3d:d6:b2:8e:fe:0c:e2:5e:8c:7b:0c:54:82:25:7c:a6:28:83:f5:e3:21:95:67:52:71:cb:b5:a8:e8
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://10.0.0.10:6443
name: my-cluster
contexts: null
current-context: “”
kind: Config
preferences: {}
users: null
```
2.2 添加账号到config文件
```yaml [root@master pki]# kubectl config set-credentials scxiang —client-key=scxiang.key —client-certificate=scxiang.crt —embed-certs=true User “scxiang” set. [root@master pki]# kubectl config view apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://10.0.0.10:6443 name: kubernetes contexts:
- context: cluster: kubernetes user: kubernetes-admin name: kubernetes-admin@kubernetes current-context: kubernetes-admin@kubernetes kind: Config preferences: {} users:
- name: kubernetes-admin user: client-certificate-data: REDACTED client-key-data: REDACTED
- name: scxiang
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
```
2.3 设置set-context并切换至指定用户
```yaml [root@master pki]# kubectl config set-context scxiang@kubernetes —cluster=kubernetes —user=scxiang Context “scxiang@kubernetes” created. [root@master pki]# kubectl config view apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://10.0.0.10:6443 name: kubernetes contexts:
- context: cluster: kubernetes user: kubernetes-admin name: kubernetes-admin@kubernetes
- context: cluster: kubernetes user: scxiang name: scxiang@kubernetes current-context: kubernetes-admin@kubernetes kind: Config preferences: {} users:
- name: kubernetes-admin user: client-certificate-data: REDACTED client-key-data: REDACTED
- name: scxiang user: client-certificate-data: REDACTED client-key-data: REDACTED 切换到指定账号账号
<a name="vAGJO"></a>## 2.4 切换账号```yaml[root@master pki]# kubectl config use-context scxiang@kubernetesSwitched to context "scxiang@kubernetes".
3 授权
格式Object URl:
/apis/
授权插件:Node,ABAC ,RBAC ,Webhook
RBAC:Role based Access Controle
角色(Role)
许可(peimission)
role
- operations
- objects
rolebinding
- user account OR service account
- role
3.1 基于名称空间内的角色创建与绑定
创建角色role[root@master role]# kubectl create role pods-reader --verb=get,list,watch --resource=pods --dry-run -o yaml >role-demo.yaml[root@master role]# cat role-demo.yamlapiVersion: rbac.authorization.k8s.io/v1kind: Rolemetadata:creationTimestamp: nullname: pods-readerrules:- apiGroups:- ""resources:- podsverbs:- get- list- watch[root@master role]# kubectl apply -f role-demo.yamlrole.rbac.authorization.k8s.io/pods-reader created查看角色的描述[root@master role]# kubectl describe role pods-readerName: pods-readerLabels: <none>Annotations: <none>PolicyRule:Resources Non-Resource URLs Resource Names Verbs--------- ----------------- -------------- -----pods [] [] [get list watch]将角色与用户进行绑定[root@master ~]# kubectl create rolebinding scxiang-to-pods-reader --role=pods-reader --user=scxiangrolebinding.rbac.authorization.k8s.io/scxiang-to-pods-reader created[root@master ~]# kubectl describe rolebinding scxiang-to-pods-readerName: scxiang-to-pods-readerLabels: <none>Annotations: <none>Role:Kind: RoleName: pods-readerSubjects:Kind Name Namespace---- ---- ---------User scxiang切换到scxiang账户[root@master ~]# kubectl config use-context scxiang@kubernetesSwitched to context "scxiang@kubernetes".基于scxiang账户查看默认名称空间的pods信息[root@master ~]# kubectl get podsNAME READY STATUS RESTARTS AGEsa-demo-d877998d-mr8lk 1/1 Running 0 27hweb01-bbc5667-8h944 1/1 Running 0 28hweb01-bbc5667-l2dvl 1/1 Running 0 28hweb01-bbc5667-rk9ng 1/1 Running 0 28hscxiang账户无权限查看kube-system名称空间的信息[root@master ~]# kubectl get pods -n kube-systemError from server (Forbidden): pods is forbidden: User "scxiang" cannot list resource "pods" in API group "" in the namespace "kube-system"
3.2 创建集群角色并绑定
[root@master ~]# kubectl create clusterrole cluster-reader --verb=get,list,watch --resource=pods -o yaml --dry-run >/root/role/clusterrole-demo.yamlapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:name: cluster-readerrules:- apiGroups:- ""resources:- podsverbs:- get- list- watch[root@master ~]# kubectl apply -f role/clusterrole-demo.yamlclusterrole.rbac.authorization.k8s.io/cluster-reader created[root@master ~]# kubectl describe clusterrole cluster-readerName: cluster-readerLabels: <none>Annotations: <none>PolicyRule:Resources Non-Resource URLs Resource Names Verbs--------- ----------------- -------------- -----pods [] [] [get list watch][root@master ~]# kubectl create clusterrolebinding scxiang-to-cluster-reader --clusterrole=cluster-reader --user=scxiang -o yaml --dry-run >/root/role/clusterrolebinding-demo.yaml[root@master ~]# kubectl apply -f /root/role/clusterrolebinding-demo.yamlclusterrolebinding.rbac.authorization.k8s.io/scxiang-to-cluster-reader created[root@master ~]# kubectl describe clusterrolebinding scxiang-to-cluster-readerName: scxiang-to-cluster-readerLabels: <none>Annotations: <none>Role:Kind: ClusterRoleName: cluster-readerSubjects:Kind Name Namespace---- ---- ---------User scxiang此clusterrolebinding可以查看集群的所有名称空间的POD,但是无法做其它操作[root@master ~]# kubectl config use-context scxiang@kubernetesSwitched to context "scxiang@kubernetes".[root@master ~]# kubectl get podNAME READY STATUS RESTARTS AGEsa-demo-d877998d-mr8lk 1/1 Running 0 29hweb01-bbc5667-8h944 1/1 Running 0 29hweb01-bbc5667-l2dvl 1/1 Running 0 29hweb01-bbc5667-rk9ng 1/1 Running 0 29h[root@master ~]# kubectl get pod -n kube-systemNAME READY STATUS RESTARTS AGEcalico-kube-controllers-97769f7c7-xf2f7 1/1 Running 5 58dcalico-node-bw7sd 1/1 Running 14 62dcalico-node-ndv7l 1/1 Running 14 62dcalico-node-vbxsz 1/1 Running 14 62dcoredns-7f89b7bc75-7lslf 1/1 Running 6 58dcoredns-7f89b7bc75-msr7c 1/1 Running 6 58detcd-master 1/1 Running 6 62dkube-apiserver-master 1/1 Running 7 58dkube-controller-manager-master 0/1 CreateContainerError 11 29hkube-proxy-jpqkp 1/1 Running 5 58dkube-proxy-nrtxj 1/1 Running 5 58dkube-proxy-pnbdz 1/1 Running 6 58dkube-scheduler-master 0/1 CreateContainerError 11 29hmetrics-server-84f9866fdf-wh47t 1/1 Running 5 58d[root@master ~]# kubectl get deployError from server (Forbidden): deployments.apps is forbidden: User "scxiang" cannot list resource "deployments" in API group "apps" in the namespace "default"
4 Dashboard的部署与认证、分级授权
4.1 部署
上github可以查看其部署方式:
[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml
修改svc的类型为NodePort
[root@master ~]# kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-systemservice/kubernetes-dashboard patched[root@master-01 ~]# kubectl get svc -n kubernetes-dashboardNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEdashboard-metrics-scraper ClusterIP 10.111.129.21 <none> 8000/TCP 5h22mkubernetes-dashboard NodePort 10.111.186.10 <none> 443:31477/TCP 5h22m
4.2 Token方式登录
创建一个sa账户,并绑定集群角色
[root@master ~]# kubectl create serviceaccount dashboard-admin -n kube-system[root@master ~]# kubectl get sa -n kube-system | grep dashboard-admindashboard-admin 1 30m[root@master ~]# kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin[root@master ~]# kubectl get clusterrolebinding -n -kube-system | grep dashboard-cluster-admindashboard-cluster-admin ClusterRole/cluster-admin 27m查看其token[root@master ~]# kubectl get secret -n kube-system | grep dashboard-admindashboard-admin-token-nhp2t kubernetes.io/service-account-token 3 32m[root@master ~]# kubectl describe secret dashboard-admin-token-nhp2t -n kube-systemName: dashboard-admin-token-nhp2tNamespace: kube-systemLabels: <none>Annotations: kubernetes.io/service-account.name: dashboard-adminkubernetes.io/service-account.uid: f9fe0cef-4717-49a8-a203-17198a8f38d9Type: kubernetes.io/service-account-tokenData====ca.crt: 1066 bytesnamespace: 11 bytestoken: eyJhbGciOiJSUzI1NiIsImtpZCI6InItd0k3Vk10SXVKTHRiQms2TVlYUFZNV0s4WGVZRnBVMXF1Z2VBWFNvQU0ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tbmhwMnQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZjlmZTBjZWYtNDcxNy00OWE4LWEyMDMtMTcxOThhOGYzOGQ5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.WnwviGbRPKCbj0xTrFrE0psy3sPpy72jW0GIrYNQMVp6MwYPLST-rxrKqA4GQfYt7BsK-Nu_TVHANPJYo9MsKEuhm_P57aIDpiGJB-n7fv0BzjJFS0_pHi6HMoGco4qGbsqCBM6zMQ-v521yc0X26GPNl3HT6wRX10VD-EODpyvAiJd5d5j7WGboI22S1_CB4cDQgNHUdtrfQ4wQ7Sr_aWX-E9Id7S0fNeaFRRVzo28wadZ8wuHsCIErMUYSSj1TOelVSWHNUhRPipT6MaNdx8RwS3GZ0VHWz3oTFNWSyU8vVgeR-hN7EjOZsmL9hXtuHCipFTCTPbF6P3nJYNK4Rg
4.3 kubeconfig认证方式登录
[root@master-01 ~]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.crt --server="https://10.0.0.10:6443" --embed-certs=true --kubeconfig=/root/dashboard-admin.confCluster "kubernetes" set.[root@master-01 ~]# kubectl config view --kubeconfig=/root/dashboard-admin.confapiVersion: v1clusters:- cluster:certificate-authority-data: DATA+OMITTEDserver: https://10.0.0.10:6443name: kubernetescontexts: nullcurrent-context: ""kind: Configpreferences: {}users: null添加用户[root@master-01 ~]# dashboard_admin_token=`kubectl get secret dashboard-admin-token-hlxj4 -o jsonpath={.data.token} | base64 -d`[root@master-01 ~]# kubectl config set-credentials dashboard-admin --token=$dashboard_admin_token --kubeconfig=/root/dashboard-admin.confUser "cluster-dashboard-admin" set.[root@master-01 ~]# kubectl config view --kubeconfig=/root/dashboard-admin.confapiVersion: v1clusters:- cluster:certificate-authority-data: DATA+OMITTEDserver: https://10.0.0.10:6443name: kubernetescontexts: nullcurrent-context: ""kind: Configpreferences: {}users:- name: dashboard-adminuser:token: REDACTED[root@master-01 ~]# kubectl config set-context dashboard-admin@kubernetes --cluster=kubernetes --user=dashboard-admin --kubeconfig=/root/dashboard-admin.confContext "dashboard-admin@kubernetes" created.[root@master-01 ~]# kubectl config view --kubeconfig=/root/dashboard-admin.confapiVersion: v1clusters:- cluster:certificate-authority-data: DATA+OMITTEDserver: https://10.0.0.10:6443name: kubernetescontexts:- context:cluster: kubernetesuser: dashboard-adminname: dashboard-admin@kubernetescurrent-context: ""kind: Configpreferences: {}users:- name: dashboard-adminuser:token: REDACTED[root@master-01 ~]# kubectl config use-context dashboard-admin@kubernetes --kubeconfig=/root/dashboard-admin.conf[root@master-01 ~]# kubectl config view --kubeconfig=/root/dashboard-admin.confapiVersion: v1clusters:- cluster:certificate-authority-data: DATA+OMITTEDserver: https://10.0.0.10:6443name: kubernetescontexts:- context:cluster: kubernetesuser: dashboard-adminname: dashboard-admin@kubernetescurrent-context: dashboard-admin@kuberneteskind: Configpreferences: {}users:- name: dashboard-adminuser:token: REDACTED
kubeconfig方式登录
