1. Namespaces资源配额
**
# 创建namespaces
kubectl create namespace compute
# 创建计算资源配额

  1. cat << EOF > compute-resources.yaml
  2. apiVersion: v1
  3. kind: ResourceQuota
  4. metadata:
  5. name: compute
  6. namespace: compute
  7. spec:
  8. hard:
  9. cpu: 5
  10. memory: 2Gi
  11. pods: "10"
  12. services: "2"
  13. requests.cpu: 500m
  14. limits.memory: 1Gi
  15. persistentvolumeclaims: "2"
  16. services.nodeports: "2"
  17. configmaps: "2"
  18. secrets: "2"
  19. EOF

kubectl create -f compute-resources.yaml

查看配额的最新信息
kubectl -n compute describe resourcequotas compute

2. 应用资源限制
2.1 内存限制

  1. cat << EOF > stress-mem.yaml
  2. apiVersion: v1
  3. kind: Pod
  4. metadata:
  5. name: memory-demo
  6. spec:
  7. containers:
  8. - name: memory-demo-2-ctr
  9. image: polinux/stress
  10. imagePullPolicy: IfNotPresent
  11. resources:
  12. requests:
  13. memory: "50Mi"
  14. limits:
  15. memory: "100Mi"
  16. command: ["stress"]
  17. args: ["--vm", "1", "--vm-bytes", "250M", "--vm-hang", "1"]
  18. EOF

kubectl apply -f stress-mem.yaml
### 查看会发现memory-demo-2这个pod状态变为OOMKilled,因为它是内存不足所以显示Container被杀死
[root@master01 ~]# kubectl get pod -n mem-example
NAME READY STATUS RESTARTS AGE
memory-demo 1/1 Running 0 7m13s
memory-demo-2 0/1 OOMKilled 2 20s

2.2 CPU限制

  1. cat << EOF > stress-cpu.yaml
  2. apiVersion: v1
  3. kind: Pod
  4. metadata:
  5. name: cpu-demo
  6. spec:
  7. containers:
  8. - name: cpu-demo-ctr
  9. image: vish/stress
  10. resources:
  11. limits:
  12. cpu: "1"
  13. requests:
  14. cpu: "0.5"
  15. args:
  16. - -cpus
  17. - "2"
  18. EOF

kubectl create -f stress-cpu.yaml
# 查看cpu使用
root@master01:~/yaml# kubectl top pod cpu-demo
NAME CPU(cores) MEMORY(bytes)
cpu-demo 1001m 0Mi

3. Limit Range
LimitRange是用来给namespace内的pod设置一个默认的request和limit值
先创建namespace
kubectl create namespace default-cpu-example
使用以下yaml文件创建limitRange,指定一个默认的cpu request和cpu limit
创建个名为cpu-limit-range.yaml的文件,填入以下内容

  1. apiVersion: v1
  2. kind: LimitRange
  3. metadata:
  4. name: cpu-limit-range
  5. spec:
  6. limits:
  7. - default:
  8. cpu: 1
  9. defaultRequest:
  10. cpu: 0.5
  11. type: Container

执行
应用到default-cpu-example这个namespace
kubectl apply -f 1.yaml —namespace=default-cpu-example
使用以下yaml文件创建pod
创建名为default-cpu-demo.yaml的yaml文件,输入以下内容

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: default-cpu-demo
  5. spec:
  6. containers:
  7. - name: default-cpu-demo-ctr
  8. image: nginx

部署pod
kubectl apply -f default-cpu-demo.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo —output=yaml —namespace=default-cpu-example
可以看见Resource这里自动配置了limit和request,这个就是继承LimitRange

  1. resources:
  2. limits:
  3. cpu: "1"
  4. requests:
  5. cpu: 500m

场景二:只配置pod Resource的limit不配置request 使用以下yaml文件创建pod
default-cpu-demo-2.yaml

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: default-cpu-demo-2
  5. spec:
  6. containers:
  7. - name: default-cpu-demo-2-ctr
  8. image: nginx
  9. resources:
  10. limits:
  11. cpu: "1"

部署pod
kubectl apply -f default-cpu-demo-2.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
需要注意的是这里request值没有继承LimitRange配置的值0.5而是直接根limit相等。

  1. resources:
  2. limits:
  3. cpu: "1"
  4. requests:
  5. cpu: "1"

场景三:指定容器请求值,不指定容器限额值 使用以下yaml文件创建pod test.yaml

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: default-cpu-demo-3
  5. spec:
  6. containers:
  7. - name: default-cpu-demo-3-ctr
  8. image: nginx
  9. resources:
  10. requests:
  11. cpu: "0.75"

部署pod
kubectl apply -f test.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
需要注意的是这里request值没有继承LimitRange配置的值,而是直接是我们在pod中配置的值,limit继承的是LimitRange的值

  1. resources:
  2. limits:
  3. cpu: "1"
  4. requests:
  5. cpu: 750m

总结:

  • 如果没有在pod内设置request和limit默认就继承在namespace中配置的LimitRange。
  • 如果在pod只配置了Resource的limit没配置request,这时request值不会继承LimitRange配置的值而是直接根pod中配置limit相等。
  • 如果在pod中配置了request没有配置limit,这时request值以pod中配置的为准,limit值以namespace中的LimitRange为主。

4. Network Policies
相同namespace的NetworkPolicy的隔离性
# 创建一个namespace
kubectl create namespace policy-demo
# 创建pod
kubectl run —namespace=policy-demo nginx —replicas=1 —image=nginx —labels=app=nginx

创建service
kubectl expose —namespace=policy-demo deployment nginx —port=80

测试访问正常
kubectl -n policy-demo run test —image=busybox:1.28 —labels=app=busybox —command — sleep 3600000 kubectl -n policy-demo exec -it test-6778df478f-gnkp8 — wget -q nginx -O -
结果:访问正常

创建NetworkPolicy规则

  1. cat <<EOF> default-deny.yaml
  2. kind: NetworkPolicy
  3. apiVersion: networking.k8s.io/v1
  4. metadata:
  5. name: default-deny
  6. namespace: policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels: {}
  10. EOF

kubectl create -f default-deny.yaml
此规则表示拒绝pod连接policy-demo namespace下的pod
测试
kubectl -n policy-demo exec -it test-6778df478f-gnkp8 — wget -q nginx -O -
wget: Download time out
可以看见被拒绝访问了
#添加允许规则

  1. cat <<EOF> access-nginx.yaml
  2. kind: NetworkPolicy
  3. apiVersion: networking.k8s.io/v1
  4. metadata:
  5. name: access-nginx
  6. namespace: policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels:
  10. app: nginx
  11. ingress:
  12. - from:
  13. - podSelector:
  14. matchLabels:
  15. app: busybox
  16. EOF

kubectl create -f access-nginx.yaml
这条规则意思,允许,label为 app:busyboxy的pod访问policy-demo namespace下label为app:nginx的pod
# 在次测试,可以访问成功
kubectl -n policy-demo exec -it test-6bdbf87b87-cx5r7 — wget -q nginx -O -\
结果:访问成功

创建一个不是app: busybox的pod去测试访问
kubectl -n policy-demo run demo —image=busybox:1.28 —labels=app=test —command — sleep 3600000 kubectl -n policy-demo exec -it demo-6778df478f-gnkp8 — wget -q nginx -O -
结果:wget:Download time out
结论:同namespace下可以使用Policy在限制pod与pod之间的访问

清空环境
kubectl delete namespace policy-demo

不同namespace pod的隔离性
创建两个namespace policy-demo、policy-demo2,然后在policy-demo里面创建nginx-pod和对应的service和busybox,在policy-demo2里面创建busybox,两个namespace的busybox去访问policy-demo里面的nginx

kubectl create namespace policy-demo kubectl create namespace policy-demo2
kubectl run —namespace=policy-demo nginx —replicas=1 —image=nginx —labels=app=nginx
kubectl expose —namespace=policy-demo deployment nginx —port=80
kubectl -n policy-demo2 run test —image=busybox:1.28 —labels=app=busybox —command — sleep 3600000

测试访问正常,还没设置NetworkPolicy时分别从policy-demo和policy-demo2两个namespace去busybox去访问nginx,访问成功。
需要注意的是 policy-demo2去访问要接上namespace名
kubectl -n policy-demo2 exec -it test-6778df478f-gnkp8 — wget -q nginx.policy-demo -O -

配置NetworkPolicy

  1. cat <<EOF> default-deny.yaml
  2. kind: NetworkPolicy
  3. apiVersion: networking.k8s.io/v1
  4. metadata:
  5. name: default-deny
  6. namespace: policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels: {}
  10. EOF

kubectl create -f default-deny.yaml
配置拒绝所有Policy,此时policy-demo2 namespace的busybox都不能访问了

配置允许policy-demo2下的app:busybox标签的POD访问policy-demo namespace下的app:nginx服务

  1. cat <<EOF> namespace-access.yaml
  2. kind: NetworkPolicy
  3. apiVersion: networking.k8s.io/v1
  4. metadata:
  5. name: namespace-access
  6. namespace: policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels:
  10. app: nginx
  11. ingress:
  12. - from:
  13. - namespaceSelector:
  14. matchLabels:
  15. project: policy-demo2
  16. - podSelector:
  17. matchLabels:
  18. app: busybox
  19. EOF

kubectl create -f namespace-access.yaml
此时policy-demo2下的run:access标签的POD访问policy-demo namespace下的run:nginx服务,但其他标签不可以。
运行个run:access2标签的busybox去访问policy-demo namespace下的run:nginx服务
kubectl -n policy-demo2 exec -it test-6bdbf87b87-6vxkc — wget -q nginx.policy-demo -O -
结果:wget:Download time out

注意:

  1. - namespaceSelector:
  2. matchLabels:
  3. project: policy-demo2

是namespaces的标签,想要夸namespaces访问通过需要给namespaces打标签并添加到matchLabels里。
kubectl label namespaces policy-demo2 project=policy-demo2
# 在测试
kubectl -n policy-demo2 exec -it test-6bdbf87b87-6vxkc — wget -q nginx.policy-demo -O -

  1. <!DOCTYPE html>
  2. <html>
  3. <head>
  4. <title>Welcome to nginx!</title>
  5. <style>
  6. body {
  7. width: 35em;
  8. margin: 0 auto;
  9. font-family: Tahoma, Verdana, Arial, sans-serif;
  10. }
  11. </style>
  12. </head>
  13. <body>
  14. <h1>Welcome to nginx!</h1>
  15. <p>If you see this page, the nginx web server is successfully installed and
  16. working. Further configuration is required.</p>
  17. <p>For online documentation and support please refer to
  18. <a href="http://nginx.org/">nginx.org</a>.<br/>
  19. Commercial support is available at
  20. <a href="http://nginx.com/">nginx.com</a>.</p>
  21. <p><em>Thank you for using nginx.</em></p>
  22. </body>
  23. </html>

基于角色的访问权限配置
创建用户
# 使用root账户拷贝ca证书到指定目录下
mkdir /home/damon/certificate
cd /etc/kubernetes/ssl
cp kube-ca-key.pem kube-ca.pem /home/damon/certificate/
chown -R damon:damon /home/damon/

生成user密钥文件
(umask 077; openssl genrsa -out damon.key 2048)

创建证书签署请求
openssl req -new -key damon.key -out damon.csr -subj “/CN=damon/O=local”
注意:
CN=<填写username>
O=与集群名称一致,kubectl config get-contexts 查看

基于ca签署证书
openssl x509 -req -in damon.csr -CA kube-ca.pem -CAkey kube-ca-key.pem -CAcreateserial -out damon.crt -days 3650

验证证书
openssl x509 -noout -text -in damon.crt

配置集群信息
kubectl config set-cluster local \
—embed-certs=true —certificate-authority=kube-ca.pem \
—server=https://172.17.224.182:6443

配置客户端证书和密钥
kubectl config set-credentials damon \
—embed-certs=true \
—client-certificate=damon.crt \
—client-key=damon.key

设置上下文
kubectl config set-context damon@local —cluster=local —user=damon

查看上下文
kubectl config get-contexts

切换上下文
kubectl config use-context damon@local

Role和RoleBinding创建
# 创建角色 定义这个角色只能对default这个namespace 执行get、watch、list权限 定义角色

  1. cat << EOF > role.yaml
  2. kind: Role
  3. apiVersion: rbac.authorization.k8s.io/v1beta1
  4. metadata:
  5. namespace: default
  6. name: pod-reader
  7. rules:
  8. - apiGroups: [""] # 空字符串""表明使用core API group
  9. resources: ["pods"]
  10. verbs: ["get", "watch", "list"]
  11. ---
  12. kind: RoleBinding
  13. apiVersion: rbac.authorization.k8s.io/v1beta1
  14. metadata:
  15. name: read-pods
  16. namespace: default
  17. subjects:
  18. - kind: User
  19. name: damon
  20. apiGroup: rbac.authorization.k8s.io
  21. roleRef:
  22. kind: Role
  23. name: pod-reader
  24. apiGroup: rbac.authorization.k8s.io
  25. EOF

使用admin创建
kubectl apply -f role.yaml

检查
kubectl get role
kubectl get rolebinding

kubectl get pod
kubectl get nodes
get pod可以但get node不行,因为我们刚刚配置role只有pod权限

删除pod看看
kubectl delete pod/http-app-844765cb6c-nfp7l Error from server (Forbidden): pods “http-app-844765cb6c-nfp7l” is forbidden: User “test-cka” cannot delete pods in the namespace “default”

你会发现也删不掉,因为我们role里面配置的权限是watch和list和get

kubectl get pod -n kube-system No resources found. Error from server (Forbidden): pods is forbidden: User “test-cka”” cannot list pods in the namespace “kube-system” [root@master pki]#

可以看见damon这个用户只能访问default这个namespace的pod资源,其他的namespace都访问不了,同样namespace的其他资源也访问不了

ClusterRole和ClusterRoleBinding创建

  1. cat << EOF > cluster_role.yaml
  2. kind: ClusterRole
  3. apiVersion: rbac.authorization.k8s.io/v1
  4. metadata:
  5. name: secret-reader
  6. rules:
  7. - apiGroups: [""]
  8. resources: ["nodes"]
  9. verbs: ["get", "watch", "list"]
  10. ---
  11. kind: ClusterRoleBinding
  12. apiVersion: rbac.authorization.k8s.io/v1
  13. metadata:
  14. name: read-secrets-global
  15. subjects:
  16. - kind: User
  17. name: damon
  18. apiGroup: rbac.authorization.k8s.io
  19. roleRef:
  20. kind: ClusterRole
  21. name: secret-reader
  22. apiGroup: rbac.authorization.k8s.io
  23. EOF

**
# 创建
kubectl apply -f cluster_role.yaml

15-集群安全配置与管理 - 图1

**