- 查看配额的最新信息
kubectl -n compute describe resourcequotas compute - 创建service
kubectl expose —namespace=policy-demo deployment nginx —port=80 - 测试访问正常
kubectl -n policy-demo run test —image=busybox:1.28 —labels=app=busybox —command — sleep 3600000 kubectl -n policy-demo exec -it test-6778df478f-gnkp8 — wget -q nginx -O -
结果:访问正常 - 创建NetworkPolicy规则
- 创建一个不是app: busybox的pod去测试访问
kubectl -n policy-demo run demo —image=busybox:1.28 —labels=app=test —command — sleep 3600000 kubectl -n policy-demo exec -it demo-6778df478f-gnkp8 — wget -q nginx -O -
结果:wget:Download time out
结论:同namespace下可以使用Policy在限制pod与pod之间的访问 - 测试访问正常,还没设置NetworkPolicy时分别从policy-demo和policy-demo2两个namespace去busybox去访问nginx,访问成功。
需要注意的是 policy-demo2去访问要接上namespace名
kubectl -n policy-demo2 exec -it test-6778df478f-gnkp8 — wget -q nginx.policy-demo -O - - 配置NetworkPolicy
- 配置允许policy-demo2下的app:busybox标签的POD访问policy-demo namespace下的app:nginx服务
- 生成user密钥文件
(umask 077; openssl genrsa -out damon.key 2048) - 创建证书签署请求
openssl req -new -key damon.key -out damon.csr -subj “/CN=damon/O=local”
注意:
CN=<填写username>
O=与集群名称一致,kubectl config get-contexts 查看 - 基于ca签署证书
openssl x509 -req -in damon.csr -CA kube-ca.pem -CAkey kube-ca-key.pem -CAcreateserial -out damon.crt -days 3650 - 验证证书
openssl x509 -noout -text -in damon.crt - https://172.17.224.182:6443">配置集群信息
kubectl config set-cluster local \
—embed-certs=true —certificate-authority=kube-ca.pem \
—server=https://172.17.224.182:6443 - 配置客户端证书和密钥
kubectl config set-credentials damon \
—embed-certs=true \
—client-certificate=damon.crt \
—client-key=damon.key - 设置上下文
kubectl config set-context damon@local —cluster=local —user=damon - 查看上下文
kubectl config get-contexts - 切换上下文
kubectl config use-context damon@local - 使用admin创建
kubectl apply -f role.yaml - 检查
kubectl get role
kubectl get rolebinding
1. Namespaces资源配额
**
# 创建namespaces
kubectl create namespace compute
# 创建计算资源配额
cat << EOF > compute-resources.yamlapiVersion: v1kind: ResourceQuotametadata:name: computenamespace: computespec:hard:cpu: 5memory: 2Gipods: "10"services: "2"requests.cpu: 500mlimits.memory: 1Gipersistentvolumeclaims: "2"services.nodeports: "2"configmaps: "2"secrets: "2"EOF
kubectl create -f compute-resources.yaml
查看配额的最新信息
kubectl -n compute describe resourcequotas compute
2. 应用资源限制
2.1 内存限制
cat << EOF > stress-mem.yamlapiVersion: v1kind: Podmetadata:name: memory-demospec:containers:- name: memory-demo-2-ctrimage: polinux/stressimagePullPolicy: IfNotPresentresources:requests:memory: "50Mi"limits:memory: "100Mi"command: ["stress"]args: ["--vm", "1", "--vm-bytes", "250M", "--vm-hang", "1"]EOF
kubectl apply -f stress-mem.yaml
### 查看会发现memory-demo-2这个pod状态变为OOMKilled,因为它是内存不足所以显示Container被杀死
[root@master01 ~]# kubectl get pod -n mem-example
NAME READY STATUS RESTARTS AGE
memory-demo 1/1 Running 0 7m13s
memory-demo-2 0/1 OOMKilled 2 20s
2.2 CPU限制
cat << EOF > stress-cpu.yamlapiVersion: v1kind: Podmetadata:name: cpu-demospec:containers:- name: cpu-demo-ctrimage: vish/stressresources:limits:cpu: "1"requests:cpu: "0.5"args:- -cpus- "2"EOF
kubectl create -f stress-cpu.yaml
# 查看cpu使用
root@master01:~/yaml# kubectl top pod cpu-demo
NAME CPU(cores) MEMORY(bytes)
cpu-demo 1001m 0Mi
3. Limit Range
LimitRange是用来给namespace内的pod设置一个默认的request和limit值
先创建namespace
kubectl create namespace default-cpu-example
使用以下yaml文件创建limitRange,指定一个默认的cpu request和cpu limit
创建个名为cpu-limit-range.yaml的文件,填入以下内容
apiVersion: v1kind: LimitRangemetadata:name: cpu-limit-rangespec:limits:- default:cpu: 1defaultRequest:cpu: 0.5type: Container
执行
应用到default-cpu-example这个namespace
kubectl apply -f 1.yaml —namespace=default-cpu-example
使用以下yaml文件创建pod
创建名为default-cpu-demo.yaml的yaml文件,输入以下内容
apiVersion: v1kind: Podmetadata:name: default-cpu-demospec:containers:- name: default-cpu-demo-ctrimage: nginx
部署pod
kubectl apply -f default-cpu-demo.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo —output=yaml —namespace=default-cpu-example
可以看见Resource这里自动配置了limit和request,这个就是继承LimitRange
resources:limits:cpu: "1"requests:cpu: 500m
场景二:只配置pod Resource的limit不配置request 使用以下yaml文件创建pod
default-cpu-demo-2.yaml
apiVersion: v1kind: Podmetadata:name: default-cpu-demo-2spec:containers:- name: default-cpu-demo-2-ctrimage: nginxresources:limits:cpu: "1"
部署pod
kubectl apply -f default-cpu-demo-2.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
需要注意的是这里request值没有继承LimitRange配置的值0.5而是直接根limit相等。
resources:limits:cpu: "1"requests:cpu: "1"
场景三:指定容器请求值,不指定容器限额值 使用以下yaml文件创建pod test.yaml
apiVersion: v1kind: Podmetadata:name: default-cpu-demo-3spec:containers:- name: default-cpu-demo-3-ctrimage: nginxresources:requests:cpu: "0.75"
部署pod
kubectl apply -f test.yaml —namespace=default-cpu-example
查看pod yaml配置
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
kubectl get pod default-cpu-demo-2 —output=yaml —namespace=default-cpu-example
需要注意的是这里request值没有继承LimitRange配置的值,而是直接是我们在pod中配置的值,limit继承的是LimitRange的值
resources:limits:cpu: "1"requests:cpu: 750m
总结:
- 如果没有在pod内设置request和limit默认就继承在namespace中配置的LimitRange。
- 如果在pod只配置了Resource的limit没配置request,这时request值不会继承LimitRange配置的值而是直接根pod中配置limit相等。
- 如果在pod中配置了request没有配置limit,这时request值以pod中配置的为准,limit值以namespace中的LimitRange为主。
4. Network Policies
相同namespace的NetworkPolicy的隔离性
# 创建一个namespace
kubectl create namespace policy-demo
# 创建pod
kubectl run —namespace=policy-demo nginx —replicas=1 —image=nginx —labels=app=nginx
创建service
kubectl expose —namespace=policy-demo deployment nginx —port=80
测试访问正常
kubectl -n policy-demo run test —image=busybox:1.28 —labels=app=busybox —command — sleep 3600000 kubectl -n policy-demo exec -it test-6778df478f-gnkp8 — wget -q nginx -O -
结果:访问正常
创建NetworkPolicy规则
cat <<EOF> default-deny.yamlkind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata:name: default-denynamespace: policy-demospec:podSelector:matchLabels: {}EOF
kubectl create -f default-deny.yaml
此规则表示拒绝pod连接policy-demo namespace下的pod
测试
kubectl -n policy-demo exec -it test-6778df478f-gnkp8 — wget -q nginx -O -
wget: Download time out
可以看见被拒绝访问了
#添加允许规则
cat <<EOF> access-nginx.yamlkind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata:name: access-nginxnamespace: policy-demospec:podSelector:matchLabels:app: nginxingress:- from:- podSelector:matchLabels:app: busyboxEOF
kubectl create -f access-nginx.yaml
这条规则意思,允许,label为 app:busyboxy的pod访问policy-demo namespace下label为app:nginx的pod
# 在次测试,可以访问成功
kubectl -n policy-demo exec -it test-6bdbf87b87-cx5r7 — wget -q nginx -O -\
结果:访问成功
创建一个不是app: busybox的pod去测试访问
kubectl -n policy-demo run demo —image=busybox:1.28 —labels=app=test —command — sleep 3600000 kubectl -n policy-demo exec -it demo-6778df478f-gnkp8 — wget -q nginx -O -
结果:wget:Download time out
结论:同namespace下可以使用Policy在限制pod与pod之间的访问
清空环境
kubectl delete namespace policy-demo
不同namespace pod的隔离性
创建两个namespace policy-demo、policy-demo2,然后在policy-demo里面创建nginx-pod和对应的service和busybox,在policy-demo2里面创建busybox,两个namespace的busybox去访问policy-demo里面的nginx
kubectl create namespace policy-demo kubectl create namespace policy-demo2
kubectl run —namespace=policy-demo nginx —replicas=1 —image=nginx —labels=app=nginx
kubectl expose —namespace=policy-demo deployment nginx —port=80
kubectl -n policy-demo2 run test —image=busybox:1.28 —labels=app=busybox —command — sleep 3600000
测试访问正常,还没设置NetworkPolicy时分别从policy-demo和policy-demo2两个namespace去busybox去访问nginx,访问成功。
需要注意的是 policy-demo2去访问要接上namespace名
kubectl -n policy-demo2 exec -it test-6778df478f-gnkp8 — wget -q nginx.policy-demo -O -
配置NetworkPolicy
cat <<EOF> default-deny.yamlkind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata:name: default-denynamespace: policy-demospec:podSelector:matchLabels: {}EOF
kubectl create -f default-deny.yaml
配置拒绝所有Policy,此时policy-demo2 namespace的busybox都不能访问了
配置允许policy-demo2下的app:busybox标签的POD访问policy-demo namespace下的app:nginx服务
cat <<EOF> namespace-access.yamlkind: NetworkPolicyapiVersion: networking.k8s.io/v1metadata:name: namespace-accessnamespace: policy-demospec:podSelector:matchLabels:app: nginxingress:- from:- namespaceSelector:matchLabels:project: policy-demo2- podSelector:matchLabels:app: busyboxEOF
kubectl create -f namespace-access.yaml
此时policy-demo2下的run:access标签的POD访问policy-demo namespace下的run:nginx服务,但其他标签不可以。
运行个run:access2标签的busybox去访问policy-demo namespace下的run:nginx服务
kubectl -n policy-demo2 exec -it test-6bdbf87b87-6vxkc — wget -q nginx.policy-demo -O -
结果:wget:Download time out
注意:
- namespaceSelector:matchLabels:project: policy-demo2
是namespaces的标签,想要夸namespaces访问通过需要给namespaces打标签并添加到matchLabels里。
kubectl label namespaces policy-demo2 project=policy-demo2
# 在测试
kubectl -n policy-demo2 exec -it test-6bdbf87b87-6vxkc — wget -q nginx.policy-demo -O -
<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;}</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p><p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p></body></html>
基于角色的访问权限配置
创建用户
# 使用root账户拷贝ca证书到指定目录下
mkdir /home/damon/certificate
cd /etc/kubernetes/ssl
cp kube-ca-key.pem kube-ca.pem /home/damon/certificate/
chown -R damon:damon /home/damon/
生成user密钥文件
(umask 077; openssl genrsa -out damon.key 2048)
创建证书签署请求
openssl req -new -key damon.key -out damon.csr -subj “/CN=damon/O=local”
注意:
CN=<填写username>
O=与集群名称一致,kubectl config get-contexts 查看
基于ca签署证书
openssl x509 -req -in damon.csr -CA kube-ca.pem -CAkey kube-ca-key.pem -CAcreateserial -out damon.crt -days 3650
验证证书
openssl x509 -noout -text -in damon.crt
配置集群信息
kubectl config set-cluster local \
—embed-certs=true —certificate-authority=kube-ca.pem \
—server=https://172.17.224.182:6443
配置客户端证书和密钥
kubectl config set-credentials damon \
—embed-certs=true \
—client-certificate=damon.crt \
—client-key=damon.key
设置上下文
kubectl config set-context damon@local —cluster=local —user=damon
查看上下文
kubectl config get-contexts
切换上下文
kubectl config use-context damon@local
Role和RoleBinding创建
# 创建角色 定义这个角色只能对default这个namespace 执行get、watch、list权限 定义角色
cat << EOF > role.yamlkind: RoleapiVersion: rbac.authorization.k8s.io/v1beta1metadata:namespace: defaultname: pod-readerrules:- apiGroups: [""] # 空字符串""表明使用core API groupresources: ["pods"]verbs: ["get", "watch", "list"]---kind: RoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata:name: read-podsnamespace: defaultsubjects:- kind: Username: damonapiGroup: rbac.authorization.k8s.ioroleRef:kind: Rolename: pod-readerapiGroup: rbac.authorization.k8s.ioEOF
使用admin创建
kubectl apply -f role.yaml
检查
kubectl get role
kubectl get rolebinding
kubectl get pod
kubectl get nodes
get pod可以但get node不行,因为我们刚刚配置role只有pod权限
删除pod看看
kubectl delete pod/http-app-844765cb6c-nfp7l Error from server (Forbidden): pods “http-app-844765cb6c-nfp7l” is forbidden: User “test-cka” cannot delete pods in the namespace “default”
你会发现也删不掉,因为我们role里面配置的权限是watch和list和get
kubectl get pod -n kube-system No resources found. Error from server (Forbidden): pods is forbidden: User “test-cka”” cannot list pods in the namespace “kube-system” [root@master pki]#
可以看见damon这个用户只能访问default这个namespace的pod资源,其他的namespace都访问不了,同样namespace的其他资源也访问不了
ClusterRole和ClusterRoleBinding创建
cat << EOF > cluster_role.yamlkind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata:name: secret-readerrules:- apiGroups: [""]resources: ["nodes"]verbs: ["get", "watch", "list"]---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata:name: read-secrets-globalsubjects:- kind: Username: damonapiGroup: rbac.authorization.k8s.ioroleRef:kind: ClusterRolename: secret-readerapiGroup: rbac.authorization.k8s.ioEOF
**
# 创建
kubectl apply -f cluster_role.yaml

**
