PodSecurityPolicy

PodSecurityPolicy是集群级别的Pod安全策略，自动为集群中的Pod和Volume设置Security Context。

Admission Controller（准入控制器）拦截对 kube-apiserver 的请求，拦截发生在请求的对象被持久化之前，但是在请求被验证和授权之后。这样我们就可以查看请求对象的来源，并验证需要的内容是否正确。通过将它们添加到 kube-apiserver 的—enable-admission-plugins参数中来启用准入控制器。所以如果我们要使用PSP，我们就需要在kube-apiserver中添加起参数，如下：

--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy

其他插件是kubernetes官方推荐的插件。

然后重启kube-apiserver：

# systemctl daemon-reload
# systemctl restart kube-apiserver

这时候就已经启动了PSP控制器，如果我们现在创建Pod试试：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent

然后我们kubectl get pod的时候发现并没有pod。我们再查看deploy的状态，如下：

# kubectl get deployments.
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
nginx                  0/1     0            0           117s
# kubectl get replicasets.
NAME                              DESIRED   CURRENT   READY   AGE
nginx-55fc968d9                   1         0         0       2m17s

我们看到replicaset和deploy都创建成功了，但是replicaset并没有创建pod，这是因为我们集群现在缺少安全策略，所以创建新的Pod不会成功，这时我们就需要使用ServiceAccount。

正常情况下，我们并不会直接创建Pod，都是通过其他控制器比如Deployment、StatefulSet等来创建Pod。我们现在要使用PSP，需要配置kube-controller-manager来为其包含的每个控制器单独使用ServiceAccount，我们可以通过以下参数来添加，如下：

--use-service-account-credentials=true

然后重启controller-manager：

# systemctl daemon-reload
# systemctl restart kube-controller-manager

然后kubenetes就会自动生成如下一些SA，这些SA就指定了哪个控制器可以解析哪些策略：

# kubectl get serviceaccount -n kube-system | egrep -o '[A-Za-z0-9-]+-controller'
attachdetach-controller
certificate-controller
clusterrole-aggregation-controller
cronjob-controller
daemon-set-controller
deployment-controller
disruption-controller
endpoint-controller
expand-controller
job-controller
namespace-controller
node-controller
pv-protection-controller
pvc-protection-controller
replicaset-controller
replication-controller
resourcequota-controller
service-account-controller
service-controller
statefulset-controller
traefik-ingress-controller
ttl-controller

PSP提供一种声明式的方式，用于表达运行的用户和ServiceAccount在我们集群中创建的内容。其主要的策略有：

在上面的示例中，我们需要两个策略：
1、提供限制访问的默认策略，保证使用特权设置无法创建Pod；
2、提升许可策略，允许将特权设置用于某些Pod，比如允许在特定命名空间下创建Pod；

首先，创建一个默认策略：
psp-restrictive.yaml

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restrictive
spec:
  privileged: false
  hostNetwork: false
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  hostPID: false
  hostIPC: false
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - 'configMap'
  - 'downwardAPI'
  - 'emptyDir'
  - 'persistentVolumeClaim'
  - 'secret'
  - 'projected'
  allowedCapabilities:
  - '*'

然后直接创建这个PSP对象：

# kubectl apply -f psp-restrictive.yaml 
podsecuritypolicy.policy/restrictive created
# kubectl get psp
NAME          PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
restrictive   false   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            configMap,downwardAPI,emptyDir,persistentVolumeClaim,secret,projected

其次，创建一个提升策略，用于那些需要提升权限的Pod，比如kube-proxy，它就需要hostNetwork权限：
psp-permissive.yaml

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: permissive
spec:
  privileged: true
  hostNetwork: true
  hostIPC: true
  hostPID: true
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  hostPorts:
  - min: 0
    max: 65535
  volumes:
  - '*'

然后创建这个PSP对象：

# kubectl apply -f psp-permissive.yaml 
podsecuritypolicy.policy/permissive created
# kubectl get psp
NAME          PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
permissive    true           RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *
restrictive   false   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            configMap,downwardAPI,emptyDir,persistentVolumeClaim,secret,projected

但是仅仅部署了这两个策略是不够的，我们RBAC进行授权，不然我们的Pod还是不能创建成功。RBAC确定一个ServiceAccount可以使用的策略，如果使用ClusterRoleBinding可以为ServiceAccount提供限制性策略（restrictive）的访问，如果使用RoleBinding可以为SeriveAccount提供虚空策略的访问。

首先创建允许使用restrictive策略的ClusterRole，然后再创建一个ClusterRoleBinding将所有控制器的ServiceAccount进行绑定：
psp-restrictive-rbac.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp-restrictive
rules:
- apiGroups:
  - extensions
  resources:
  - podsecruritypolicies
  resourceNames:
  - restrictive
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: psp-default
subjects:
- kind: Group
  name: system:serviceaccounts
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: psp-restrictive
  apiGroup: rbac.authorization.k8s.io

然后创建RBAC资源对象：

# kubectl apply -f  psp-restrictive-rbac.yaml 
clusterrole.rbac.authorization.k8s.io/psp-restrictive created
clusterrolebinding.rbac.authorization.k8s.io/psp-default created

然后我们可以看到外面刚开始创建的Pod现在可以创建了：

# kubectl get pod
NAME                                    READY   STATUS    RESTARTS   AGE
nginx-55fc968d9-qn2vr                   1/1     Running   0          17m

但是如果我们现在给这个Deployment清单加一个hostNetwork=true这个特权，观察Pod是否能够创：
nginx-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
      hostNetwork: true

然后我们再次创建这个Deployment：

# kubectl apply -f nginx-deploy.yaml 
deployment.apps/nginx created
# kubectl get pod
NAME                                    READY   STATUS    RESTARTS   AGE
# kubectl get deployments.
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
nginx                  0/1     0            0           12s
# kubectl get replicasets.
NAME                              DESIRED   CURRENT   READY   AGE
nginx-5cd65fd4c6                  1         0         0       18s

我们可以看到Pod并未被创建，我们describe一个replicaset，发现如下日志：

# kubectl describe rs nginx-5cd65fd4c6
......
Events:
  Type     Reason        Age                   From                   Message
  ----     ------        ----                  ----                   -------
  Warning  FailedCreate  41s (x16 over 3m24s)  replicaset-controller  Error creating: pods "nginx-5cd65fd4c6-" is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]

提示我们hostNetwork不允许被使用。

但是在某些情况下，我们需要在某个命名空间下使用特权，这时候我们就可以创建一个允许使用特权的ClusterRole，但是这里我们为特定ServiceAccount设置RoleBinding，如下：
psp-permissive-rbac.yaml

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-permissive
rules:
- apiGroups:
  - extensions
  resources:
  - podsecuritypolicies
  resourceNames:
  - permissive
  verbs:
  - use
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: psp-permissive
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp-permissive
subjects:
- kind: ServiceAccount
  name: daemon-set-controller
  namespace: kube-system
- kind: ServiceAccount
  name: replicaset-controller
  namespace: kube-system
- kind: ServiceAccount
  name: job-controller
  namespace: kube-system

上面定义了对kube-system中的daemonset，replicaset，job拥有特权创建Pod。
然后我们创建RBAC资源清单：

# kubectl apply -f psp-permissive-rbac.yaml 
clusterrole.rbac.authorization.k8s.io/psp-permissive created
rolebinding.rbac.authorization.k8s.io/psp-permissive created

现在我们定义一个测试Deployment：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
      hostNetwork: true

然后创建资源对象：

# kubectl apply -f nginx-deploy.yaml 
deployment.apps/nginx created
[root@ecs-5704-0003 kubernetes]# kubectl get pod -n kube-system 
NAME                                   READY   STATUS    RESTARTS   AGE
......
nginx-5cd65fd4c6-7pn9z                 1/1     Running   0          4s

然后我们可以看到Pod可以正常被创建。

另外还有一种特殊的需求，就是在某个命令空间下只有某个应用可以使用特权，那么针对这类应用就需要单独创建一个SA，然后和permissive策略镜像RoleBinding了，如下：
（1）、创建可以使用特权的SA

# kubectl create serviceaccount specialsa
serviceaccount/specialsa created

（2）、创建RoleBinding
specialsa-psp.yaml

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: specialsa-psp-permissive
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp-permissive
subjects:
- kind: ServiceAccount
  name: specialsa
  namespace: default

然后创建上面的RoleBinding对象：

# kubectl apply -f specialsa-psp.yaml 
rolebinding.rbac.authorization.k8s.io/specialsa-psp-permissive created

然后创建一个测试的Deployment：
ng-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-hostnetwork-deploy
  namespace: default
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
      hostNetwork: true
      serviceAccount: specialsa  # 注意这里使用的sa的权限绑定

然后创建资源对象：

# kubectl apply -f ng-deploy.yaml 
deployment.apps/nginx-hostnetwork-deploy created
# kubectl get pod
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-hostnetwork-deploy-7b65cf7bbd-g5wl5   1/1     Running   0          2s

然后可以发现在default命名空间下可以创建拥有特权的Pod了。