Different ways to change the behavior of your Kubernetes cluster.

Kubernetes is highly configurable and extensible. As a result, there is rarely a need to fork or submit patches to the Kubernetes project code.

This guide describes the options for customizing a Kubernetes cluster. It is aimed at cluster operators who want to understand how to adapt their Kubernetes cluster to the needs of their work environment. Developers who are prospective Platform Developers or Kubernetes Project Contributors will also find it useful as an introduction to what extension points and patterns exist, and their trade-offs and limitations.

Overview概述

Customization approaches can be broadly divided into configuration, which only involves changing flags, local configuration files, or API resources; and extensions, which involve running additional programs or services. This document is primarily about extensions.

k8s的定制方法可分为配置(configutation)和扩展(extensions).前者值改变参数标志、本地配置文件和API资源。后者包含运行额外的程序或服务。本文主要考略扩展。

Configuration配置

Configuration files and flags are documented in the Reference section of the online documentation, under each binary:

配置文件和参数标志的说明文档才参考章节，按可执行文件组织

Flags and configuration files may not always be changeable in a hosted Kubernetes service or a distribution with managed installation. When they are changeable, they are usually only changeable by the cluster administrator. Also, they are subject to change in future Kubernetes versions, and setting them may require restarting processes. For those reasons, they should be used only when there are no other options.

Built-in Policy APIs, such as ResourceQuota, PodSecurityPolicies, NetworkPolicy and Role-based Access Control (RBAC), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a defined support policy like other Kubernetes APIs. For these reasons, they are preferred over configuration files and flags where suitable.

Extensions扩展

Extensions are software components that extend and deeply integrate with Kubernetes. They adapt it to support new types and new kinds of hardware.

扩展是k8s的软件组件，扩充了k8s的能力并和k8s深度集成。通过适配组件来支持新的类型和新的硬件种类。

Most cluster administrators will use a hosted or distribution instance of Kubernetes. As a result, most Kubernetes users will not need to install extensions and fewer will need to author new ones.

大多数集群管理员使用一种托管的k8s或某种发型版本。因此，大多数的k8s用户不需要安装扩展，至于需要自己编写新的扩展的情况就更少了。

Extension Patterns扩展模式

Kubernetes is designed to be automated by writing client programs. Any program that reads and/or writes to the Kubernetes API can provide useful automation. Automation can run on the cluster or off it. By following the guidance in this doc you can write highly available and robust automation. Automation generally works with any Kubernetes cluster, including hosted clusters and managed installations.

k8s从设计上即支持通过编写客户端程序来将其操作自动化。任何可以读写k8s API的程序都可以提供有用的自动化能力。自动化可以在集群或非集群上运行。通过遵循以下部署，你可以编写高可用的，鲁棒性强的自动化组件。自动化组件通常可以用于所有k8s集群，包括托管集群和受控的安装环境。

There is a specific pattern for writing client programs that work well with Kubernetes called the Controller pattern. Controllers typically read an object’s .spec, possibly do things, and then update the object’s .status.

有一种特殊的编写客户端程序的模式叫做控制器模式，能够和好的和k8s协同工作 。控制器会读取一个对象的.spec字段，执行相应的操作，之后更新对象的.status字段。

A controller is a client of Kubernetes. When Kubernetes is the client and calls out to a remote service, it is called a Webhook. The remote service is called a Webhook Backend. Like Controllers, Webhooks do add a point of failure.

控制器是k8s的客户端。当k8s充当客户端，调用远程服务器时，对象的远程组件叫做Webhook。远程服务被称为Webhook后端。和controller一样，webhook也会在整个架构中引入失效点(point of failure).

In the webhook model, Kubernetes makes a network request to a remote service. In the Binary Plugin model, Kubernetes executes a binary (program). Binary plugins are used by the kubelet (e.g. Flex Volume Plugins and Network Plugins) and by kubectl.

在webhook模式中，k8s向远程服务发出网络请求。在二进制插件模块，k8s执行可执行程序。二进制模块由kubelet和kubectl调用。

Below is a diagram showing how the extension points interact with the Kubernetes control plane.

下图展示了这些扩展点如何与k8s控制面进行交互。

Extension Points扩展点

This diagram shows the extension points in a Kubernetes system.

  1. Users often interact with the Kubernetes API using kubectl. Kubectl plugins extend the kubectl binary. They only affect the individual user’s local environment, and so cannot enforce site-wide policies.

    用户通常使用kubectl和k8s API进行交互。kubectl 插件用于扩展kubectl程序。只会影响单个用户的本地环境，因此无法强制实施整个站点范围的策略。

  1. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the API Access Extensions section.

    API 服务器处理所有的请求，API服务器中的部分扩展点类型支持认证请求，或根据和基于其内容、编辑内容，删除处理来进行阻拦。这些扩展点在API Access Extensions节详细描述。

  1. The apiserver serves various kinds of resources. Built-in resource kinds, like pods, are defined by the Kubernetes project and can’t be changed. You can also add resources that you define, or that other projects have defined, called Custom Resources, as explained in the Custom Resources section. Custom Resources are often used with API Access Extensions.

    API服务器为各种资源提供服务。内置资源，例如pods，由k8s项目定义，无法被修改。你可以添加自定义的资源或其他项目定义的资源，这些资源被称为定制资源，在定制资源章节会有响应的描述。定制资源通过会和API访问扩展结合使用。

  1. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the Scheduler Extensions section.

    k8s调度器决定了pod运行的结点。有很多方式来扩展调度器。详细可以查看调度器扩展章节。

  1. Much of the behavior of Kubernetes is implemented by programs called Controllers which are clients of the API-Server. Controllers are often used in conjunction with Custom Resources.

    k8s的大多数行为都由被称为控制器的程序实现，控制器是API服务器的一个客户端。控制器通常会和定制资源结合使用。

  1. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. Network Plugins allow for different implementations of pod networking.

    kubelet运行在各个结点上。帮助pod展现为虚拟的服务器并在集群网络中拥有自己的IP。网络插件使得k8s能够采用不同的技术来连接pod网络。

  1. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via Storage Plugins.

    kubelet也会挂载和卸载容器的存储卷。可以通过存储插件来扩展存储类型。

If you are unsure where to start, this flowchart can help. Note that some solutions may involve several types of extensions.

API ExtensionsAPI扩展

User-Defined Types用户定义类型

Consider adding a Custom Resource to Kubernetes if you want to define new controllers, application configuration objects or other declarative APIs, and to manage them using Kubernetes tools, such as kubectl.

如果需要定义新的控制器，应用程序配置对象或其他声明式API，并且可以使用k8s的工具，例如kubectl来进行管理。则考虑向K8S中添加定制资源，

Do not use a Custom Resource as data storage for application, user, or monitoring data.

For more about Custom Resources, see the Custom Resources concept guide.

不要使用定制资源作为应用的数据存储，用户或监控数据

关于定制资源的更多信息，可以查看定制资源概念指南

Combining New APIs with Automation整合新API和自动化组件

The combination of a custom resource API and a control loop is called the Operator pattern. The Operator pattern is used to manage specific, usually stateful, applications. These custom APIs and control loops can also be used to control other resources, such as storage or policies.

定制资源API和控制回路的组合被称为Operator模式。Operator模式用来管理特定的、通常是有状态的应用。这些自定义的API和控制回路也可以用于管理其他资源，例如存储和策略。

Changing Built-in Resources更改内置资源

When you extend the Kubernetes API by adding custom resources, the added resources always fall into a new API Groups. You cannot replace or change existing API groups. Adding an API does not directly let you affect the behavior of existing APIs (e.g. Pods), but API Access Extensions do.

当通过添加定制资源来扩展k8s API时，添加的资源会归到新的API群组。用户无法更改现存的API群组。添加API的操作不会让你直接影响的显存API（例如Pods）的行为。但是API访问扩展能够实现这一点。

API Access Extensions API访问扩展

When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See Controlling Access to the Kubernetes API for more on this flow.

但一个请求到达API服务器时，首先需要被认证、授权。之后服从于不同的准入控制。

查看控制K8S API访问权限章节来查阅详细内容。

Each of these steps offers extension points.

Kubernetes has several built-in authentication methods that it supports. It can also sit behind an authenticating proxy, and it can send a token from an Authorization header to a remote service for verification (a webhook). All of these methods are covered in the Authentication documentation.

这些步骤中都可以设置扩展点。

k8s提供多种内置的认证方法。也可以使用认证代理，由k8s将授权标头中的令牌发送到远程服务(webhook)来执行认证操作。所有这些方法都在身份认证文档中由详细论述。

Authentication身份认证

Authentication maps headers or certificates in all requests to a username for the client making the request.

身份认证将所有请求的标头或证书映射到发出该请求的客户端的用户名下。

Kubernetes provides several built-in authentication methods, and an Authentication webhook method if those don’t meet your needs.

k8s提供了多种内置的认证方法，如果不满足要求的话，也可以使用webhook认证方法

Authorization 授权认证

Authorization determines whether specific users can read, write, and do other operations on API resources. It works at the level of whole resources — it doesn’t discriminate based on arbitrary object fields. If the built-in authorization options don’t meet your needs, and Authorization webhook allows calling out to user-provided code to make an authorization decision.

认证过程决定了特定的用户是否可以读写或对API资源执行其他操作，它作用于整个资源的层面。不会基于对象的特定字段作出不同的判决。如果内置的授权方法不满足你的要求，可以调用webhook授权来调用用户提供的代码可进行授权认证过程。

Dynamic Admission Control动态准入控制

After a request is authorized, if it is a write operation, it also goes through Admission Control steps. In addition to the built-in steps, there are several extensions:

  • The Image Policy webhook restricts what images can be run in containers.
  • To make arbitrary admission control decisions, a general Admission webhook can be used. Admission Webhooks can reject creations or updates.

一个请求被授权之后，如果是写入操作，则会经过准备控制检查剪短。除了内置的步骤之类，还有其他扩展：

  1. Image Policy webhook 限制了容器内可以运行的镜像
  2. 为了创建任意的准入控制结果，可以使用通用的准入webhook。准入webhook可以拒绝对象的创建或更新。

Infrastructure Extensions基础设施扩展

Storage Plugins存储插件

Flex Volumes allow users to mount volume types without built-in support by having the Kubelet call a Binary Plugin to mount the volume.

Flex卷支持用户挂在无需内建支持的卷类型。kubelet会调用响应的可执行文件来挂在对应的存储。

Device Plugins设备插件

Device plugins allow a node to discover new Node resources (in addition to the builtin ones like cpu and memory) via a Device Plugin.

使用设备插件能让结点发现新的结点资源（除了内置的，例如内存和CPU之类的资源）

Network Plugins网络插件

Different networking fabrics can be supported via node-level Network Plugins.

通过结点层级的网络插件可以支持不同的网络设置

Scheduler Extensions调度器扩展

The scheduler is a special type of controller that watches pods, and assigns pods to nodes. The default scheduler can be replaced entirely, while continuing to use other Kubernetes components, or multiple schedulers can run at the same time.

This is a significant undertaking, and almost all Kubernetes users find they do not need to modify the scheduler.

The scheduler also supports a webhook that permits a webhook backend (scheduler extension) to filter and prioritize the nodes chosen for a pod.

调度器是特殊类型的控制器，用于监控POD并将POD分配给制定结点。默认的调度器可以被完全取代，同时仍然使用k8s的其他组件。多个调度器可以同时运行。

这是一项重大的任务，几乎所有的k8s用户都不需要修改调度器

调度器也支持webhook，允许webhook后端（调度器的扩展）为Pod筛选和优化结点。

What’s next