- Custom resources自定义资源
- Custom controllers自定义控制器
- Should I add a custom resource to my Kubernetes Cluster?是否应该向k8s中添加定制资源?
- Should I use a configMap or a custom resource?应该使用configMap还是定制资源
- Adding custom resources添加定制资源
- CustomResourceDefinitions定制资源定义
- API server aggregationAPI服务器聚合
- Choosing a method for adding custom resources选择添加定制资源的方式
- Preparing to install a custom resource安装定制资源的准备
- Accessing a custom resource访问定制资源
- What’s next
本文由 简悦 SimpRead 转码, 原文地址 kubernetes.io
Custom resources are extensions of the Kubernetes API. This page discusses when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.
自定义资源是对k8s API的扩充。本文讨论了应合适向k8s集群中添加自定义资源以及合适使用独立服务。本文描述了两种添加自定义资源的方法,以及如何在他们之前作出选择。
Custom resources自定义资源
A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind; for example, the built-in pods resource contains a collection of Pod objects.
资源是k8s API中用于存储一系列特定API对象的端点。例如,内置的POD类型的资源中就存储了一系统Pod对象。
A custom resource is an extension of the Kubernetes API that is not necessarily available in a default Kubernetes installation. It represents a customization of a particular Kubernetes installation. However, many core Kubernetes functions are now built using custom resources, making Kubernetes more modular.
自定义资源是k8s API的扩展,默认安装k8s时不一定可用。其代表自定义的特殊的k8s安装。然而,很多k8s函数现在使用自定义资源进行构建,从而让k8s更加模块化。
Custom resources can appear and disappear in a running cluster through dynamic registration, and cluster admins can update custom resources independently of the cluster itself. Once a custom resource is installed, users can create and access its objects using kubectl, just as they do for built-in resources like Pods.
自定义资源可以仔任意一个运行的集群中,通过动态方式进行注册。集群管理员可以独立更新自定义资源。自定义资源安装后,用户可以使用kubecet来创建和访问它对象,就像使用内建的自定义资源PODs一样。
使用现有资源创建和使用CR的区别:
- 使用CRD之后,对资源的查找更加方便,也更易于维护
- 便于编制各种自定义的事件
Custom controllers自定义控制器
On their own, custom resources let you store and retrieve structured data. When you combine a custom resource with a custom controller, custom resources provide a true declarative API.
定制资源本身只能让你存储和检索结构化的数据。当你将定制资源和定制控制器结合起来时,自定义资源提供了真正的声明式的API。
A declarative API allows you to declare or specify the desired state of your resource and tries to keep the current state of Kubernetes objects in sync with the desired state. The controller interprets the structured data as a record of the user’s desired state, and continually maintains this state.
声明式的API允许声明或定义资源的期待状态,并会自动将k8s对象的当前状态同步到期待状态。控制器将结构化的数据转义为一条用户期待状态的记录,之后持续的维护这个状态。
You can deploy and update a custom controller on a running cluster, independently of the cluster’s lifecycle. Custom controllers can work with any kind of resource, but they are especially effective when combined with custom resources. The Operator pattern combines custom resources and custom controllers. You can use custom controllers to encode domain knowledge for specific applications into an extension of the Kubernetes API.
可以在运行的集群中部署和更新自定义控制器,而不用考虑集群的生命周期。定制控制器可以用于所有类型的资源,不过他们与定制资源结合起来使用时最为有效。
operator模式就是将定制资源和定制控制器结合在一起来使用的。可以使用定制控制器去编码特定应用的相关知识,并将其扩充到k8s API中。
Should I add a custom resource to my Kubernetes Cluster?是否应该向k8s中添加定制资源?
When creating a new API, consider whether to aggregate your API with the Kubernetes cluster APIs or let your API stand alone.
当创建一个新的API时,考虑是否将API聚合到k8s集群API中或让API独立
Consider API aggregation if: | Prefer a stand-alone API if: |
---|---|
Your API is Declarative. | Your API does not fit the Declarative model. |
You want your new types to be readable and writable using kubectl . |
kubectl support is not required |
You want to view your new types in a Kubernetes UI, such as dashboard, alongside built-in types. | Kubernetes UI support is not required. |
You are developing a new API. | You already have a program that serves your API and works well. |
You are willing to accept the format restriction that Kubernetes puts on REST resource paths, such as API Groups and Namespaces. (See the API Overview.) | You need to have specific REST paths to be compatible with an already defined REST API. |
Your resources are naturally scoped to a cluster or namespaces of a cluster. | Cluster or namespace scoped resources are a poor fit; you need control over the specifics of resource paths. |
You want to reuse Kubernetes API support features. | You don’t need those features. |
Declarative APIs声明式API
In a Declarative API, typically:
- Your API consists of a relatively small number of relatively small objects (resources).
- The objects define configuration of applications or infrastructure.
- The objects are updated relatively infrequently.
- Humans often need to read and write the objects.
- The main operations on the objects are CRUD-y (creating, reading, updating and deleting).
- Transactions across objects are not required: the API represents a desired state, not an exact state.
在声明式API中:
- 你的API包含相对而言数量不多的,尺寸较小的对象(资源)
- 对象定义了应用或基础设施的配置信息
- 对象更新操作频率相对较低
- 通常需要任来读写对象
- 对象的主要操作是CRUD风格的(创建、读取、更新、删除)
- 不要求跨对象的事务支持,API代表的是一个期待的状态而不是确切的状态。
Imperative APIs are not declarative. Signs that your API might not be declarative include:
- The client says “do this”, and then gets a synchronous response back when it is done.
- The client says “do this”, and then gets an operation ID back, and has to check a separate Operation object to determine completion of the request.
- You talk about Remote Procedure Calls (RPCs).
- Directly storing large amounts of data; for example, > a few kB per object, or > 1000s of objects.
- High bandwidth access (10s of requests per second sustained) needed.
- Store end-user data (such as images, PII, etc.) or other large-scale data processed by applications.
- The natural operations on the objects are not CRUD-y.
- The API is not easily modeled as objects.
- You chose to represent pending operations with an operation ID or an operation object.
命令式的API不是声明式的。以下迹象表明你的API可能不是声明式的:
- 客户端发出指令,在完成后得到同步响应
- 客户端发出指令,之后得到操作ID,之后必须检查一个操作(Operation)对象来判断请求是否完成。
- 考虑将此API作为远程过程调用RPC
- 直接存储大量数据,
- 在对象上的原始操作不是CRUD风格的
- API不容易作为对象来建模
- 决定使用操作ID或操作对象来表现等待的操作
Should I use a configMap or a custom resource?应该使用configMap还是定制资源
Use a ConfigMap if any of the following apply:
- There is an existing, well-documented config file format, such as a
mysql.cnf
orpom.xml
. - You want to put the entire config file into one key of a configMap.
- The main use of the config file is for a program running in a Pod on your cluster to consume the file to configure itself.
- Consumers of the file prefer to consume via file in a Pod or environment variable in a pod, rather than the Kubernetes API.
- You want to perform rolling updates via Deployment, etc., when the file is updated.
Note: Use a secret for sensitive data, which is similar to a configMap but more secure.
在以下应用中使用ConfigMap :
- 已有现存的、文档完备的配置文件格式,例如mysql.cnf或pom.xml等等
- 希望将整个配置文件放在一个configMap的一个主键下面
- 配置文件的主要用途是让Pod中运行的一个程序来配置自身的信息
- 配置文件的消费者倾向于通过Pod中的文件或环境变量进行消费,而不是k8s API
- 当文件更新时,希望通过Depolyment之类的方式进行滚动更新
Use a custom resource (CRD or Aggregated API) if most of the following apply:
- You want to use Kubernetes client libraries and CLIs to create and update the new resource.
- You want top-level support from
kubectl
; for example,kubectl get my-object object-name
. - You want to build new automation that watches for updates on the new object, and then CRUD other objects, or vice versa.
- You want to write automation that handles updates to the object.
- You want to use Kubernetes API conventions like
.spec
,.status
, and.metadata
. - You want the object to be an abstraction over a collection of controlled resources, or a summarization of other resources.
当需要应用以下情景时,使用定制资源(CRD或聚合API)
- 希望使用k8s客户端库,或CLI工具来创建和更新资源
- 希望使用
kubectl
操作你的资源,例如kubectl get my-object object-name
- 希望构建新的自动化机智,检测新对象上的更新事件,并对其他对象执行CRUD操作,或者监控后者更新前者。
- 希望编写自动化组件来处理对象的更新
- 希望使用k8s api的编写规范,例如
.spec
,status
,.metadata
- 希望对象是一组受控资源的抽象,或其他资源的一个归纳总结
Adding custom resources添加定制资源
Kubernetes provides two ways to add custom resources to your cluster:
- CRDs are simple and can be created without any programming.
- API Aggregation requires programming, but allows more control over API behaviors like how data is stored and conversion between API versions.
k8s提供了两种方式来向集群中添加定制的资源
- CRD 使用简单,并且不需编程即可创建
- 聚合API,要求编程,但允许对API的行为进行更多的控制,例如数据是如何存储的,以及API版本之间的转换
Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility is compromised.
Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called API Aggregation (AA). To users, the Kubernetes API appears extended.
CRDs allow users to create new types of resources without adding another API server. You do not need to understand API Aggregation to use CRDs.
Regardless of how they are installed, the new resources are referred to as Custom Resources to distinguish them from built-in Kubernetes resources (like pods).
k8s提供这两个选项来满足不同用户的需求,因此,易用性和灵活性都不会受到影响。
聚合API是指,位于主API服务器后的,下位API服务器,主服务器以代理的方式工作。,这种安排方式被称作API聚合。对于用户而言,k8s API看起来是被扩展了。
CRD 允许用户创建新的资源类型,而不需要添加额外的API 服务器。使用CRD时,不需要理解API聚合。
不管资源是如何安装的,新的资源都会作为定制资源被引用,以便与k8s的内置资源进行区分。
CustomResourceDefinitions定制资源定义
The CustomResourceDefinition API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource. The name of a CRD object must be a valid DNS subdomain name.
CustomResourceDefinition
API资源,支持用户定义定制资源。定义CRD对象的操作会按照用户设定的策略创建一个具名资源。 k8s API服务会服务于此资源并管理该资源的存储。CRD对象的名字必须是一个有效的DNS子域名。
This frees you from writing your own API server to handle the custom resource, but the generic nature of the implementation means you have less flexibility than with API server aggregation.
因此用户不必编写自己的API服务器来管理定制资源,但是实现机制的通用性意味着CRD不如聚合API更加灵活
Refer to the custom controller example for an example of how to register a new custom resource, work with instances of your new resource type, and use a controller to handle events.
可以参照
custom controller example
章节来查看如何注册一个新的定制资源,以及如何使用此资源、如何利用控制器来处理事件。
API server aggregationAPI服务器聚合
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like pods and services, and can also generically handle custom resources through CRDs.
通常,k8s api中的每个资源都必须要由处理REST请求和管理对象持久化存储的代码。k8s api主服务可以管理内建资源,例如pods,services,并且也可以使用通用方法来管理通过CRD创建的定制资源。
The aggregation layer allows you to provide specialized implementations for your custom resources by writing and deploying your own standalone API server. The main API server delegates requests to you for the custom resources that you handle, making them available to all of its clients.
聚合层(aggregation layer),支持用户通过编写和部署自己独立的API服务来为定制资源提供特定的实现方式。API主服务器将对定制资源的请求委托给用户进行处理,同时保证其下所有的客户端都可以访问这些资源。
Choosing a method for adding custom resources选择添加定制资源的方式
CRDs are easier to use. Aggregated APIs are more flexible. Choose the method that best meets your needs.
CRD使用简单。聚合API更加灵活。选择最适合需求的方法。
Typically, CRDs are a good fit if:
- You have a handful of fields
- You are using the resource within your company, or as part of a small open-source project (as opposed to a commercial product)
通常,CRD更适合:
- 定制资源的字段不多
- 在公司内部使用资源,或作为小型开源项目使用(不是作为商用产品)
Comparing ease of use 易用性对比
CRDs are easier to create than Aggregated APIs.
CRDs | Aggregated API |
---|---|
Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. |
No additional service to run; CRDs are handled by API server. | An additional service to create and that could fail. |
No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. | May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated API server. |
No need to handle multiple versions of your API; for example, when you control the client for this resource, you can upgrade it in sync with the API. | You need to handle multiple versions of your API; for example, when developing an extension to share with the world. |
Advanced features and flexibility特性和灵活性对比
Aggregated APIs offer more advanced API features and customization of other features; for example, the storage layer.
Feature | Description | CRDs | Aggregated API |
---|---|---|---|
Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can’t all update at the same time. | Yes. Most validation can be specified in the CRD using OpenAPI v3.0 validation. Any other validations supported by addition of a Validating Webhook. | Yes, arbitrary validation checks |
Defaulting | See above | Yes, either via OpenAPI v3.0 validation default keyword (GA in 1.17), or via a Mutating Webhook (though this will not be run when reading from etcd for old objects). |
Yes |
Multi-versioning | Allows serving the same object through two API versions. Can help ease API changes like renaming fields. Less important if you control your client versions. | Yes | Yes |
Custom Storage | If you need storage with a different performance mode (for example, a time-series database instead of key-value store) or isolation for security (for example, encryption of sensitive information, etc.) | No | Yes |
Custom Business Logic | Perform arbitrary checks or actions when creating, reading, updating or deleting an object | Yes, using Webhooks. | Yes |
Scale Subresource | Allows systems like HorizontalPodAutoscaler and PodDisruptionBudget interact with your new resource | Yes | Yes |
Status Subresource | Allows fine-grained access control where user writes the spec section and the controller writes the status section. Allows incrementing object Generation on custom resource data mutation (requires separate spec and status sections in the resource) | Yes | Yes |
Other Subresources | Add operations other than CRUD, such as “logs” or “exec”. | No | Yes |
strategic-merge-patch | The new endpoints support PATCH with Content-Type: application/strategic-merge-patch+json . Useful for updating objects that may be modified both locally, and by the server. For more information, see “Update API Objects in Place Using kubectl patch” |
No | Yes |
Protocol Buffers | The new resource supports clients that want to use Protocol Buffers | No | Yes |
OpenAPI Schema | Is there an OpenAPI (swagger) schema for the types that can be dynamically fetched from the server? Is the user protected from misspelling field names by ensuring only allowed fields are set? Are types enforced (in other words, don’t put an int in a string field?) |
Yes, based on the OpenAPI v3.0 validation schema (GA in 1.16). | Yes |
Common Features共同特性
When you create a custom resource, either via a CRD or an AA, you get many features for your API, compared to implementing it outside the Kubernetes platform:
当你创建一个定制资源时,不管是通过CRD还是AA,你的API与在k8s平台外使用相比,会获得很多特性。
Feature | What it does |
---|---|
CRUD | The new endpoints support CRUD basic operations via HTTP and kubectl |
Watch | The new endpoints support Kubernetes Watch operations via HTTP |
Discovery | Clients like kubectl and dashboard automatically offer list, display, and field edit operations on your resources |
json-patch | The new endpoints support PATCH with Content-Type: application/json-patch+json |
merge-patch | The new endpoints support PATCH with Content-Type: application/merge-patch+json |
HTTPS | The new endpoints uses HTTPS |
Built-in Authentication | Access to the extension uses the core API server (aggregation layer) for authentication |
Built-in Authorization | Access to the extension can reuse the authorization used by the core API server; for example, RBAC. |
Finalizers | Block deletion of extension resources until external cleanup happens. |
Admission Webhooks | Set default values and validate extension resources during any create/update/delete operation. |
UI/CLI Display | Kubectl, dashboard can display extension resources. |
Unset versus Empty | Clients can distinguish unset fields from zero-valued fields. |
Client Libraries Generation | Kubernetes provides generic client libraries, as well as tools to generate type-specific client libraries. |
Labels and annotations | Common metadata across objects that tools know how to edit for core and custom resources. |
Preparing to install a custom resource安装定制资源的准备
There are several points to be aware of before adding a custom resource to your cluster.
Third party code and new points of failure第三方代码和新失效点
While creating a CRD does not automatically add any new points of failure (for example, by causing third party code to run on your API server), packages (for example, Charts) or other installation bundles often include CRDs as well as a Deployment of third-party code that implements the business logic for a new custom resource.
Installing an Aggregated API server always involves running a new Deployment.
尽管安装CRD不会自动添加新的失效点,(例如,导致第三方代码在API服务器上运行),包(例如helm的chart)或其他安装包通常会包含CRD以及含有第三方代码的Deployment,负责实现新定制资源的业务逻辑。
安装聚合API服务器时,也总会包含一个新的Deployment
Storage存储
Custom resources consume storage space in the same way that ConfigMaps do. Creating too many custom resources may overload your API server’s storage space.
和ConfigMap一样,定制资源也会消耗存储空间。创建太多的定制资源可能会导致API服务器的存储空间超载。
Aggregated API servers may use the same storage as the main API server, in which case the same warning applies.
聚合API服务器可以和主服务器使用同一存储空间,也需要注意此警告。
CRDs always use the same authentication, authorization, and audit logging as the built-in resources of your API server.
CRD通常与API服务器的内建资源使用相同的授权,认证,日志监控。
If you use RBAC for authorization, most RBAC roles will not grant access to the new resources (except the cluster-admin role or any role created with wildcard rules). You’ll need to explicitly grant access to the new resources. CRDs and Aggregated APIs often come bundled with new role definitions for the types they add.
Aggregated API servers may or may not use the same authentication, authorization, and auditing as the primary API server.
如果使用RBAC来进行授权认证,大多数的RBAC角色没有访问新资源的权限,(除了集群管理员角色,或使用通配符规则创建的角色),需要显示的添加对这些资源的访问权限。CRD和聚合API通常和与他们所添加的新的类型向对比的新的角色绑定发布。
Accessing a custom resource访问定制资源
Kubernetes client libraries can be used to access custom resources. Not all client libraries support custom resources. The Go and Python client libraries do.
k8s客户端库可以用于访问定制资源。但不是所有的客户端都支持定制资源。GO和Pyton客户端是支持的。
When you add a custom resource, you can access it using:
kubectl
- The kubernetes dynamic client.
- A REST client that you write.
- A client generated using Kubernetes client generation tools (generating one is an advanced undertaking, but some projects may provide a client along with the CRD or AA).
添加了新的定制资源后,可以使用一下方法进行访问 :
kubectl
- k8s动态客户端
- 自己编写的REST客户端
- 使用k8s客户端生成工具创建的客户端。(生成客户端的工具有些困难,但一些项目客户端会随着CRD或聚合API一起提供一个客户端)
What’s next
Learn how to Extend the Kubernetes API with the aggregation layer.
Learn how to Extend the Kubernetes API with CustomResourceDefinition.
Last modified March 29, 2021 at 11:54 AM PST: update the correct operator link (ecede1568)