Rancher Integration with Logging Services


Logging is helpful because it allows you to:
日志记录非常有用,因为它允许您:

  • Capture and analyze the state of your cluster

捕获并分析集群的状态

  • Look for trends in your environment

预测环境的未来趋势

  • Save your logs to a safe location outside of your cluster

保持日志到集群外的安全位置

  • Stay informed of events like a container crashing, a pod eviction, or a node dying

持续关注诸如容器crashig、pod驱逐、节点宕机等事件

  • More easily debug and troubleshoot problems

让调试和故障排查更容器

Rancher supports integration with the following services:
支持以下日志服务的集成:

  • Elasticsearch
  • Splunk
  • Kafka
  • Syslog
  • Fluentd

This section covers the following topics:
本章将涉及以下话题:

日志服务是如何工作的?

要求?

日志的范围

开启集群日志

How Logging Integrations Work 日志集成如何工作

Rancher can integrate with popular external services used for event streams, telemetry, or search. These services can log errors and warnings in your Kubernetes infrastructure to a stream.

Rancher可以与流行的事件流(event streams)、遥测(telemetry)或搜索(search)等外部服务集成。这些服务可以将Kubernetes基础架构中的错误和警告记录到流中。

These services collect container log events, which are saved to the /var/log/containers directory on each of your nodes. The service collects both standard and error events. You can then log into your services to review the events collected, leveraging each service’s unique features.

这些服务收集每个节点上的/var/log/container目录中的容器日志事件。该服务同时收集标准事件和错误事件。然后,您可以登录到您的服务以查看已收集的事件,利用每个服务的独特功能。

When configuring Rancher to integrate with these services, you’ll have to point Rancher toward the service’s endpoint and provide authentication information.

在配置Rancher与这些服务集成时,您必须将Rancher指向服务的端点(endpoint)并提供身份验证信息。

Additionally, you’ll have the opportunity to enter key-value pairs to filter the log events collected. The service will only collect events for containers marked with your configured key-value pairs.

此外,您将有机会输入键值对以过滤收集的日志事件。该服务将仅收集标记有您配置的键值对的容器的事件。

NOTE:You can only configure one logging service per cluster or per project. 注意:你只可为每个集群或每个项目配置一个日志服务

Requirements 要求

The Docker daemon on each node in the cluster should be configured with the (default) log-driver: json-file. You can check the log-driver by running the following command:

应将每个节点上的Docker daemon使用默认的日志驱动(log-driver):json-file。您可以通过运行以下命令来检查日志驱动(log-driver)程序:

  1. $ docker info | grep 'Logging Driver'
  2. Logging Driver: json-file

Logging Scope 日志收集范围

You can configure logging at either cluster level or project level.
你可篇日志集群范围或项目范围的日志。

  • Cluster logging writes logs for every pod in the cluster, i.e. in all the projects. For RKE clusters, it also writes logs for all the Kubernetes system components.

集群级别的日志将收集集群中每个pod的日志,例如,收集所有项目的。对于RKE集群,它还收集K8s系统组件的日志。

项目级别的日志服务将收集特定项目中的每个pod的日志。

Logs that are sent to your logging service are from the following locations:
以下位置的日志将发送到日志服务系统中:

  • Pod logs stored at /var/log/containers.

保持在/var/log/containers的pod日志;

  • Kubernetes system components logs stored at /var/lib/rancher/rke/log/.

k8s系统组件日志保存在/var/lib/rancher/rke/log/

Enabling Cluster Logging 开启集群级别日志服务

As an administrator or cluster owner, you can configure Rancher to send Kubernetes logs to a logging service.

  1. From the Global view, navigate to the cluster that you want to configure cluster logging.

  2. Select Tools > Logging in the navigation bar.

  3. Select a logging service and enter the configuration. Refer to the specific service for detailed configuration. Rancher supports integration with the following services:

  4. (Optional) Instead of using the UI to configure the logging services, you can enter custom advanced configurations by clicking onEdit as File, which is located above the logging targets. This link is only visible after you select a logging service.

    • With the file editor, enter raw fluentd configuration for any logging service. Refer to the documentation for each logging service on how to setup the output configuration.
    • If the logging service is using TLS, you also need to complete theSSL Configurationform.
      1. Provide theClient Private KeyandClient Certificate. You can either copy and paste them or upload them by using theRead from a filebutton.
        • You can use either a self-signed certificate or one provided by a certificate authority.
        • You can generate a self-signed certificate using an openssl command. For example:
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
  1. If you are using a self-signed certificate, provide the **CA Certificate PEM**.<br />
  1. (Optional) Complete theAdditional Logging Configurationform.
    1. Optional: Use the Add Field button to add custom log fields to your logging configuration. These fields are key value pairs (such as foo=bar) that you can use to filter the logs from another system.
    2. Enter a Flush Interval. This value determines how often Fluentd flushes data to the logging server. Intervals are measured in seconds.
    3. Include System Log. The logs from pods in system project and RKE components will be sent to the target. Uncheck it to exclude the system logs.
  2. Click Test. Rancher sends a test log to the service.

    NOTE:This button is replaced with Dry Run if you are using the custom configuration editor. In this case, Rancher calls the fluentd dry run command to validate the configuration.

  3. Click Save.

Result: Rancher is now configured to send logs to the selected service. Log into the logging service so that you can start viewing the logs.