0 基本简介

Filebeat是一个开源的文本日志收集器,它是elastic公司Beats数据采集产品的一个子产品，采用go语言开发，一般安装在业务服务器上作为代理来监测日志目录或特定的日志文件，并把它们发送到logstash、elasticsearch、redis或Kafka等。可以在官方地址https://www.elastic.co/downloads/beats下载各个版本的Filebeat。
Filebeat是一个轻量级的日志监测、传输工具，它最大的特点是性能稳定、配置简单、占用系统资源很少。这也是强烈推荐Filebeat的原因。下图是官方给出的Filebeat架构图:
image.png
从图中可以看出，Filebeat主要由两个组件构成: prospector(探测器）和harvester(收集器)。这两类组件一起协作完成Filebeat的工作。
其中，Harvester负责进行单个文件的内容收集，在运行过程中，每一个Harvester会对一个文件逐行进行内容读取，并且把读写到的内容发送到配置的output中。当Harvester开始进行文件的读取后，将会负责这个文件的打开和关闭操作，因此，在Harvester运行过程中，文件都处于打开状态。如果在收集过程中，删除了这个文件或者是对文件进行了重命名，Filebeat依然会继续对这个文件进行读取，这时候将会一直占用着文件所对应的磁盘空间，直到Harvester关闭。
Prospector负责管理Harvster，它会找到所有需要进行读取的数据源。然后交给Harvster进行内容收集，如果
input type配置的是log类型，Prospector将会去配置路径下查找所有能匹配上的文件，然后为每一个文件创建一个Harvster。
综上所述，filebeat的工作流程为︰当开启filebeat程序的时候，它会启动一个或多个探测器(prospector)去检
测指定的日志目录或文件，对于探测器找出的每一个日志文件，filebeat会启动收集进程(harvester)，每一个收集进程读取一个日志文件的内容，然后将这些日志数据发送到后台处理程序(spooler），后台处理程序会集合这些事件，最后发送集合的数据到output指定的目的地。

1 安装部署

elastic官网提供了多种安装方式，具体方法参见官方网站elastic.co

2 主配置文件

2.1 配置文件语法

  1. filebeat.inputs:
  2. - type: log
  3. enabled: true #是否开启日志采集功能
  4. paths: 
  5.   - /var/log/*.log    #指定日志收集的路径，支持通配符，可以写多个
  6. filebeat.config.modules: #内置的收集日志的模块配置文件的存放路径
  7.   path: ${path.config}/modules.d/*.yml
  8.   reload.enabled: false    #当模块的配置文件有更新时，此程序是否要自动加载，false不加载，true加载
  9. setup.template.settings:
  10.   index.number_of_shards: 1
  11. output.console: #添加输出到终端即屏幕上
  12.   pretty: true
  13. # 输出到logstash
  14. #output.logstash:
  15.   #  logstash的IP和端口
  16.   #  hosts: ["172.26.139.148.5044"]
  17. processors: #
  18.   - add_host_metadata: #添加此主机的源数据信息到输出数据中，比如IP MAC OS等信息
  19.       when.not.contains.tags: forwarded

2.2 输出示例

下图message字段的内容就是我自己定义的一个测试文件中的内容
备注：当运行filebeat输出到控制台时，当我们终止了filebeat进程时，需要删除/var/lib/filebeat/目录下的文件才可以正常运行第二次filebeat输出到终端

  1. [root@gitlab ~]# filebeat 
  2. {
  3.   "@timestamp": "2021-07-28T01:32:35.942Z",
  4.   "@metadata": {
  5.     "beat": "filebeat",
  6.     "type": "_doc",
  7.     "version": "7.13.4"
  8.   },
  9.   "log": {
  10.     "offset": 0,
  11.     "file": {
  12.       "path": "/test/test.log"
  13.     }
  14.   },
  15.   "message": "Hello  xiang shi chuan,This is filebeat test logs",
  16.   "input": {
  17.     "type": "log"
  18.   },
  19.   "agent": {
  20.     "type": "filebeat",
  21.     "version": "7.13.4",
  22.     "hostname": "gitlab.xsc.org",
  23.     "ephemeral_id": "9fe4035e-d1ba-4a56-bdc2-3b5e82972ea9",
  24.     "id": "7ac65998-f73a-4ab5-87eb-03a01dffd3d3",
  25.     "name": "gitlab.xsc.org"
  26.   },
  27.   "ecs": {
  28.     "version": "1.8.0"
  29.   },
  30.   "host": {
  31.     "ip": [
  32.       "10.0.0.253",
  33.       "fe80::60b4:786d:82a1:67c5",
  34.       "192.168.235.253",
  35.       "fe80::1dc7:a766:ef32:5e31",
  36.       "172.17.0.1",
  37.       "172.18.0.1",
  38.       "fe80::42:c4ff:fe99:ea06"
  39.     ],
  40.     "mac": [
  41.       "00:0c:29:9d:f9:ae",
  42.       "00:0c:29:9d:f9:b8",
  43.       "02:42:a0:f8:8b:f7",
  44.       "02:42:c4:99:ea:06"
  45.     ],
  46.     "name": "gitlab.xsc.org",
  47.     "hostname": "gitlab.xsc.org",
  48.     "architecture": "x86_64",
  49.     "os": {
  50.       "type": "linux",
  51.       "platform": "centos",
  52.       "version": "7 (Core)",
  53.       "family": "redhat",
  54.       "name": "CentOS Linux",
  55.       "kernel": "3.10.0-957.el7.x86_64",
  56.       "codename": "Core"
  57.     },
  58.     "id": "c3a242bc4cc14d5cb4bff8d8c65e27e6",
  59.     "containerized": false
  60.   }
  61. }

2.3 自定义输出格式

  1. output.console: #添加输出到终端即屏幕上
  2.   codec.format:
  3.     string: '%{[@timestamp]} %{[message]}'
  4.   pretty: true

3 模块使用

3.1 使能模块

filebeat提供了很多已经写好的公共模块，我们可以直接使用，就不需要去修改主配置文件了，直接enable开启就可以使用，同时也可以对模块做一些自定义的修改

  1. [root@gitlab modules.d]# filebeat modules  enable nginx
  2. Enabled nginx

3.2 修改模块配置

在默认的配置上添加了var.paths路径，可以定义多个，但是默认情况下Filebeat还是会去/var/log/nginx/目录下去查找日志

  1. [root@gitlab ~]# cat /etc/filebeat/modules.d/nginx.yml 
  2. # Module: nginx
  3. # Docs: https://www.elastic.co/guide/en/beats/filebeat/7.13/filebeat-module-nginx.html
  5. - module: nginx
  6.   # Access logs
  7.   access:
  8.     enabled: true
  10.     # Set custom paths for the log files. If left empty,
  11.     # Filebeat will choose the paths depending on your OS.
  12.     var.paths:
  13.       - "/var/log/test/*.log"
  15.   # Error logs
  16.   error:
  17.     enabled: true
  19.     # Set custom paths for the log files. If left empty,
  20.     # Filebeat will choose the paths depending on your OS.
  21.     #var.paths:
  23.   # Ingress-nginx controller logs. This is disabled by default. It could be used in Kubernetes environments to parse ingress-nginx logs
  24.   ingress_controller:
  25.     enabled: false
  27.     # Set custom paths for the log files. If left empty,
  28.     # Filebeat will choose the paths depending on your OS.
  29.     #var.paths:

3.3 查看模块输出

filebeat -e 可以查看扩展模块的输出信息

  1. [root@gitlab modules.d]# filebeat -e
  2. 2021-07-28T10:43:45.205+0800    INFO    instance/beat.go:665    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
  3. 2021-07-28T10:43:45.207+0800    INFO    instance/beat.go:673    Beat ID: 62e4bec4-db87-45cc-9be8-1cceeaa98e00
  4. 2021-07-28T10:43:45.209+0800    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:101    add_cloud_metadata: hosting provider type not detected.
  5. 2021-07-28T10:43:45.233+0800    INFO    [seccomp]    seccomp/seccomp.go:124    Syscall filter successfully installed
  6. 2021-07-28T10:43:45.233+0800    INFO    [beat]    instance/beat.go:1014    Beat info    {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "62e4bec4-db87-45cc-9be8-1cceeaa98e00"}}}
  7. 2021-07-28T10:43:45.233+0800    INFO    [beat]    instance/beat.go:1023    Build info    {"system_info": {"build": {"commit": "1907c246c8b0d23ae4027699c44bf3fbef57f4a4", "libbeat": "7.13.4", "time": "2021-07-14T18:42:41.000Z", "version": "7.13.4"}}}
  8. 2021-07-28T10:43:45.233+0800    INFO    [beat]    instance/beat.go:1026    Go runtime info    {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.15.14"}}}
  9. 2021-07-28T10:43:45.233+0800    INFO    [beat]    instance/beat.go:1030    Host info    {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-07-26T07:39:52+08:00","containerized":false,"name":"gitlab.xsc.org","ip":["127.0.0.1/8","::1/128","10.0.0.253/24","fe80::60b4:786d:82a1:67c5/64","192.168.235.253/24","fe80::1dc7:a766:ef32:5e31/64","172.17.0.1/16","172.18.0.1/16","fe80::42:c4ff:fe99:ea06/64"],"kernel_version":"3.10.0-957.el7.x86_64","mac":["00:0c:29:9d:f9:ae","00:0c:29:9d:f9:b8","02:42:a0:f8:8b:f7","02:42:c4:99:ea:06"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":6,"patch":1810,"codename":"Core"},"timezone":"CST","timezone_offset_sec":28800,"id":"c3a242bc4cc14d5cb4bff8d8c65e27e6"}}}
  10. 2021-07-28T10:43:45.234+0800    INFO    [beat]    instance/beat.go:1059    Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/etc/filebeat/modules.d", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 50432, "ppid": 125280, "seccomp": {"mode":"filter"}, "start_time": "2021-07-28T10:43:44.940+0800"}}}
  11. 2021-07-28T10:43:45.234+0800    INFO    instance/beat.go:309    Setup Beat: filebeat; Version: 7.13.4
  12. 2021-07-28T10:43:45.234+0800    INFO    [publisher]    pipeline/module.go:113    Beat name: gitlab.xsc.org
  13. 2021-07-28T10:43:45.234+0800    WARN    beater/filebeat.go:178    Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
  14. 2021-07-28T10:43:45.235+0800    INFO    [monitoring]    log/log.go:117    Starting metrics logging every 30s
  15. 2021-07-28T10:43:45.235+0800    INFO    instance/beat.go:473    filebeat start running.
  16. 2021-07-28T10:43:45.235+0800    INFO    memlog/store.go:119    Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=0
  17. 2021-07-28T10:43:45.235+0800    INFO    memlog/store.go:124    Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=0
  18. 2021-07-28T10:43:45.235+0800    WARN    beater/filebeat.go:381    Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
  19. 2021-07-28T10:43:45.235+0800    INFO    [registrar]    registrar/registrar.go:109    States Loaded from registrar: 0
  20. 2021-07-28T10:43:45.235+0800    INFO    [crawler]    beater/crawler.go:71    Loading Inputs: 1
  21. 2021-07-28T10:43:45.236+0800    INFO    log/input.go:157    Configured paths: [/test/*.log]
  22. 2021-07-28T10:43:45.236+0800    INFO    [crawler]    beater/crawler.go:141    Starting input (ID: 12725964982972559051)
  23. 2021-07-28T10:43:45.237+0800    INFO    log/harvester.go:302    Harvester started for file: /test/test.log
  24. 2021-07-28T10:43:45.237+0800    INFO    log/input.go:157    Configured paths: [/var/log/test/*.log]
  25. 2021-07-28T10:43:45.237+0800    INFO    log/input.go:157    Configured paths: [/var/log/nginx/error.log*]
  26. 2021-07-28T10:43:45.237+0800    INFO    [crawler]    beater/crawler.go:108    Loading and starting Inputs completed. Enabled inputs: 1
  27. 2021-07-28T10:43:45.237+0800    INFO    cfgfile/reload.go:164    Config reloader started
  28. 2021-07-28T10:43:45.238+0800    INFO    log/input.go:157    Configured paths: [/var/log/test/*.log]
  29. 2021-07-28T10:43:45.238+0800    INFO    log/input.go:157    Configured paths: [/var/log/nginx/error.log*]
  30. 2021-07-28T10:43:45.238+0800    INFO    cfgfile/reload.go:224    Loading of config files completed.
  31. 2021-07-28T10:43:45.239+0800    INFO    log/harvester.go:302    Harvester started for file: /var/log/test/test.log
  32. {
  33.   "@timestamp": "2021-07-28T02:43:45.237Z",
  34.   "@metadata": {
  35.     "beat": "filebeat",
  36.     "type": "_doc",
  37.     "version": "7.13.4"
  38.   },
  39.   "host": {
  40.     "os": {
  41.       "type": "linux",
  42.       "platform": "centos",
  43.       "version": "7 (Core)",
  44.       "family": "redhat",
  45.       "name": "CentOS Linux",
  46.       "kernel": "3.10.0-957.el7.x86_64",
  47.       "codename": "Core"
  48.     },
  49.     "id": "c3a242bc4cc14d5cb4bff8d8c65e27e6",
  50.     "containerized": false,
  51.     "ip": [
  52.       "10.0.0.253",
  53.       "fe80::60b4:786d:82a1:67c5",
  54.       "192.168.235.253",
  55.       "fe80::1dc7:a766:ef32:5e31",
  56.       "172.17.0.1",
  57.       "172.18.0.1",
  58.       "fe80::42:c4ff:fe99:ea06"
  59.     ],
  60.     "mac": [
  61.       "00:0c:29:9d:f9:ae",
  62.       "00:0c:29:9d:f9:b8",
  63.       "02:42:a0:f8:8b:f7",
  64.       "02:42:c4:99:ea:06"
  65.     ],
  66.     "hostname": "gitlab.xsc.org",
  67.     "architecture": "x86_64",
  68.     "name": "gitlab.xsc.org"
  69.   },
  70.   "log": {
  71.     "offset": 0,
  72.     "file": {
  73.       "path": "/test/test.log"
  74.     }
  75.   },
  76.   "message": "Hello  xiang shi chuan,This is filebeat test logs",
  77.   "input": {
  78.     "type": "log"
  79.   },
  80.   "agent": {
  81.     "version": "7.13.4",
  82.     "hostname": "gitlab.xsc.org",
  83.     "ephemeral_id": "b4277349-4c3e-4c28-85e3-5a2688011bdb",
  84.     "id": "62e4bec4-db87-45cc-9be8-1cceeaa98e00",
  85.     "name": "gitlab.xsc.org",
  86.     "type": "filebeat"
  87.   },
  88.   "ecs": {
  89.     "version": "1.8.0"
  90.   }
  91. }
  92. {
  93.   "@timestamp": "2021-07-28T02:43:45.237Z",
  94.   "@metadata": {
  95.     "beat": "filebeat",
  96.     "type": "_doc",
  97.     "version": "7.13.4"
  98.   },
  99.   "log": {
  100.     "file": {
  101.       "path": "/test/test.log"
  102.     },
  103.     "offset": 50
  104.   },
  105.   "message": "Hello This two line message",
  106.   "input": {
  107.     "type": "log"
  108.   },
  109.   "host": {
  110.     "name": "gitlab.xsc.org",
  111.     "id": "c3a242bc4cc14d5cb4bff8d8c65e27e6",
  112.     "containerized": false,
  113.     "ip": [
  114.       "10.0.0.253",
  115.       "fe80::60b4:786d:82a1:67c5",
  116.       "192.168.235.253",
  117.       "fe80::1dc7:a766:ef32:5e31",
  118.       "172.17.0.1",
  119.       "172.18.0.1",
  120.       "fe80::42:c4ff:fe99:ea06"
  121.     ],
  122.     "mac": [
  123.       "00:0c:29:9d:f9:ae",
  124.       "00:0c:29:9d:f9:b8",
  125.       "02:42:a0:f8:8b:f7",
  126.       "02:42:c4:99:ea:06"
  127.     ],
  128.     "hostname": "gitlab.xsc.org",
  129.     "architecture": "x86_64",
  130.     "os": {
  131.       "version": "7 (Core)",
  132.       "family": "redhat",
  133.       "name": "CentOS Linux",
  134.       "kernel": "3.10.0-957.el7.x86_64",
  135.       "codename": "Core",
  136.       "type": "linux",
  137.       "platform": "centos"
  138.     }
  139.   },
  140.   "agent": {
  141.     "type": "filebeat",
  142.     "version": "7.13.4",
  143.     "hostname": "gitlab.xsc.org",
  144.     "ephemeral_id": "b4277349-4c3e-4c28-85e3-5a2688011bdb",
  145.     "id": "62e4bec4-db87-45cc-9be8-1cceeaa98e00",
  146.     "name": "gitlab.xsc.org"
  147.   },
  148.   "ecs": {
  149.     "version": "1.8.0"
  150.   }
  151. }
  152. {
  153.   "@timestamp": "2021-07-28T02:43:45.239Z",
  154.   "@metadata": {
  155.     "beat": "filebeat",
  156.     "type": "_doc",
  157.     "version": "7.13.4",
  158.     "pipeline": "filebeat-7.13.4-nginx-access-pipeline"
  159.   },
  160.   "service": {
  161.     "type": "nginx"
  162.   },
  163.   "agent": {
  164.     "type": "filebeat",
  165.     "version": "7.13.4",
  166.     "hostname": "gitlab.xsc.org",
  167.     "ephemeral_id": "b4277349-4c3e-4c28-85e3-5a2688011bdb",
  168.     "id": "62e4bec4-db87-45cc-9be8-1cceeaa98e00",
  169.     "name": "gitlab.xsc.org"
  170.   },
  171.   "fileset": {
  172.     "name": "access"
  173.   },
  174.   "input": {
  175.     "type": "log"
  176.   },
  177.   "event": {
  178.     "module": "nginx",
  179.     "dataset": "nginx.access",
  180.     "timezone": "+08:00"
  181.   },
  182.   "ecs": {
  183.     "version": "1.9.0"
  184.   },
  185.   "host": {
  186.     "hostname": "gitlab.xsc.org",
  187.     "architecture": "x86_64",
  188.     "os": {
  189.       "platform": "centos",
  190.       "version": "7 (Core)",
  191.       "family": "redhat",
  192.       "name": "CentOS Linux",
  193.       "kernel": "3.10.0-957.el7.x86_64",
  194.       "codename": "Core",
  195.       "type": "linux"
  196.     },
  197.     "id": "c3a242bc4cc14d5cb4bff8d8c65e27e6",
  198.     "containerized": false,
  199.     "ip": [
  200.       "10.0.0.253",
  201.       "fe80::60b4:786d:82a1:67c5",
  202.       "192.168.235.253",
  203.       "fe80::1dc7:a766:ef32:5e31",
  204.       "172.17.0.1",
  205.       "172.18.0.1",
  206.       "fe80::42:c4ff:fe99:ea06"
  207.     ],
  208.     "name": "gitlab.xsc.org",
  209.     "mac": [
  210.       "00:0c:29:9d:f9:ae",
  211.       "00:0c:29:9d:f9:b8",
  212.       "02:42:a0:f8:8b:f7",
  213.       "02:42:c4:99:ea:06"
  214.     ]
  215.   },
  216.   "log": {
  217.     "offset": 0,
  218.     "file": {
  219.       "path": "/var/log/test/test.log"
  220.     }
  221.   },
  222.   "message": "Nginx test1 message!!!!!!!!!"
  223. }
  224. {
  225.   "@timestamp": "2021-07-28T02:43:45.239Z",
  226.   "@metadata": {
  227.     "beat": "filebeat",
  228.     "type": "_doc",
  229.     "version": "7.13.4",
  230.     "pipeline": "filebeat-7.13.4-nginx-access-pipeline"
  231.   },
  232.   "log": {
  233.     "offset": 29,
  234.     "file": {
  235.       "path": "/var/log/test/test.log"
  236.     }
  237.   },
  238.   "message": "Nginx test2 message!!!!!!!!!",
  239.   "fileset": {
  240.     "name": "access"
  241.   },
  242.   "service": {
  243.     "type": "nginx"
  244.   },
  245.   "event": {
  246.     "module": "nginx",
  247.     "dataset": "nginx.access",
  248.     "timezone": "+08:00"
  249.   },
  250.   "input": {
  251.     "type": "log"
  252.   },
  253.   "ecs": {
  254.     "version": "1.9.0"
  255.   },
  256.   "host": {
  257.     "architecture": "x86_64",
  258.     "os": {
  259.       "platform": "centos",
  260.       "version": "7 (Core)",
  261.       "family": "redhat",
  262.       "name": "CentOS Linux",
  263.       "kernel": "3.10.0-957.el7.x86_64",
  264.       "codename": "Core",
  265.       "type": "linux"
  266.     },
  267.     "id": "c3a242bc4cc14d5cb4bff8d8c65e27e6",
  268.     "containerized": false,
  269.     "name": "gitlab.xsc.org",
  270.     "ip": [
  271.       "10.0.0.253",
  272.       "fe80::60b4:786d:82a1:67c5",
  273.       "192.168.235.253",
  274.       "fe80::1dc7:a766:ef32:5e31",
  275.       "172.17.0.1",
  276.       "172.18.0.1",
  277.       "fe80::42:c4ff:fe99:ea06"
  278.     ],
  279.     "mac": [
  280.       "00:0c:29:9d:f9:ae",
  281.       "00:0c:29:9d:f9:b8",
  282.       "02:42:a0:f8:8b:f7",
  283.       "02:42:c4:99:ea:06"
  284.     ],
  285.     "hostname": "gitlab.xsc.org"
  286.   },
  287.   "agent": {
  288.     "ephemeral_id": "b4277349-4c3e-4c28-85e3-5a2688011bdb",
  289.     "id": "62e4bec4-db87-45cc-9be8-1cceeaa98e00",
  290.     "name": "gitlab.xsc.org",
  291.     "type": "filebeat",
  292.     "version": "7.13.4",
  293.     "hostname": "gitlab.xsc.org"
  294.   }
  295. }

4 输出到Elasticsearch

配置output模块，指定es的地址和端口，indices字段可以进行自定义索引，默认的索引不方便我们查看

  1. # ---------------------------- Elasticsearch Output ----------------------------
  2. output.elasticsearch:
  3.   # Array of hosts to connect to.
  4.   hosts: ["10.0.0.51:9200"]
  5.   indices:
  6.     - index: "nginx_test_%{+yyyy.MM.dd}"