批量注册exporter到consul

date: 2021-01-11title: 批量注册exporter到consul并配置Prometheus自动发现 #标题
tags: 自动发现 #标签
categories: 监控 # 分类

此文章用于记录下将各种exporter批量注册到consul中的脚本，以及Prometheus增加相关job，实现自动发现。

注册node_exporter到consul集群

$ cat registry.sh    # 脚本内容如下
#!/usr/bin/env bash
while read host_name host_addr
do
    curl -X PUT -d '{"id": "'${host_name}'","name": "node-exporter","address": "'${host_addr}'","port":9100,"tags": ["node-exporter"],"checks": [{"http": "http://'${host_addr}':9100/","interval": "5s"}]}' http://192.168.20.2:8500/v1/agent/service/register
done < /root/registry/hosts
$ cat hosts          # 脚本中读取的hosts文件如下
node-exporter-20-2 192.168.20.2
node-exporter-20-3 192.168.20.3
node-exporter-20-4 192.168.20.4

注册mysqld_exporter到consul集群

$ cat registry_mysql.sh         # 脚本文件如下
#!/usr/bin/env bash
while read mysql_name mysql_addr
do
    curl -X PUT -d '{"id": "'${mysql_name}'","name": "mysql-exporter","address": "'${mysql_addr}'","port":9104,"tags": ["mysql-exporter"],"checks": [{"http": "http://'${mysql_addr}':9104/","interval": "5s"}]}' http://192.168.20.2:8500/v1/agent/service/register
done < /root/registry/mysql_instance
$ cat mysql_instance      # 读取的文件如下
mysql-20-3 192.168.20.3
mysql-20-4 192.168.20.4

注册redis_exporter集群到consul集群

可参考博文：Prometheus监控redis集群 启动redis_exporter。

$ cat registry_redis.sh       # 先将redis-exporter注册到consul中
#!/usr/bin/env bash
while read redis_name redis_addr port
do
    curl -X PUT -d '{"id": "'${redis_name}'","name": "redis-exporter","address": "'${redis_addr}'","port":'${port}',"tags": ["redis-exporter"],"checks": [{"http": "http://'${redis_addr}':'${port}'/","interval": "5s"}]}' http://192.168.20.2:8500/v1/agent/service/register
done < /root/registry/redis_exporter
$ cat redis_exporter    # 以下写入各个redis-exporter
redis-exporter-20-2-9121 192.168.20.2 9121
$ cat registry_redis_instance.sh           # 再将各个redis节点注册到consul中
#!/usr/bin/env bash
while read redis_name redis_addr port
do
    curl -X PUT -d '{"id": "'${redis_name}'","name": "redis-instance","address": "'${redis_addr}'","port":'${port}',"tags": ["redis-instance"],"checks": [{"http": "http://'${redis_addr}':'${port}'/","interval": "5s"}]}' http://192.168.20.2:8500/v1/agent/service/register
done < /root/registry/redis_instance
$ cat redis_instance          # redis_instance 文件内容如下
redis-instance-20-2-7001 192.168.20.2 7001
redis-instance-20-2-7002 192.168.20.2 7002
redis-instance-20-3-7001 192.168.20.3 7001
redis-instance-20-3-7002 192.168.20.3 7002
redis-instance-20-4-7001 192.168.20.4 7001
redis-instance-20-4-7002 192.168.20.4 7002

注意：redis各个实例注册到consul中，健康检查都是失败的，不过没关系，并不影响Prometheus自动发现。

查看consul中注册的服务

可以看到如下注册的服务，就说明没有问题。

配置Prometheus基于consul实现自动发现

$ vim prometheus.yml     # prometheus.yml 配置文件中增加如下job
             # .......   省略部分内容
# 这里配置多个job的目的，就是为了在Prometheus的target列表中，可以针对不同的服务进行分组
  - job_name: node-exporter
    honor_labels: true
    metrics_path: /metrics
    scheme: http
    consul_sd_configs:
      - server: 192.168.20.2:8500
      - server: 192.168.20.3:8500
      - server: 192.168.20.4:8500
        services: []
    relabel_configs:
    - source_labels: ['__meta_consul_tags']
      target_label: 'product'
    - source_labels: ['__meta_consul_dc']
      target_label: 'idc'
    - source_labels: ['product']
      regex: ",node-exporter,"
      action: keep
  - job_name: mysqld-exporter
    honor_labels: true
    metrics_path: /metrics
    scheme: http
    consul_sd_configs:
      - server: 192.168.20.2:8500
      - server: 192.168.20.3:8500
      - server: 192.168.20.4:8500
        services: []
    relabel_configs:
    - source_labels: ['__meta_consul_tags']
      target_label: 'product'
    - source_labels: ['__meta_consul_dc']
      target_label: 'idc'
    - source_labels: ['product']
      regex: ",mysql-exporter,"
      action: keep
  - job_name: redis-exporter
    honor_labels: true
    metrics_path: /metrics
    scheme: http
    consul_sd_configs:
      - server: 192.168.20.2:8500
      - server: 192.168.20.3:8500
      - server: 192.168.20.4:8500
        services: []
    relabel_configs:
    - source_labels: ['__meta_consul_tags']
      target_label: 'product'
    - source_labels: ['__meta_consul_dc']
      target_label: 'idc'
    - source_labels: ['product','__meta_consul_service_port']
      regex: ",redis-exporter,;9121"
      action: keep
  - job_name: redis-instance
    honor_labels: true
    metrics_path: /metrics
    scheme: http
    consul_sd_configs:
      - server: 192.168.20.2:8500
      - server: 192.168.20.3:8500
      - server: 192.168.20.4:8500
        services: []
    relabel_configs:
    - source_labels: ['__address__']
      target_label: '__param_target'
    - source_labels: ['__param_target']
      target_label: 'instace'
    - target_label: '__address__'
      replacement: 192.168.20.2:9121
    - source_labels: ['__meta_consul_tags']
      target_label: 'product'
    - source_labels: ['__meta_consul_dc']
      target_label: 'idc'
    - source_labels: ['product']
      regex: ",redis-instance,"
      action: keep

确认Prometheus的target列表无误

如果访问Prometheus的target列表，可以看到所有注册到consul中的exporter，则表明配置无误。