在机器上部署 blackbox_exporter

  1. - 项目地址 https://github.com/prometheus/blackbox_exporter


使用ansible部署 blackbox_exporter

  1. ansible-playbook -i host_file  service_deploy.yaml  -e "tgz=blackbox_exporter-0.18.0.linux-amd64.tar.gz" -e "app=blackbox_exporter"

访问页面

  1. http://ip:9115/

image.png

http trace中对于http各个状态的描述

  1. - dns解析时间:              DNSDone-DNSStart
  2. - tls握手时间:              gotConn - DNSDone
  3. - tls connect连接时间:      connectDone - DNSDone
  4. - 非tls connect连接时间:    gotConn - DNSDone
  7. - processing 服务端处理时间: responseStart - gotConn
  8. - transfer 数据传输时间:     end - responseStart
  1. trace := &httptrace.ClientTrace{
  2.     DNSStart:             tt.DNSStart,
  3.     DNSDone:              tt.DNSDone,
  4.     ConnectStart:         tt.ConnectStart,
  5.     ConnectDone:          tt.ConnectDone,
  6.     GotConn:              tt.GotConn,
  7.     GotFirstResponseByte: tt.GotFirstResponseByte,
  8. }


blackbox_exporter 需要传入target 和 module 参数，采用下列方式加入的采集池中 

  1.   - job_name: 'blackbox-http'
  3.     metrics_path: /probe
  4.     params:
  5.       module: [http_2xx]  # Look for a HTTP 200 response.
  6.     static_configs:
  7.       - targets:
  8.         - http://prometheus.io    # Target to probe with http.
  9.         - https://www.baidu.com   # Target to probe with https.
  10.         - http://172.16.58.78:3000 # Target to probe with http on port 3000.
  11.     relabel_configs:
  12.       - source_labels: [__address__]
  13.         target_label: __param_target
  14.       - source_labels: [__param_target]
  15.         target_label: instance
  16.       - target_label: __address__
  17.         replacement: 172.16.58.78:9115  # The blackbox exporter's real hostname:port.

会发现如此配置之后 实例数据只有blackbox_exporter的地址 而没有target的地址

  1. probe_duration_seconds{instance="172.20.70.205:9115", job="blackbox-http"}

blackbox_exporter 采集加入的采集池中

  3.   - job_name: 'blackbox-ssh'
  4.     metrics_path: /probe
  5.     params:
  6.       module: [ssh_banner]  # Look for a HTTP 200 response.
  7.     static_configs:
  8.       - targets:
  9.         - 172.16.58.78:22   # Target to probe with http.
  10.         - 172.16.58.79:22   # Target to probe with https.
  11.     relabel_configs:
  12.       - source_labels: [__address__]
  13.         target_label: __param_target
  14.       - source_labels: [__param_target]
  15.         target_label: instance
  16.       - target_label: __address__
  17.         replacement: 172.16.58.78:9115  # The blackbox exporter's real hostname:port.
  21.   - job_name: 'blackbox-ping'
  22.     metrics_path: /probe
  23.     params:
  24.       module: [icmp]  # Look for a HTTP 200 response.
  25.     static_configs:
  26.       - targets:
  27.         - 172.16.58.78   # Target to probe with http.
  28.         - 172.16.58.79   # Target to probe with https.
  29.         - 8.8.8.8
  30.     relabel_configs:
  31.       - source_labels: [__address__]
  32.         target_label: __param_target
  33.       - source_labels: [__param_target]
  34.         target_label: instance
  35.       - target_label: __address__
  36.         replacement: 172.16.58.78:9115  # The blackbox exporter's real hostname:port.

grafana 上导入 blackbox_exporter dashboard

  1. # 模块使用 ssh_banner 探测172.20.70.215:22
  2. http://172.20.70.205:9115/probe?module=ssh_banner&target=172.20.70.215:22
  4. # 结果解读
  6. # HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds
  7. # TYPE probe_dns_lookup_time_seconds gauge
  8. probe_dns_lookup_time_seconds 2.5331e-05
  9. # HELP probe_duration_seconds Returns how long the probe took to complete in seconds
  10. # TYPE probe_duration_seconds gauge
  11. probe_duration_seconds 0.02228226
  12. # HELP probe_failed_due_to_regex Indicates if probe failed due to regex
  13. # TYPE probe_failed_due_to_regex gauge
  14. probe_failed_due_to_regex 0
  15. # HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes.
  16. # TYPE probe_ip_addr_hash gauge
  17. probe_ip_addr_hash 9.51584696e+08
  18. # HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6
  19. # TYPE probe_ip_protocol gauge
  20. probe_ip_protocol 4
  21. # HELP probe_success Displays whether or not the probe was a success
  22. # TYPE probe_success gauge
  23. probe_success 1
  25. # ssh_banner 模块解读
  26. # 使用tcp进行探测，并且 期望得到 SSH-2.0-的响应
  27.   ssh_banner:
  28.     prober: tcp
  29.     tcp:
  30.       query_response:
  31.       - expect: "^SSH-2.0-"
  33. # 和telnet结果一致
  34. [root@prome_master_01 blackbox_exporter]# telnet 172.20.70.215 22
  35. Trying 172.20.70.215...
  36. Connected to 172.20.70.215.
  37. Escape character is '^]'.
  38. SSH-2.0-OpenSSH_7.4
  40. Protocol mismatch.
  41. Connection closed by foreign host.
  2.   - job_name: 'blackbox-ssh'
  3.     metrics_path: /probe
  4.     params:
  5.       module: [ssh_banner]  # Look for a HTTP 200 response.
  6.     static_configs:
  7.       - targets:
  8.         - 172.16.58.78:22   # Target to probe with http.
  9.         - 172.16.58.79:22   # Target to probe with https.
  10.     relabel_configs:
  11.       - source_labels: [__address__]
  12.         target_label: __param_target
  13.       - source_labels: [__param_target]
  14.         target_label: instance
  15.       - target_label: __address__
  16.         replacement: 172.16.58.78:9115  # The blackbox exporter's real hostname:port.

进行ping探测

  1. http://172.20.70.205:9115/probe?module=icmp&target=www.baidu.com
  3. # 结果解读
  4. # HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds
  5. # TYPE probe_dns_lookup_time_seconds gauge
  6. probe_dns_lookup_time_seconds 0.195704171
  7. # HELP probe_duration_seconds Returns how long the probe took to complete in seconds
  8. # TYPE probe_duration_seconds gauge
  9. probe_duration_seconds 0.378563375
  10. # HELP probe_icmp_duration_seconds Duration of icmp request by phase
  11. # TYPE probe_icmp_duration_seconds gauge
  12. probe_icmp_duration_seconds{phase="resolve"} 0.195704171
  13. probe_icmp_duration_seconds{phase="rtt"} 0.182456226
  14. probe_icmp_duration_seconds{phase="setup"} 0.000145827
  15. # HELP probe_icmp_reply_hop_limit Replied packet hop limit (TTL for ipv4)
  16. # TYPE probe_icmp_reply_hop_limit gauge
  17. probe_icmp_reply_hop_limit 49
  18. # HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes.
  19. # TYPE probe_ip_addr_hash gauge
  20. probe_ip_addr_hash 2.282787449e+09
  21. # HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6
  22. # TYPE probe_ip_protocol gauge
  23. probe_ip_protocol 4
  24. # HELP probe_success Displays whether or not the probe was a success
  25. # TYPE probe_success gauge
  26. probe_success 1
  42. - job_name: 'blackbox-ping'
  43.     metrics_path: /probe
  44.     params:
  45.       module: [icmp]  # Look for a HTTP 200 response.
  46.     static_configs:
  47.       - targets:
  48.         - 172.16.58.78   # Target to probe with http.
  49.         - 172.16.58.79   # Target to probe with https.
  50.         - 8.8.8.8
  51.     relabel_configs:
  52.       - source_labels: [__address__]
  53.         target_label: __param_target
  54.       - source_labels: [__param_target]
  55.         target_label: instance
  56.       - target_label: __address__
  57.         replacement: 172.16.58.78:9115  # The blackbox exporter's real hostname:port.
  64. ssh探测过程说明
  65. prometheus --> blackbox_exporter 使用配置 http://192.168.0.112:9115/probe?module=ssh_banner&target=192.168.0.127%3A22  --> 192.168.0.127:22