资源规划

组件	bigdata-node1	bigdata-node2	bigdata-node3
OS	centos7.6	centos7.6	centos7.6
JDK	jvm	jvm	jvm
Kafka	kafka	kafka	kafka
Influxdb	N.A	Influxdb	N.A
JmxTrans	N.A	jmxtrans	N.A
Grafana	N.A	Grafana	N.A

案例规划

CentOS7_JMX-Kafka - 图1

开启Kafka JMX监控

修改Kafka各节点JMX启动配置，开启监控功能：

vi ~/modules/kafka_2.11-0.11.0.3/bin/kafka-server-start.sh

配置如下：

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then 
    export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70" 
    export JMX_PORT="9999" 
fi

分发脚本：

cd ~/modules/kafka_2.11-0.11.0.3/bin
scp -r kafka-server-start.sh vagrant@bigdata-node1:~/modules/kafka_2.11-0.11.0.3/bin/
scp -r kafka-server-start.sh vagrant@bigdata-node2:~/modules/kafka_2.11-0.11.0.3/bin/

分发完成后，请重新启动Kafka集群。

创建Influxdb JMX存储

-- 启动Influxdb
influx
-- 创建用户，并分配权限
create user "root" with password '123456' with all privileges;
create database zkdb;

创建JmxTrans JMX代理

指标配置：

cd /home/vagrant/modules/jmxtrans-270/bin
vi kafka1-jmxtrans.json

配置如下：

{
  "servers": [
    {
      "port": "9999",
      "host": "192.168.0.101",
      "alias": "kafka-1",
      "queries": [
        {
          "obj": "java.lang:type=Memory",
          "attr": [
            "HeapMemoryUsage",
            "NonHeapMemoryUsage"
          ],
          "resultAlias": "jvmMemory",
          "outputWriters": [
            {
              "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory",
              "url": "http://192.168.0.102:8086/",
              "username": "root",
              "password": "123456",
              "database": "jmxDB",
              "tags": {
                "application": "kafka"
              }
            }
          ]
        }
      ]
    }
  ]
}

启动JMX代理：

cd /home/vagrant/modules/jmxtrans-270/
# 脚本赋权
sudo chmod +x ./bin/jmxtrans.sh
# 加载所有*.json
JAR_FILE=./lib/jmxtrans-all.jar ./bin/jmxtrans.sh start
# 指定加载某类型的json
JAR_FILE=./lib/jmxtrans-all.jar ./bin/jmxtrans.sh start ./bin/kafka*-jmxtrans.json
# 指定文件方式启动
bin/jmxtrans.sh start ./bin/kafka1-jmxtrans.json
bin/jmxtrans.sh stop
bin/jmxtrans.sh status

创建Grafana图表

配置数据源：

创建dashboard：

参考

Kafka监控参考：https://softwaremill.com/monitoring-apache-kafka-with-influxdb-grafana/
kafka监控实战(jmxtrans+InfluxDb+Grafana)：https://www.jianshu.com/p/ac30f06d433f
JSON：kafka.json kafka-jmxtrans.json