实战环境
192.168.10.12  Logstash  Kafka  Filebeat
 
Kafka
1.      Kafka依赖于Zookeeper
2.      两个都依赖于Java
 
Kafka依赖于Zookeeper
ZooKeeper是一个分布式的,开放源码的分布式应用程序协调服务,是Google的Chubby一个开源的实现,是Hadoop和Hbase的重要组件。它是一个为分布式应用提供一致性服务的软件,提供的功能包括:配置维护、域名服务、分布式同步、组服务等。
ZooKeeper的目标就是封装好复杂易出错的关键服务,将简单易用的接口和性能高效、功能稳定的系统提供给用户。
ZooKeeper包含一个简单的原语集,提供Java和C的接口。
 
3.      官方网站:https://zookeeper.apache.org/
4.      下载ZK的二进制包
5.      解压到对应目录完成安装
 
ZK的安装命令
tar xvf zookeeper-3.4.13.tar.gz -C /usr/local
cp /usr/local/zookeeper-3.4.13/conf/zoo_sample.cfg /usr/local/zookeeper-3.4.13/conf/zoo.cfg
 
ZK的启动
6.      更改配置:clientPortAddress=0.0.0.0
[root@server12 ~]# vim /usr/local/zookeeper-3.4.13/conf/zoo.cfg
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPortAddress=0.0.0.0
clientPort=2181
# the maximum number of client connections.
7.      启动:/usr/local/zookeeper-3.4.13/bin/zkServer.sh start
    [root@server12 ~]# /usr/local/zookeeper-3.4.13/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.13/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
[root@server12 ~]# netstat -pantul | grep java
tcp6       0      0 127.0.0.1:9600          :::                    LISTEN      18372/java          
tcp6       0      0 :::2181                 :::                    LISTEN      20951/java          
tcp6       0      0 :::37190                :::                    LISTEN      20951/java          
tcp6       0      0 192.168.10.12:36038     192.168.10.11:9200      ESTABLISHED 18372/java          
tcp6       0      0 192.168.10.12:44542     192.168.10.12:6379      ESTABLISHED 18372/java          
tcp6       0      0 192.168.10.12:36036     192.168.10.11:9200      ESTABLISHED 18372/java
 
Kafka下载地址
8.      Kafka官网:http://kafka.apache.org/
9.      下载Kafka的二进制包
10.   解压到对应目录完成安装
 
Kafka的安装命令
tar xvf kafka_2.11-2.1.1.tgz
mv kafka_2.11-2.1.1 /usr/local/kafka_2.11
 
Kafka的启动
11.   更改kafka的配置:更改监听地址、更改连接zk的地址
[root@server12 ~]# vim /usr/local/kafka_2.11/config/server.properties
zookeeper.connect=192.168.10.12:2181
listeners=PLAINTEXT://192.168.10.12:9092
 
12.   前台启动:/usr/local/kafka_2.11/bin/kafka-server-start.sh /usr/local/kafka_2.11/config/server.properties
13.   启动kafka:nohup /usr/local/kafka_2.11/bin/kafka-server-start.sh /usr/local/kafka_2.11/config/server.properties >/tmp/kafka.log 2>&1 &
[root@server12 ~]# netstat -pantul | grep java
tcp6       0      0 :::44377                :::                    LISTEN      21604/java          
tcp6       0      0 127.0.0.1:9600          :::                    LISTEN      18372/java          
tcp6       0      0 192.168.10.12:9092      :::                    LISTEN      21604/java          
tcp6       0      0 :::2181                 :::                    LISTEN      20951/java          
tcp6       0      0 :::37190                :::                    LISTEN      20951/java          
tcp6       0      0 192.168.10.12:2181      192.168.10.12:33320     ESTABLISHED 20951/java          
tcp6       0      0 192.168.10.12:36038     192.168.10.11:9200      ESTABLISHED 18372/java          
tcp6       0      0 192.168.10.12:44542     192.168.10.12:6379      ESTABLISHED 18372/java          
tcp6       0      0 192.168.10.12:9092      192.168.10.12:58276     ESTABLISHED 21604/java          
tcp6       0      0 192.168.10.12:36036     192.168.10.11:9200      ESTABLISHED 18372/java          
tcp6       0      0 192.168.10.12:33320     192.168.10.12:2181      ESTABLISHED 21604/java          
tcp6       0      0 192.168.10.12:58276     192.168.10.12:9092      ESTABLISHED 21604/java
 
                         
                                

