单节点单Broker部署

  1. ##############
  2. ## 解压
  3. tar -zxvf kafka_2.10-0.8.1.1.tgz
  5. ##############
  6. ## 关于zookeeper
  7. cd kafka_2.10-0.8.1.1
  9. vi config/zookeeper.properties 
  11. #修改ZooKeeper的数据目录
  12. dataDir=/usr/local/db/zookeeper
  14. #配置host.name和advertised.host.name为IP地址，防止通过Java客户端连接时解析为localhost
  15. host.name=192.168.209.121
  17. advertised.host.name=192.168.209.121
  19. # 后台启动zooKeeper
  20. ./zookeeper-server-start.sh  -daemon /usr/local/kafka_2.10-0.8.1.1/config/zookeeper.properties
  22. ##############
  23. ## 关于kafka
  25. vi config/server.properties 
  27. log.dirs=/usr/local/kafka/kafka-logs
  29. # 后台启动kafka
  30. ./kafka-server-start.sh  -daemon /usr/local/kafka_2.10-0.8.1.1/config/server.properties
  32. ##############
  33. ## 关于topic
  35. # 创建topic
  36. cd /usr/local/kafka_2.10-0.8.1.1/bin/
  37. ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
  38. Created topic "test".
  40. # 查看刚才的topic
  41. ./kafka-topics.sh --list --zookeeper localhost:2181
  42. test
  44. # 生产者发送消息
  45. ./kafka-console-producer.sh --broker-list localhost:9092 --topic test
  46. This is a message
  47. This is My mesage
  49. # 消费者接受消息
  50. ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
  51. This is a message
  52. This is My me

感谢以下链接：
https://blog.csdn.net/l1028386804/article/details/78348367
https://blog.csdn.net/HG_Harvey/article/details/79174104

  1. cat zookeeper.service 
  2. [Unit]
  3. Description=Zookeeper service
  4. After=network.target
  6. [Service]
  7. Type=simple
  8. Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/java/bin"
  9. User=zookeeper
  10. Group=zookeeper
  11. ExecStart=bin/zkServer.sh start
  12. ExecStop=bin/zkServer.sh stop
  13. Restart=on-failure
  14. WorkingDirectory=/usr/local/zookeeper
  16. [Install]
  17. WantedBy=multi-user.target
  18. #原文链接：https://blog.csdn.net/weixin_34284188/article/details/92926437
  21. cat kafka.service 
  22. [Unit]
  23. Description=Apache Kafka server (broker)
  24. After=network.target  zookeeper.service
  26. [Service]
  27. Type=simple
  28. Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/java/bin"
  29. User=kafka
  30. Group=kafka
  31. ExecStart=bin/kafka-server-start.sh config/server.properties
  32. ExecStop=bin/kafka-server-stop.sh
  33. Restart=on-failure
  34. WorkingDirectory=/usr/local/kafka
  36. [Install]
  37. WantedBy=multi-user.target
  38. #原文链接：https://blog.csdn.net/qq_37707251/article/details/104929254
  40. ## 
  41. ln -s /usr/local/kafka_2.12-2.7.0/ /usr/local/kafka
  42. ln -s /usr/local/apache-zookeeper-3.6.2-bin/ /usr/local/zookeeper
  44. systemctl enable zookeeper
  45. systemctl enable kafka
  47. id kafka || useradd -s /sbin/nologin -M kafka
  48. id zookeeper || useradd -s /sbin/nologin -M zookeeper
  50. chown -R zookeeper.zookeeper /usr/local/zookeeper
  51. chown -R kafka.kafka /usr/local/kafka

kafka相关术语

Broker：消息中间件处理节点；每个Kafka服务节点称之为一个Broker，一个Kafka集群由一个或多个Broker组成Topic：一类特定数据集合的统称；可类比DB中Table的概念；逻辑概念 Producer：消息的生产者，向Broker发送消息的客户端 Consumer：消息的消费者，向Broker读取消息的客户端 Consumer Group：每一个Consumer隶属于一个特定的Consumer Group，一条消息可以被不同Group中的Consumer消费，但同一Group内的消息只能被一个Consumer消费 Partition：是对Topic中所包含数据集的物理分区；物理概念 Replication：副本集；是Kafka高可用的一种保障机制

原文链接：https://blog.csdn.net/YHYR_YCY/article/details/85016112

kafka几个应用场景

五种场景：异步处理，应用解耦，流量削锋，日志采集和消息通讯 原文链接：https://blog.csdn.net/leehsiao/article/details/103589126