消息队列 - RocketMQ群集部署及web管理页面 - 《运维大世界》

date: 2020-06-07title: RocketMQ学习（3）——群集部署及web管理页面 #标题
tags: RocketMQ #标签
categories: MQ # 分类

date: 2020-06-07title: RocketMQ学习（3）——群集部署及web管理页面 #标题
tags: RocketMQ #标签
categories: MQ # 分类

新入职的公司，在项目中用到了rocketmq，自己之前没有接触过，所以花时间把它学习一下。以便日后遇到问题可以有思路去解决。
参考文档：

和大多数消息中间件类似，都包含了producer、consumer、broker等角色，还有一个角色是namesrv，关于其各个组件的功能，在之前的文章中有写到，这里就不在赘述了。

来总结下，整个Rocket MQ消息集群就是由NameSrv/Broker、Producer/Consumer组成的，为了更清晰的理解他们之间的关系，我们以一条完整的信息流转为例，来看看rocketmq消息系统是如何运转的，如下所示：

RocketMQ群集部署及web管理页面 - 图1

下面是一个公司生产环境中的实际架构：

RocketMQ群集部署及web管理页面 - 图2

在上面的架构中，部署了3个namesrv节点，broker采用2主2从的异步复制模式进行集群部署。

废话不多说，开搞。

环境准备

OS	主机名	IP	roles
Centos 7.5	rocketmq01	192.168.20.2	namesrv/master-broker-a
Centos 7.5	rocketmq02	192.168.20.3	namesrv/slave-broker-a
Centos 7.5	rocketmq03	192.168.20.4	namesrv/master-broker-b
Centos 7.5	rocketmq04	192.168.20.5	slave-broker-b

注：文中用到的所有软件包，均可在我的网盘链接中找到（提取码: ye9e）。
另外需要注意，只要没有特别说明，以下操作是需要在所有节点上执行的，So….，建议有需要的话自己开发一个脚本吧。

安装jdk

jdk版本须在1.8以上，（下载地址）自行选择相应的版本。我这里选择自己之前下载的jdk，可能版本和链接中的有些不一样。

[root@rocketmq01 ~]# rpm -qa | grep jdk | xargs rpm -e --nodeps    # 卸载系统自带jdk 
[root@rocketmq01 ~]# tar zxf jdk-8u211-linux-x64.tar.gz -C /usr/local/
[root@rocketmq01 ~]# tail -n4 /etc/profile    # 该文件末尾写入以下内容
export JAVA_HOME=/usr/local/jdk1.8.0_211
export JRE_HOME=/usr/local/jdk1.8.0_211/jre
export CLASSPATH=$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
[root@rocketmq01 ~]# source /etc/profile    # 重新加载该文件
[root@rocketmq01 ~]# java -version     # 检测java版本
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

安装maven

# 下载并配置环境变量
[root@rocketmq01 ~]# wget https://downloads.apache.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
[root@rocketmq01 ~]# tar zxf apache-maven-3.6.3-bin.tar.gz
[root@rocketmq01 ~]# mv apache-maven-3.6.3 /usr/local/maven
[root@rocketmq01 ~]# tail -n2 /etc/profile    # 在文件中写入以下内容
export MAVEN_HOME=/usr/local/maven
export PATH=$PATH:$MAVEN_HOME/bin
[root@rocketmq01 ~]# source /etc/profile    # 重新加载文件
[root@rocketmq01 ~]# mvn -v      # 可以查看到版本信息即可
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/maven
Java version: 1.8.0_211, vendor: Oracle Corporation, runtime: /usr/local/jdk1.8.0_211/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-862.el7.x86_64", arch: "amd64", family: "unix"

由于maven默认的仓库在国外，所以我们还要更改其仓库地址，如下：

[root@rocketmq01 ~]# vim /usr/local/maven/conf/settings.xml   # 修改此文件
       ....................#省略部分内容
      <url>http://my.repository.com/repo/path</url>
    </mirror>
     -->           #定位到该行（158行），写入以下内容
    <mirror>
      <id>aliyun</id>
      <mirrorOf>central</mirrorOf>
            <name>aliyun maven</name>
      <url>https://maven.aliyun.com/nexus/content/groups/public/</url>
    </mirror>
      </mirrors>     # 写在该行上面

至此，准备工作就完成了。

安装rocketmq

[root@rocketmq01 ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.7.0/rocketmq-all-4.7.0-source-release.zip
[root@rocketmq01 ~]# unzip rocketmq-all-4.7.0-source-release.zip
[root@rocketmq01 ~]# mv rocketmq-all-4.7.0-source-release /usr/local/rocketmq-4.7.0
[root@rocketmq01 ~]# cd /usr/local/rocketmq-4.7.0/
# 编译
[root@rocketmq01 rocketmq-4.7.0]# mvn -Prelease-all -DskipTests clean install -U
# 创建所需目录
[root@rocketmq01 rocketmq-4.7.0]# mkdir -p /usr/local/rocketmq-4.7.0/data/store{commitlog,consumequeue,index,checkpoint,abort}

修改配置文件

在修改配置文件之前，我这里先解释下配置文件中的含义，以便根据生产中实际情况来修改

[root@rocketmq01 conf]# pwd    # 确定当前目录
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf
[root@rocketmq01 conf]# tree -L 1
.
├── 2m-2s-async    # 多Master多Slave模式，异步复制模式
├── 2m-2s-sync      # 多Master多Slave模式，同步双写
├── 2m-noslave      # 多Master模式
├── broker.conf
├── dledger
├── logback_broker.xml
├── logback_namesrv.xml
├── logback_tools.xml
├── plain_acl.yml
└── tools.yml

在rocketmq中部署群集，根据brokerid来判断主从关系，brokerId=0表示master，非0表示slave。

主配置文件解释如下：

# 所属集群名字
brokerClusterName=Rocketmq-Cluster
# broker名字，注意此处不同的配置文件填写的不一样
brokerName=broker-a
# 0表示Master  >0表示Slave
brokerId=0
# 删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=48
brokerRole=ASYNC_MASTER
# 刷盘方式
# - ASYNC_FLUSH 异步刷盘
# - SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
# nameServer地址，分号分割
namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
# Broker 对外服务的监听端口
listenPort=10911
# 在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
# 是否允许 Broker 自动创建Topic
autoCreateTopicEnable=true
# 是否允许 Broker 自动创建订阅组
autoCreateSubscriptionGroup=true
# commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
# ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
# 检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
# 存储路径
storePathRootDir=/usr/local/rocketmq-4.7.0/data/store
# commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq-4.7.0/data/store/commitlog
# 消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq-4.7.0/data/store/consumequeue
# 消息索引存储路径
storePathIndex=/usr/local/rocketmq-4.7.0/data/store/index
# checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq-4.7.0/data/store/checkpoint
# abort 文件存储路径
abortFile=/usr/local/rocketmq-4.7.0/data/store/abort
# 限制的消息大小
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
# 发消息线程池数量
sendMessageThreadPoolNums=128
# 拉消息线程池数量
pullMessageThreadPoolNums=128

接下来就去修改每个主机的配置文件，注意：每台主机要修改的配置文件都是不一样的。

rocketmq01主机配置文件

[root@rocketmq01 2m-2s-async]# pwd  # 确定当前目录
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async
[root@rocketmq01 2m-2s-async]# egrep -v '^$|^#' broker-a.properties 
brokerClusterName=Rocketmq-Cluster
brokerName=broker-a
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
listenPort=10911
defaultTopicQueueNums=4
autoCreateTopicEnable=false
autoCreateSubscriptionGroup=false
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq-4.7.0/data/store
storePathCommitLog=/usr/local/rocketmq-4.7.0/data/store/commitlog
storePathConsumeQueue=/usr/local/rocketmq-4.7.0/data/store/consumequeue
storePathIndex=/usr/local/rocketmq-4.7.0/data/store/index
storeCheckpoint=/usr/local/rocketmq-4.7.0/data/store/checkpoint
abortFile=/usr/local/rocketmq-4.7.0/data/store/abort
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

rocketmq02主机配置文件

[root@rocketmq02 2m-2s-async]# pwd
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async
[root@rocketmq02 2m-2s-async]# egrep -v '^$|^#' broker-a-s.properties 
brokerClusterName=Rocketmq-Cluster
brokerName=broker-a
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
listenPort=10911
defaultTopicQueueNums=4
autoCreateTopicEnable=false
autoCreateSubscriptionGroup=false
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq-4.7.0/data/store
storePathCommitLog=/usr/local/rocketmq-4.7.0/data/store/commitlog
storePathConsumeQueue=/usr/local/rocketmq-4.7.0/data/store/consumequeue
storePathIndex=/usr/local/rocketmq-4.7.0/data/store/index
storeCheckpoint=/usr/local/rocketmq-4.7.0/data/store/checkpoint
abortFile=/usr/local/rocketmq-4.7.0/data/store/abort
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

rocketmq03主机配置文件

[root@rocketmq03 2m-2s-async]# pwd
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async
[root@rocketmq03 2m-2s-async]# egrep -v '^$|^#' broker-b.properties 
brokerClusterName=Rocketmq-Cluster
brokerName=broker-b
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
listenPort=10911
defaultTopicQueueNums=4
autoCreateTopicEnable=false
autoCreateSubscriptionGroup=false
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq-4.7.0/data/store
storePathCommitLog=/usr/local/rocketmq-4.7.0/data/store/commitlog
storePathConsumeQueue=/usr/local/rocketmq-4.7.0/data/store/consumequeue
storePathIndex=/usr/local/rocketmq-4.7.0/data/store/index
storeCheckpoint=/usr/local/rocketmq-4.7.0/data/store/checkpoint
abortFile=/usr/local/rocketmq-4.7.0/data/store/abort
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

rocketmq04主机配置文件

[root@rocketmq04 2m-2s-async]# pwd
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async
[root@rocketmq04 2m-2s-async]# egrep -v '^$|^#' broker-b-s.properties 
brokerClusterName=Rocketmq-Cluster
brokerName=broker-b
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
listenPort=10911
defaultTopicQueueNums=4
autoCreateTopicEnable=false
autoCreateSubscriptionGroup=false
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq-4.7.0/data/store
storePathCommitLog=/usr/local/rocketmq-4.7.0/data/store/commitlog
storePathConsumeQueue=/usr/local/rocketmq-4.7.0/data/store/consumequeue
storePathIndex=/usr/local/rocketmq-4.7.0/data/store/index
storeCheckpoint=/usr/local/rocketmq-4.7.0/data/store/checkpoint
abortFile=/usr/local/rocketmq-4.7.0/data/store/abort
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128

修改java虚拟机默认参数

这个步骤根据实际情况，选择性的配置即可，如果你的机器内存较小，建议修改，否则会因内存分配失败而无法启动broker。

修改runser.sh文件

[root@rocketmq01 rocketmq-4.7.0]# pwd
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0
[root@rocketmq01 rocketmq-4.7.0]# vim bin/runserver.sh    # 修改此文件如下配置
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx1g -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
# Xms：jvm虚拟机最小可用内存大小
# Xmx：jvm虚拟机最大可用内存大小
# Xmn：年轻代内存大小

修改runbroker.sh文件

[root@rocketmq01 rocketmq-4.7.0]# pwd
/usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0
[root@rocketmq01 rocketmq-4.7.0]# vim bin/runbroker.sh       # 修改此文件内容如下
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx1g -Xmn256m"

启动namesrv

[root@rocketmq01 rocketmq-4.7.0]# nohup sh bin/mqnamesrv &
[root@rocketmq03 rocketmq-4.7.0]# ss -lnpt | grep 9876   # 确定端口在监听
LISTEN     0      128         :::9876                    :::*                   users:(("java",pid=58985,fd=68))
# 如果端口没有正常监听，请从日志或当前目录下的nohup.out文件中找问题
[root@rocketmq01 rocketmq-4.7.0]# cat ~/logs/rocketmqlogs/namesrv.log   # 日志文件

启动broker集群

注意：每台主机启动时指定的配置文件都是不一样的。

# 启动master broker-a服务（主机rocketmq01）
[root@rocketmq01 rocketmq-4.7.0]# nohup sh bin/mqbroker -c /usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async/broker-a.properties &>~/logs/rocketmqlogs/br.log &
# 启动slave broker-a-s服务（主机rocketmq02）
[root@rocketmq02 rocketmq-4.7.0]# nohup sh bin/mqbroker -c /usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async/broker-a-s.properties &>~/logs/rocketmqlogs/br.log &
# 启动master broker-b服务（主机rocketmq03）
[root@rocketmq03 rocketmq-4.7.0]# nohup sh bin/mqbroker -c /usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async/broker-b.properties &>~/logs/rocketmqlogs/br.log &
#启动Slave broker-b-s服务（主机rocketmq04）
[root@rocketmq04 rocketmq-4.7.0]# nohup sh bin/mqbroker -c /usr/local/rocketmq-4.7.0/distribution/target/rocketmq-4.7.0/rocketmq-4.7.0/conf/2m-2s-async/broker-b-s.properties &>~/br.log &

查看群集消息，确定无误

[root@rocketmq01 rocketmq-4.7.0]# sh bin/mqadmin clusterList  -n 192.168.20.2:9876   # 指定任意一个namesrv节点都可查看到参与集群的节点
RocketMQLog:WARN No appenders could be found for logger (io.netty.util.internal.PlatformDependent0).
RocketMQLog:WARN Please initialize the logger system properly.
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
Rocketmq-Cluster  broker-a                0     192.168.20.2:10911     V4_7_0                   0.00(0,0ms)         0.00(0,0ms)          0 440721.12 -1.0000
Rocketmq-Cluster  broker-a                1     192.168.20.3:10911     V4_7_0                   0.00(0,0ms)         0.00(0,0ms)          0 440721.12 0.0940
Rocketmq-Cluster  broker-b                0     192.168.20.4:10911     V4_7_0                   0.00(0,0ms)         0.00(0,0ms)          0 440721.12 -1.0000
Rocketmq-Cluster  broker-b                1     192.168.20.5:10911     V4_7_0                   0.00(0,0ms)         0.00(0,0ms)          0 440721.12 0.0940

部署rocketmq的web管理界面

这个web界面的部署在任意一台有java和maven环境的主机上部署即可。

在我以这种方式部署完管理界面后，又发现了一个相比较要省事的github项目，有兴趣可以去看一下：github地址:rocketmq-console。

[root@rocketmq01 src]# git clone -b release-rocketmq-console-1.0.0 https://github.com/apache/rocketmq-externals.git
[root@rocketmq01 src]# cd rocketmq-externals/rocketmq-console/src/main/resources/
[root@rocketmq01 resources]# egrep -v '^$|^#' application.properties       # 修改配置文件
server.contextPath=
server.port=8080       # 自定义监听端口
spring.application.name=rocketmq-console
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
logging.config=classpath:logback.xml
# 指定namesrv的监听端口
rocketmq.config.namesrvAddr=192.168.20.2:9876;192.168.20.3:9876;192.168.20.4:9876
rocketmq.config.isVIPChannel=
rocketmq.config.dataPath=/tmp/rocketmq-console/data
rocketmq.config.enableDashBoardCollect=true
[root@rocketmq01 rocketmq-console]# pwd      # 确定当前目录
/usr/src/rocketmq-externals/rocketmq-console
[root@rocketmq01 rocketmq-console]# mvn clean package -Dmaven.test.skip=true      # 进行编译

启动web管理界面

[root@rocketmq01 rocketmq-console]# pwd
/usr/src/rocketmq-externals/rocketmq-console
[root@rocketmq01 rocketmq-console]# cp -a target/rocketmq-console-ng-1.0.0.jar /usr/local/rocketmq-4.7.0/
[root@rocketmq01 rocketmq-console]# cd /usr/local/rocketmq-4.7.0/
[root@rocketmq01 rocketmq-4.7.0]# nohup java -jar rocketmq-console-ng-1.0.0.jar &> rocket_console.log &
[root@rocketmq01 rocketmq-4.7.0]# ss -lnpt | grep 8080
LISTEN     0      100         :::8080                    :::*                   users:(("java",pid=61037,fd=19))

浏览器访问8080端口测试：

RocketMQ群集部署及web管理页面 - 图3

可以自行添加代理服务器，如nginx来进行访问认证。

ok了，至此结束。

RocketMQ群集部署及web管理页面

date: 2020-06-07title: RocketMQ学习（3）——群集部署及web管理页面 #标题tags: RocketMQ #标签categories: MQ # 分类