rocketmq部署过程记录

时间:2022-07-22
本文章向大家介绍rocketmq部署过程记录，主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。

这里使用了两台虚拟机，部署的是多master和多slave的异步复制模式，部署结构为：

关于jdk和maven的部署安装这里就不再写了，直接去下载二进制包，然后设置好环境变量即可

这里使用的rocketmq的包是从官方下载的二进制包，版本为4.3.2，可以直接Google rocketmq downlaod来选择相应的版本来下载

配置部署

1）下载并解压

wget http://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.3.2/rocketmq-all-4.3.2-bin-release.zip
unzip rocketmq-all-4.3.2-bin-release.zip
把rocketmq的包mv到/usr/local下并改名为rocketmq

2）修改配置文件

192.168.152.146机器：
# tree 2m-2s-async/
2m-2s-async/
├── broker-a.properties
├── broker-a-s.properties
├── broker-b.properties
└── broker-b-s.properties

分别修改broker-a.properties和broker-b-s.properties文件
# vim broker-a.properties
brokerClusterName=rocketmq-cluster 
brokerName=broker-a 
brokerId=0 
namesrvAddr=127.0.0.1:9876;192.168.152.147:9876
defaultTopicQueueNums=4 
autoCreateTopicEnable=true 
autoCreateSubscriptionGroup=true 
listenPort=10910 
deleteWhen=04 
fileReservedTime=120 
mapedFileSizeCommitLog=1073741824 
mapedFileSizeConsumeQueue=300000 
diskMaxUsedSpaceRatio=88 
storePathRootDir=/usr/local/rocketmq/store/broker-a 
storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog 
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue 
storePathIndex=/usr/local/rocketmq/store/broker-a/index 
storeCheckpoint=/usr/local/rocketmq/store/checkpoint 
abortFile=/usr/local/rocketmq/store/abort 
maxMessageSize=65536 
brokerRole=ASYNC_MASTER 
flushDiskType=ASYNC_FLUSH 

# vim broker-b-s.properties
brokerClusterName=rocketmq-cluster 
brokerName=broker-b 
brokerId=1 
namesrvAddr=127.0.0.1:9876;192.168.152.147:9876
defaultTopicQueueNums=4 
autoCreateTopicEnable=true 
autoCreateSubscriptionGroup=true 
listenPort=10940
deleteWhen=04 
fileReservedTime=120 
mapedFileSizeCommitLog=1073741824 
mapedFileSizeConsumeQueue=300000 
diskMaxUsedSpaceRatio=88 
storePathRootDir=/usr/local/rocketmq/store/broker-b-s 
storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog 
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue 
storePathIndex=/usr/local/rocketmq/store/broker-b-s/index 
storeCheckpoint=/usr/local/rocketmq/store/checkpoint 
abortFile=/usr/local/rocketmq/store/abort 
maxMessageSize=65536 
brokerRole=SLAVE 
flushDiskType=ASYNC_FLUSH
192.168.152.147：
# vim broker-a-s.properties
brokerClusterName=rocketmq-cluster 
brokerName=broker-b
brokerId=0 
namesrvAddr=127.0.0.1:9876;192.168.152.146:9876
defaultTopicQueueNums=4 
autoCreateTopicEnable=true 
autoCreateSubscriptionGroup=true 
listenPort=10930
deleteWhen=04 
fileReservedTime=120 
mapedFileSizeCommitLog=1073741824 
mapedFileSizeConsumeQueue=300000 
diskMaxUsedSpaceRatio=88 
storePathRootDir=/usr/local/rocketmq/store/broker-a 
storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog 
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue 
storePathIndex=/usr/local/rocketmq/store/broker-a/index 
storeCheckpoint=/usr/local/rocketmq/store/checkpoint 
abortFile=/usr/local/rocketmq/store/abort 
maxMessageSize=65536 
brokerRole=ASYNC_MASTER 
flushDiskType=ASYNC_FLUSH 

# vim broker-b.properties
brokerClusterName=rocketmq-cluster 
brokerName=broker-a
brokerId=1 
namesrvAddr=127.0.0.1:9876;192.168.152.146:9876
defaultTopicQueueNums=4 
autoCreateTopicEnable=true 
autoCreateSubscriptionGroup=true 
listenPort=10920
deleteWhen=04 
fileReservedTime=120 
mapedFileSizeCommitLog=1073741824 
mapedFileSizeConsumeQueue=300000 
diskMaxUsedSpaceRatio=88 
storePathRootDir=/usr/local/rocketmq/store/broker-b-s 
storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog 
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue 
storePathIndex=/usr/local/rocketmq/store/broker-b-s/index 
storeCheckpoint=/usr/local/rocketmq/store/checkpoint 
abortFile=/usr/local/rocketmq/store/abort 
maxMessageSize=65536 
brokerRole=SLAVE 
flushDiskType=ASYNC_FLUSH

以上是192.168.152.146和192.168.152.147机器上的操作，需要注意的有三个地方：

  • listenPort在broker之间的端口号之差要大于2
  • namesrvAddr在配置时，如果本机有nameserver服务，则需要在最前面写127.0.0.1的地址
  • brokerId和brokerName要对应正确，具体可看文章最后的配置参考

设置无误后，就可以启动了，但是在启动之前我们需要修改下启动脚本

3）修改启动脚本

nameserver的启动脚本为bin/runserver.sh
broker的启动脚本为bin/runbroker.sh
需要修改这两个启动脚本的内存大小和jdk的目录
例如：
# cat runserver.sh 
#!/bin/sh

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
    echo "ERROR: $1 !!"
    exit 1
}
JAVA_HOME=/usr/local/jdk    #这里最好设置一下java的家目录
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn1g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"    #这里可以根据机器的内存大小来酌情修改
JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8  -XX:-UseParNewGC"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT}  -XX:-UseLargePages"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"

$JAVA ${JAVA_OPT} $@

4）创建相关目录

192.168.152.146：
mkdir /usr/local/rocketmq/store/broker-a/{nsumequeue,commitlog,index} -p
mkdir /usr/local/rocketmq/store/broker-b-s/{nsumequeue,commitlog,index} -p
mkdir /usr/local/rocketmq/logs

192.168.152.147：
mkdir /usr/local/rocketmq/store/broker-a-s/{nsumequeue,commitlog,index} -p
mkdir /usr/local/rocketmq/store/broker-b/{nsumequeue,commitlog,index} -p
mkdir /usr/local/rocketmq/logs

5）启动

首先启动两台机器的nameserver
nohup sh /usr/local/rocketmq/bin/mqnamesrv >/usr/local/rocketmq/logs/mqnamesrv.log 2>&1 &

然后分别启动两个节点的broker-master
192.168.152.146:
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties > /usr/local/rocketmq/logs/broker-a.log 2>&1 &
192.168.152.147:
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties > /usr/local/rocketmq/logs/broker-a-s.log 2>&1 &

最后启动两个节点的broker-slave
192.168.152.146:
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b-s.properties > /usr/local/rocketmq/logs/broker-b-s.log 2>&1 &
192.168.152.147:
nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b.properties > /usr/local/rocketmq/logs/broker-b.log 2>&1 &

6）验证

sh /usr/local/rocketmq/bin/mqadmin clusterList -n 192.168.152.146:9876

#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
rocketmq-cluster   broker-a                0     192.168.152.147:10920  V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 432297.61 -1.0000
rocketmq-cluster   broker-a                0     192.168.152.146:10910  V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 432297.61 -1.0000
rocketmq-cluster   broker-b                0     192.168.152.147:10930  V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 432297.61 -1.0000
rocketmq-cluster   broker-b                0     192.168.152.146:10940  V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 432297.61 -1.0000

如果这里都没有问题之后，下面再部署console

首先在git上拉取包
git clone https://github.com/apache/rocketmq-externals.git
修改配置，指定为nameserver的服务地址
vim src/main/resources/application.properties
rocketmq.config.namesrvAddr=192.168.152.146:9876;192.168.152.147:9876
打包
mvn clean package -Dmaven.test.skip=true
启动jar包
java -jar target/rocketmq-console-ng-1.0.1.jar

访问

随机文章
本站知识点必读
最近更新