docker-compose部署单机版本分片mongo
时间:2022-07-26
本文章向大家介绍docker-compose部署单机版本分片mongo,主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项,具有一定的参考价值,需要的朋友可以参考一下。
环境介绍
- os: centos7
- docker: 18.09.0
- mongo: 4.0.5
执行步骤
1. 清理旧数据(如果需要)
执行 clean-deploy.sh
- 删除之前的容器
- 删除数据目录
DIR=/data/fates
DATA_PATH="${DIR}/mongo"
PWD='kinnylee'
# 第一次执行没有旧数据,不需要执行这步
docker-compose -f fates-mongo-compose.yaml down
if [ -d "${DATA_PATH}" ]; then
echo "delete directory: ${DATA_PATH}"
echo ${PWD} | sudo -S rm -rf ${DATA_PATH}
fi
2. 创建数据目录
执行before-deploy.sh
- 创建数据目录
PWD='kinnylee'
DATA_DIR_LIST=('config1' 'config2' 'config3' 'shard1' 'shard2' 'shard3' 'script')
function check_directory() {
if [ ! -d "${DATA_PATH}" ]; then
echo "create directory: ${DATA_PATH}"
echo ${PWD} | sudo -S mkdir -p ${DATA_PATH}
else
echo "directory ${DATA_PATH} already exists."
fi
cd "${DATA_PATH}"
for SUB_DIR in ${DATA_DIR_LIST[@]}
do
if [ ! -d "${DATA_PATH}/${SUB_DIR}" ]; then
echo "create directory: ${DATA_PATH}/${SUB_DIR}"
echo "${PWD}" | sudo -S mkdir -p "${DATA_PATH}/${SUB_DIR}"
else
echo "directory: ${DATA_PATH}/${SUB_DIR} already exists."
fi
done
echo "change directory owner to $USER:$USER"
echo "${PWD}" | sudo -S chown -R $USER:$USER "${DATA_PATH}"
}
check_directory
3. 部署docker容器
3.1 无授权模式
执行deploy.sh
docker-compose -f fates-mongo-compose.yaml up -d
yaml文件如下
version: '3.4'
services:
shard1:
image: mongo:4.0.5
# --shardsvr: 这个参数仅仅只是将默认的27017端口改为27018,如果指定--port参数,可用不需要这个参数
# --directoryperdb:每个数据库使用单独的文件夹
command: mongod --shardsvr --directoryperdb --replSet shard1
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard1:/data/db
shard2:
image: mongo:4.0.5
command: mongod --shardsvr --directoryperdb --replSet shard2
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard2:/data/db
shard3:
image: mongo:4.0.5
command: mongod --shardsvr --directoryperdb --replSet shard3
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard3:/data/db
config1:
image: mongo:4.0.5
# --configsvr: 这个参数仅仅是将默认端口由27017改为27019, 如果指定--port可不添加该参数
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config1:/data/configdb
config2:
image: mongo:4.0.5
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config2:/data/configdb
config3:
image: mongo:4.0.5
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config3:/data/configdb
mongos:
image: mongo:4.0.5
# mongo3.6版默认绑定IP为127.0.0.1,此处绑定0.0.0.0是允许其他容器或主机可以访问
command: mongos --configdb fates-mongo-config/config1:27019,config2:27019,config3:27019 --bind_ip 0.0.0.0 --port 27017
ports:
- 27017:27017
volumes:
- /etc/localtime:/etc/localtime
depends_on:
- config1
- config2
- config3
3.2 授权模式
1. 执行generate-keyfile.sh
- 生成keyfile文件
#!/bin/bash
DATA_PATH=/data/fates/mongo
PWD='kinnylee'
function check_directory() {
if [ ! -d "${DATA_PATH}" ]; then
echo "directory: ${DATA_PATH} not exists, please run before-depoly.sh."
fi
}
function generate_keyfile() {
cd "${DATA_PATH}/script"
if [ ! -f "${DATA_PATH}/script/mongo-keyfile" ]; then
echo 'create mongo-keyfile.'
openssl rand -base64 756 -out mongo-keyfile
echo "${PWD}" | sudo -S chmod 600 mongo-keyfile
echo "${PWD}" | sudo -S chown 999 mongo-keyfile
else
echo 'mongo-keyfile already exists.'
fi
}
check_directory
generate_keyfile
2. 执行deploy-key.sh
docker-compose -f fates-mongo-compose-key.yaml up -d
yaml文件如下
version: '3.4'
services:
shard1:
image: mongo:4.0.5
# --shardsvr: 这个参数仅仅只是将默认的27017端口改为27018,如果指定--port参数,可用不需要这个参数
# --directoryperdb:每个数据库使用单独的文件夹
command: mongod --shardsvr --directoryperdb --replSet shard1 --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard1:/data/db
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
shard2:
image: mongo:4.0.5
command: mongod --shardsvr --directoryperdb --replSet shard2 --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard2:/data/db
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
shard3:
image: mongo:4.0.5
command: mongod --shardsvr --directoryperdb --replSet shard3 --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/shard3:/data/db
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
config1:
image: mongo:4.0.5
# --configsvr: 这个参数仅仅是将默认端口由27017改为27019, 如果指定--port可不添加该参数
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config1:/data/configdb
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
config2:
image: mongo:4.0.5
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config2:/data/configdb
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
config3:
image: mongo:4.0.5
command: mongod --configsvr --directoryperdb --replSet fates-mongo-config --smallfiles --keyFile /data/mongo-keyfile
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/config3:/data/configdb
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
mongos:
image: mongo:4.0.5
# mongo3.6版默认绑定IP为127.0.0.1,此处绑定0.0.0.0是允许其他容器或主机可以访问
command: mongos --configdb fates-mongo-config/config1:27019,config2:27019,config3:27019 --bind_ip 0.0.0.0 --port 27017 --keyFile /data/mongo-keyfile
ports:
- 27017:27017
volumes:
- /etc/localtime:/etc/localtime
- /data/fates/mongo/script/mongo-keyfile:/data/mongo-keyfile
depends_on:
- config1
- config2
- config3
4. 配置分片信息
容器启动需要一定时间,执行这一步时要等上一步把容器都启动起来,才能执行,执行不成功就多运行几遍
执行shard-config.sh
docker-compose -f fates-mongo-compose.yaml exec config1 bash -c "echo 'rs.initiate({_id: "fates-mongo-config",configsvr: true, members: [{ _id : 0, host : "config1:27019" },{ _id : 1, host : "config2:27019" }, { _id : 2, host : "config3:27019" }]})' | mongo --port 27019"
docker-compose -f fates-mongo-compose.yaml exec shard1 bash -c "echo 'rs.initiate({_id: "shard1",members: [{ _id : 0, host : "shard1:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec shard2 bash -c "echo 'rs.initiate({_id: "shard2",members: [{ _id : 0, host : "shard2:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec shard3 bash -c "echo 'rs.initiate({_id: "shard3",members: [{ _id : 0, host : "shard3:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard1/shard1:27018")' | mongo"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard2/shard2:27018")' | mongo"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard3/shard3:27018")' | mongo"
没有出现
Connection refused
等错误就表示成功了
5. 创建用户(授权模式)
执行add-user.sh
- 需要添加用户登录权限执行这一步,不需要登录的不用执行这一步
- 脚本中管理员用户名和密码都是root,可自行修改
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn db.createUser({user:"root",pwd:"root",roles:[{role:"root",db:"admin"}]})' | mongo"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn show user' | mongo"
6. 验证
- 通过客户端连接验证是否成功
mongo {ip}
6.1 无授权模式
执行check.sh
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn show users' | mongo"
6.2 授权模式
执行check-key.sh
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn show users' | mongo -u root -p root"
完整脚本
- 容器启动需要一定时间,脚本里设置为120s,但是经常还是有节点完全没有起来,可单独执行shard_config执行分片,否则无法连接到mongo
- 包括后面添加用户这一步,也是需要等容器完全起来执行才不会报错
- 这里为了方便一键执行,把所有步骤放到一起了
DIR=/data/fates
DATA_PATH="${DIR}/mongo"
DATA_DIR_LIST=('config1' 'config2' 'config3' 'shard1' 'shard2' 'shard3' 'script')
BASE_DIR=$(cd "$(dirname "$0")";pwd)
function check_directory() {
if [ ! -d "${DATA_PATH}" ]; then
echo "create directory: ${DATA_PATH}"
sudo mkdir -p ${DATA_PATH}
else
echo "directory ${DATA_PATH} already exists."
fi
cd "${DATA_PATH}"
for SUB_DIR in ${DATA_DIR_LIST[@]}
do
if [ ! -d "${DATA_PATH}/${SUB_DIR}" ]; then
echo "create directory: ${DATA_PATH}/${SUB_DIR}"
sudo mkdir -p "${DATA_PATH}/${SUB_DIR}"
else
echo "directory: ${DATA_PATH}/${SUB_DIR} already exists."
fi
done
echo "change directory owner to $USER:$USER"
sudo chown -R $USER:$USER "${DATA_PATH}"
}
function rm_old_directory() {
if [ -d "${DATA_PATH}" ]; then
echo "delete directory: ${DATA_PATH}"
sudo rm -rf ${DATA_PATH}
fi
}
function cp_script() {
cp "${BASE_DIR}"/* "${DATA_PATH}"/script
}
function deploy_no_auth() {
cd "${DATA_PATH}"/script
docker-compose -f fates-mongo-compose.yaml up -d
}
function deploy_with_auth() {
cd "${DATA_PATH}"/script
docker-compose -f fates-mongo-compose-key.yaml up -d
}
function generate_keyfile() {
cd "${DATA_PATH}/script"
if [ ! -f "${DATA_PATH}/script/mongo-keyfile" ]; then
echo 'create mongo-keyfile.'
openssl rand -base64 756 -out mongo-keyfile
sudo chmod 600 mongo-keyfile
sudo chown 999 mongo-keyfile
else
echo 'mongo-keyfile already exists.'
fi
}
function config_shard() {
cd "${DATA_PATH}/script"
docker-compose -f fates-mongo-compose.yaml exec config1 bash -c "echo 'rs.initiate({_id: "fates-mongo-config",configsvr: true, members: [{ _id : 0, host : "config1:27019" },{ _id : 1, host : "config2:27019" }, { _id : 2, host : "config3:27019" }]})' | mongo --port 27019"
docker-compose -f fates-mongo-compose.yaml exec shard1 bash -c "echo 'rs.initiate({_id: "shard1",members: [{ _id : 0, host : "shard1:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec shard2 bash -c "echo 'rs.initiate({_id: "shard2",members: [{ _id : 0, host : "shard2:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec shard3 bash -c "echo 'rs.initiate({_id: "shard3",members: [{ _id : 0, host : "shard3:27018" }]})' | mongo --port 27018"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard1/shard1:27018")' | mongo"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard2/shard2:27018")' | mongo"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo 'sh.addShard("shard3/shard3:27018")' | mongo"
}
function add_user() {
cd "${DATA_PATH}/script"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn db.createUser({user:"root",pwd:"root",roles:[{role:"root",db:"admin"}]})' | mongo"
}
function check() {
cd "${DATA_PATH}/script"
docker-compose -f fates-mongo-compose.yaml exec mongos bash -c "echo -e 'use adminn show users' | mongo -u root -p root"
}
function main() {
rm_old_directory
check_directory
cp_script
generate_keyfile
deploy_no_auth
sleep 120
config_shard
add_user
deploy_with_auth
sleep 90
check
}
main
参考
- Codeforces 706B Interesting drink
- HDU 1232 畅通工程
- 算法--枚举策略
- 趣味理解朴素贝叶斯
- python笔记1-用python解决小学生数学题
- POJ 1012 Joseph
- 1344 走格子
- 如何利用配置中心规范构建PaaS服务配置
- Selenium2+python自动化8-SeleniumBuilder辅助定位元素
- HDU 1250 Hat's Fibonacci
- Scrapy在Ubuntu下的安装与配置
- Selenium2+python自动化20-引入unittest框架
- HDU 1002 A + B Problem II(高精度加法(C++/Java))
- POJ 1018 Communication System
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- 2.1 注释
- 2.2 标识符
- 数据科学的软件工程技巧和最佳实践
- ntp 服务开机启动失败
- 4.8 this关键字
- 使用Pyppeteer进行gmail模拟登录
- 一个没法商用,但是好玩有趣的 Python 手绘图形库!
- 使用豆瓣源安装python包
- [已解决]报错:ValueError: Expected 2D array, got scalar array instead
- [已解决]报错UnicodeDecodeError
- [已解决]报错Could not install packages due to an EnvironmentError
- 用Cython加速Python代码
- [已解决]windows安装docker的问题
- 使用VBA达到vlookup效果
- Excel简单应用