Elasticsearch Cluster 安装与配置
时间:2022-05-03
本文章向大家介绍Elasticsearch Cluster 安装与配置,主要内容包括23.1.2. Elasticsearch Cluster、23.1.3. 负载均衡配置、23.2.1. elasticsearch-analysis-ik、23.3. 节点管理、23.3.2. 节点健康状态、23.3.3. 节点http状态、基本概念、基础应用、原理机制和需要注意的事项等,并结合实例形式分析了其使用技巧,希望通过本文能帮助到大家理解应用这部分内容。
本文节选自《Netkiller Database 手札》作者:netkiller 网站: http://www.netkiller.cn
23.1.2. Elasticsearch Cluster
集群模式需要两个以上的节点,通常是一个 master 节点,多个 data 节点
首先在所有节点上安装 elasticsearch,然后配置各节点的配置文件,对于 5.5.1 不需要配置决定哪些节点属于 master 节点 或者 data 节点。
curl -s https://raw.githubusercontent.com/oscm/shell/master/search/elasticsearch/elasticsearch-5.x.sh | bash
配置文件
cluster.name: elasticsearch-cluster # 配置集群名称,所有服务器服务器保持一致
node.name: node-1 # 每个节点唯一标识,每个节点只需改动这里,一次递增 node-1, node-2, node-3 ...
network.host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["172.16.0.20", "172.16.0.21","172.16.0.22"] # 所有节点的IP 地址写在这里
discovery.zen.minimum_master_nodes: 3 # 可以作为master的节点总数,有多少个节点就写多少
http.cors.enabled: true
http.cors.allow-origin: "*"
查看节点状态,使用curl工具: curl 'http://localhost:9200/_nodes/process?pretty'
root@netkiller /var/log/elasticsearch % curl 'http://localhost:9200/_nodes/process?pretty'
{
"_nodes" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"cluster_name" : "my-application",
"nodes" : {
"-lnKCmBXRpiwExLns0jc9g" : {
"name" : "node-1",
"transport_address" : "10.104.3.2:9300",
"host" : "10.104.3.2",
"ip" : "10.104.3.2",
"version" : "5.5.1",
"build_hash" : "19c13d0",
"roles" : [
"master",
"data",
"ingest"
],
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 23669,
"mlockall" : false
}
},
"WVsgYi2HT8GWnZU1kUwFwA" : {
"name" : "node-2",
"transport_address" : "10.186.7.221:9300",
"host" : "10.186.7.221",
"ip" : "10.186.7.221",
"version" : "5.5.1",
"build_hash" : "19c13d0",
"roles" : [
"master",
"data",
"ingest"
],
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 12641,
"mlockall" : false
}
}
}
}
启动节点后回生成 cluster.name 为文件名的日志文件。
谁先启动谁讲成为master
[2017-08-11T17:42:46,018][INFO ][o.e.c.s.ClusterService ] [node-1] new_master {node-1}{-lnKCmBXRpiwExLns0jc9g}{rZcJDIynSzq2Td3yP2kN5A}{10.104.3.2}{10.104.3.2:9300}, added {{node-2}{WVsgYi2HT8GWnZU1kUwFwA}{X13ShUpAQa2zA1Mgcsm3bQ}{10.186.7.221}{10.186.7.221:9300},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{node-2}{WVsgYi2HT8GWnZU1kUwFwA}{X13ShUpAQa2zA1Mgcsm3bQ}{10.186.7.221}{10.186.7.221:9300}]
如果master出现故障,其他节点会接管
[2017-08-11T17:44:52,797][INFO ][o.e.c.s.ClusterService ] [node-2] master {new {node-2}{WVsgYi2HT8GWnZU1kUwFwA}{vl8kQx8sQdGVVohrNQnZOQ}{10.186.7.221}{10.186.7.221:9300}}, removed {{node-1}{-lnKCmBXRpiwExLns0jc9g}{rZcJDIynSzq2Td3yP2kN5A}{10.104.3.2}{10.104.3.2:9300},}, added {{node-1}{-lnKCmBXRpiwExLns0jc9g}{odnoG9kpQpeX1ltx5KYTSw}{10.104.3.2}{10.104.3.2:9300},}, reason: zen-disco-elected-as-master ([1] nodes joined)[{node-1}{-lnKCmBXRpiwExLns0jc9g}{odnoG9kpQpeX1ltx5KYTSw}{10.104.3.2}{10.104.3.2:9300}]
[2017-08-11T17:44:53,184][INFO ][o.e.c.r.DelayedAllocationService] [node-2] scheduling reroute for delayed shards in [59.5s] (11 delayed shards)
[2017-08-11T17:44:53,929][INFO ][o.e.c.r.a.AllocationService] [node-2] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[information][0]] ...]).
master 节点恢复上线会提示
[2017-08-11T17:44:52,855][INFO ][o.e.c.s.ClusterService ] [node-1] detected_master {node-2}{WVsgYi2HT8GWnZU1kUwFwA}{vl8kQx8sQdGVVohrNQnZOQ}{10.186.7.221}{10.186.7.221:9300}, added {{node-2}{WVsgYi2HT8GWnZU1kUwFwA}{vl8kQx8sQdGVVohrNQnZOQ}{10.186.7.221}{10.186.7.221:9300},}, reason: zen-disco-receive(from master [master {node-2}{WVsgYi2HT8GWnZU1kUwFwA}{vl8kQx8sQdGVVohrNQnZOQ}{10.186.7.221}{10.186.7.221:9300} committed version [44]])
23.1.3. 负载均衡配置
因为 elasticsearch 没有用户认证机制我们通常在内网访问他。如果对外提供服务需要增加用户认证。
$ printf "john:$(openssl passwd -crypt s3cr3t)n" > /etc/nginx/passwords
创建 nginx 配置文件 /etc/nginx/conf.d/elasticsearch.conf
upstream elasticsearch {
server 172.16.0.10:9200;
server 172.16.0.20:9200;
server 172.16.0.30:9200;
keepalive 15;
}
server {
listen 9200;
auth_basic "Protected Elasticsearch";
auth_basic_user_file passwords;
location ~* ^(/_cluster|/_nodes) {
return 403;
break;
}
location / {
if ($request_filename ~ _shutdown) {
return 403;
break;
}
proxy_pass http://elasticsearch;
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}
}
反复使用下面方法请求,最终你会发现 total_opened 会达到你的nginx 配置数量
$ curl 'http://test:test@localhost:9200/_nodes/stats/http?pretty' | grep total_opened
# "total_opened" : 15
23.2.1. elasticsearch-analysis-ik
安装插件
root@netkiller ~ % /usr/share/elasticsearch/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.1/elasticsearch-analysis-ik-5.5.1.zip
-> Downloading https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.1/elasticsearch-analysis-ik-5.5.1.zip
[=================================================] 100%
-> Installed analysis-ik
curl -XPOST http://localhost:9200/index/fulltext/_mapping -d'
{
"properties": {
"content": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
}
}
}'
23.3. 节点管理
23.3.1. 查看索引
root@netkiller ~ % curl 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open information oygxIi-dR1eB9NoIZtJrxQ 5 1 45 42 731kb 731kb
green open .kibana 9jBBaOomTO2EakZlZqnE-g 1 1 5 1 62.5kb 31.2kb
green open logstash-api WHXZhn3vRWiuVbhR8rGoEg 5 1 565 0 3.8mb 1.9mb
23.3.2. 节点健康状态
root@netkiller ~ % curl 'http://localhost:9200/_cat/health?v'
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1502445967 18:06:07 my-application yellow 2 2 17 11 0 0 5 0 - 77.3%
root@netkiller ~ % curl 'http://localhost:9200/_cluster/health'
{"cluster_name":"my-application","status":"yellow","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":11,"active_shards":17,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":5,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":77.27272727272727}
23.3.3. 节点http状态
root@VM_3_2_centos ~ % curl 'localhost:9200/_nodes/stats/http?pretty'
{
"_nodes" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"cluster_name" : "my-application",
"nodes" : {
"-lnKCmBXRpiwExLns0jc9g" : {
"timestamp" : 1502446878773,
"name" : "node-1",
"transport_address" : "10.104.3.2:9300",
"host" : "10.104.3.2",
"ip" : "10.104.3.2:9300",
"roles" : [
"master",
"data",
"ingest"
],
"http" : {
"current_open" : 4,
"total_opened" : 29
}
},
"WVsgYi2HT8GWnZU1kUwFwA" : {
"timestamp" : 1502446878782,
"name" : "node-2",
"transport_address" : "10.186.7.221:9300",
"host" : "10.186.7.221",
"ip" : "10.186.7.221:9300",
"roles" : [
"master",
"data",
"ingest"
],
"http" : {
"current_open" : 0,
"total_opened" : 2
}
}
}
}
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- 《Java从入门到失业》第四章:类和对象(4.3):一个完整的例子带你深入类和对象
- 《Java从入门到失业》第四章:类和对象(4.2):String类
- 《Java从入门到失业》第三章:基础语法及基本程序结构(3.9):数组(数组基本使用、数组的循环、数组拷贝、数组排序、多维数组)
- Establishing SSL connection without server identity verification is not recommended
- 《Java从入门到失业》第三章:基础语法及基本程序结构(3.8):流程控制(循环语句、while语句、for语句)
- 《Java从入门到失业》第三章:基础语法及基本程序结构(3.8):流程控制(选择语句、if-else语句、switch语句)
- 线上服务启动卡死,堆栈分析
- 《Java从入门到失业》第三章:基础语法及基本程序结构(3.7):运算符(自增自减、关系运算、逻辑运算、条件运算、位运算、赋值运算、类型转换)
- JavaWeb——MyBatis框架之多表查询操作实战案例总结(MyBatis1对1查询,MyBatis1对多查询,MyBatis多对多查询)
- kubernetes(二十二) 服务网格化istio入门
- 使用Ngrok将本地服务映射为公网服务
- 前端图片下载
- 使用JDBC连接MySQL数据库--典型案例分析(七)----批量插入员工信息
- NIO删除文件提示文件AccessDeniedException
- 使用JDBC连接MySQL数据库--典型案例分析(八)----实现员工数据的分页查询