努力挣扎的生活
yangfk
2021-10-30

elasticsearch部署

//

# ES安装

安装之前先关闭防火墙，selinux

# 常规操作

# 下载ES包

官方elastic.io下载地址 (opens new window)

elasticsearch-analysis-ik插件下载地址 (opens new window)

  • 5.0.0版本

elasticsearch-5.0.0 (opens new window) elasticsearch-analysis-ik-5.0.0 (opens new window)

  • 6.8.22版本

elasticsearch-6.8.22 (opens new window) elasticsearch-analysis-ik-6.8.22 (opens new window)

  • 7.12.0版本

elasticsearch-7.12.0 (opens new window) elasticsearch-analysis-ik-7.12.0 (opens new window)

创建用户，需要切换到elastic启动

groupadd elastic
useradd elastic -g elastic -p elasticsearch
1
2

解压包，授权

tar xf elasticsearch-7.12.0-linux-x86_64.tar.gz -C /opt

mv elasticsearch-7.12.0-linux-x86_64 /opt/elasticsearch
mkdir -p /opt/elasticsearch/plugins/ik
mkdir /opt/elasticsearch/{data,logs}
mkdir /opt/elasticsearch/plugins/ik

unzip elasticsearch-analysis-ik-7.12.0.zip -d /opt/elasticsearch/plugins/ik
chown -R elastic:elastic /opt/elasticsearch/
1
2
3
4
5
6
7
8
9

系统参数修改

  • 修改文件描述符

vi /etc/security/limits.conf

*       soft    nofile  65536
*       hard    nofile  65536
*       soft    nproc  65536
*       hard    nproc  65536
1
2
3
4

echo '* soft nproc 65535' > /etc/security/limits.d/90-nproc.conf

  • 修改VMA参数

echo 'vm.max_map_count = 655360' >> /etc/sysctl.conf

sysctl系统参数生效

sysctl -p

# 安装jdk 环境

注意：如果es版本大于等于7，jdk版本选择11以上

jdk配置文档

oracleJava下载链接 (opens new window)

es6.png

# Elasticsearch 5.X

# elassticarch.yml说明

在es 5的版本中，es5 集群配置就只需要几行

cluster.name: yfk-es
network.host: 192.168.1.12
http.port: 9200
node.name: node-1 #多节点名称不能一样
discovery.zen.ping.unicast.hosts: ["192.168.1.11", "192.168.1.12"] #自动发现机制，需要加入到集群的，在这里添加，ES集群中默认是9300
node.master: true
node.data: true
http.cors.enabled: true
http.cors.allow-origin: "*"
1
2
3
4
5
6
7
8
9

# Elasticsearch 7.X

注意💥💥：ES 7以上用jdk-11版本以上

# elassticarch.yml说明

  • 如果是集群配置，集群至少3台
cluster.name: yfk-es
node.name: 192.168.5.106
node.master: true
node.data: true
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs/
#bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300

discovery.zen.minimum_master_nodes: 2
cluster.initial_master_nodes: ["192.168.5.148:9300","192.168.5.106:9300","192.168.5.119:9300"]
discovery.seed_hosts: ["192.168.5.148:9300","192.168.5.106:9300","192.168.5.119:9300"]

discovery.zen.fd.ping_interval: 1s
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5

http.cors.enabled: true
http.cors.allow-origin: "*"
action.destructive_requires_name: true
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
  • 配置详细说明
# 设置集群名称，集群内所有节点的名称必须一致。
cluster.name: yfk-es
# 设置节点名称，集群内节点名称必须唯一。
node.name: node1
# 表示该节点会不会作为主节点，true表示会；false表示不会
node.master: true
# 当前节点是否用于存储数据，是：true、否：false
node.data: true
# 索引数据存放的位置
path.data: /opt/elasticsearch/data
# 日志文件存放的位置
path.logs: /opt/elasticsearch/logs
# 需求锁住物理内存，是：true、否：false
bootstrap.memory_lock: true
# 监听地址，用于访问该es
network.host:  0.0.0.0
# es对外提供的http端口，默认 9200
http.port: 9200
# TCP的默认监听端口，默认 9300
transport.tcp.port: 9300

# 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4）
discovery.zen.minimum_master_nodes: 2
# es7.x 之后新增的配置，写入候选主节点的设备地址，在开启服务后可以被选为主节点
discovery.seed_hosts: ["192.168.5.148:9300","192.168.5.106:9300","192.168.5.119:9300"
# es7.x 之后新增的配置，初始化一个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["192.168.5.148:9300","192.168.5.106:9300","192.168.5.119:9300"]

# 一个节点多久ping一次，默认1s
discovery.zen.fd.ping_interval: 1s
# 等待ping返回时间，默认30s
discovery.zen.fd.ping_timeout: 1m
# ping超时重试次数，默认3次
discovery.zen.fd.ping_retries: 5

# 是否支持跨域，是：true，在使用head插件时需要此配置
http.cors.enabled: true
# “*” 表示支持所有域名
http.cors.allow-origin: "*"
action.destructive_requires_name: true
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

# es7集群为什么至少3台

图文出处 (opens new window)

  • Elasticsearch 有两个节点

es1.png

discovery.zen.minimum_master_nodes: 1

此时出现网络波动, 导致 A—B 之间短暂断开连接, 根据选举规则, B将自己选举为 Master, 当网络波动结束, 就会出现两个Master的情况.

es2.png

discovery.zen.minimum_master_nodes: 2

Master 出现故障, 则 B 将永远不可能将自己选择为 Master

  • Elasticsearch 有三个节点

三节点配置: discovery.zen.minimum_master_nodes: 2

es3.png

出现网络波动 A 节点 和 别的节点短暂断开连接

es4.png

A节点降级, B和C 进行选举, 此处模拟选举B为 Master Node

es5.png

网络恢复后的节点状况.

# elasticsearch-head:5

通过安装head插件查看集群信息，通过docker启动，访问9100端口查看

docker安装：docker run -d --name es_admin -p 9100:9100 mobz/elasticsearch-head:5

在节点1上修改好配置文件直接发送到节点2、3，访问head插件时候只显示一个master

解决方案：进到节点2、3的/elasticsearch/data/ 目录下删除nodes文件，之后重启节点2、3的es进程即可

es7.png

# 添加到systemd

注意：es 目录在 /opt/elasticsearch

  • ES 6 vi /usr/lib/systemd/system/elasticsearch.service
[root@hdss14-12 ~]# grep -Ev "^$|#" /usr/lib/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target
[Service]
Environment=ES_HOME=/opt/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch
WorkingDirectory=/opt/elasticsearch
User=elastic
Group=elastic
ExecStart=/opt/elasticsearch/bin/elasticsearch  
StandardOutput=journal
StandardError=inherit
LimitNOFILE=65536
LimitNPROC=4096
LimitAS=infinity
LimitFSIZE=infinity
TimeoutStopSec=0
KillSignal=SIGTERM
KillMode=process
SendSIGKILL=no
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
  • ES 7 vi /usr/lib/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Environment=ES_HOME=/opt/elasticsearch
Environment=CONF_DIR=/opt/elasticsearch/config
Environment=DATA_DIR=/opt/elasticsearch/data
Environment=LOG_DIR=/opt/elasticsearch/logs
Environment=PID_DIR=/opt/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch

WorkingDirectory=/opt/elasticsearch

User=elastic
Group=elastic

ExecStartPre=/opt/elasticsearch/bin/elasticsearch-systemd-pre-exec

ExecStart=/opt/elasticsearch/bin/elasticsearch \
                                                -p ${PID_DIR}/elasticsearch.pid \
                                                --quiet \
                                                -Edefault.path.logs=${LOG_DIR} \
                                                -Edefault.path.data=${DATA_DIR} \
                                                -Edefault.path.conf=${CONF_DIR}

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /var/log/elasticsearch and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of processes
LimitNPROC=4096

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

# Built for distribution-5.6.13 (distribution)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
  • /etc/sysconfig/elasticsearch
################################
# Elasticsearch
################################

# Elasticsearch home directory
#ES_HOME=/usr/share/elasticsearch

# Elasticsearch Java path
JAVA_HOME=/usr/local/jdk1.8.0_161

# Elasticsearch configuration directory
#CONF_DIR=/etc/elasticsearch

# Elasticsearch data directory
#DATA_DIR=/var/lib/elasticsearch

# Elasticsearch logs directory
#LOG_DIR=/var/log/elasticsearch

# Elasticsearch PID directory
#PID_DIR=/var/run/elasticsearch

# Additional Java OPTS
#ES_JAVA_OPTS=

# Configure restart on package upgrade (true, every other setting will lead to not restarting)
#RESTART_ON_UPGRADE=true

################################
# Elasticsearch service
################################

# SysV init.d
#
# When executing the init script, this user will be used to run the elasticsearch service.
# The default value is 'elasticsearch' and is declared in the init.d file.
# Note that this setting is only used by the init script. If changed, make sure that
# the configured user can read and write into the data, work, plugins and log directories.
# For systemd service, the user is usually configured in file /usr/lib/systemd/system/elasticsearch.service
#ES_USER=elasticsearch
#ES_GROUP=elasticsearch

# The number of seconds to wait before checking if Elasticsearch started successfully as a daemon process
ES_STARTUP_SLEEP_TIME=5

################################
# System properties
################################

# Specifies the maximum file descriptor number that can be opened by this process
# When using Systemd, this setting is ignored and the LimitNOFILE defined in
# /usr/lib/systemd/system/elasticsearch.service takes precedence
MAX_OPEN_FILES=65536

# The maximum number of bytes of memory that may be locked into RAM
# Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
# in elasticsearch.yml.
# When using systemd, LimitMEMLOCK must be set in a unit file such as
# /etc/systemd/system/elasticsearch.service.d/override.conf.
MAX_LOCKED_MEMORY=unlimited

# Maximum number of VMA (Virtual Memory Areas) a process can own
# When using Systemd, this setting is ignored and the 'vm.max_map_count'
# property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf
MAX_MAP_COUNT=262144
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
  • ES 7以下

es_service1.png

# 启动es

  • 普通用户启动
su - elastic
/opt/elasticsearch/bin/elasticsearch -d
1
2
  • systemd启动

systemctl start elasticsearch.service

# 设置密码

./bin/elasticsearch-setup-passwords interactive

# ES日志模板

  • 单机部署变绿
curl -H "Content-Type:application/json" -XPUT http://192.168.14.12:9200/_template/k8s -d '{
 "template" : "k8s*",
 "index_patterns": ["k8s*"], 
 "settings": {
  "number_of_shards": 5,
  "number_of_replicas": 0
 }
}'
1
2
3
4
5
6
7
8

# 个人存储下载地址。。。

点击，获取下面下载资源

#jdk
java-1.8.0_161-1.x86_64.rpm
jdk-11.0.5-1.x86_64.rpm
jdk-15.0.1.tar.gz
openjdk-1.8.0_252.tar.gz
jdk-8u251-linux-arm64-vfp-hflt.tar.gz

#es
elasticsearch-5.0.0.tar.gz
elasticsearch-hq.tar.gz
elasticsearch-HQ-3.5.2.zip

elasticsearch-analysis-ik-5.0.0.zip
elasticsearch-analysis-ik-6.8.23.zip
elasticsearch-analysis-ik-7.12.0.zip
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//
如果此文章对您有帮助，点击 -->> 请博主喝咖啡 (opens new window)
上次更新: 2023/11/20, 16:23:11
Graylog日志告警
Elasticsearch HQ与curl操作

← Elasticsearch HQ与curl操作→

最近更新
01
openStack配置GPU直通(zed)
 10-21
02
nginx禁用真实IP
 10-15
03
docker运行一个bind9和常用的网络命令
 09-27
更多文章>
Theme by Vdoing | Copyright © 2019-2024 yangfk | 湘ICP备2021014415号-1
×
//