ES集群搭建实践
创建虚拟机
创建Linux新用户elastic
必须信息
使用elasticsearch账号操作:启动,关闭等
配置网卡信息
<pre mdtype="fences" cid="n29" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">vi /etc/sysconfig/network-scripts/ifcfg-ens33</pre>
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" cid="n549" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="none" # 关闭dhcp,需要固定ip
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens33"
UUID="6dcded77-ba54-4f70-a16c-80535656ba86"
DEVICE="ens33"
ONBOOT="yes"
IPADDR="192.168.3.81" #修改每个节点的ip地址
PREFIX="24"
GATEWAY="192.168.3.1" ## 确定网关地址
DNS1="114.114.114.114"
DOMAIN="8.8.8.8"
IPV6_PRIVACY="no"</pre>
重启网卡
service network restart
网卡重启报错的话,重启虚拟机
网卡验证
远程登录验证
修改密码命令
passwd 用户名
useradd <username>
passwd <username>
配置Java环境
ES下载地址
https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rpm.html#rpm-repo
ES7
linux version
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" cid="n558" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">https://www.elastic.co/downloads/past-releases/elasticsearch-7-17-10
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/rpm.html#rpm-repo
https://www.elastic.co/downloads/past-releases#elasticsearch 浏览各种release版本</pre>
docker version
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" cid="n83" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docker.html</pre>
yuminstall
<pre mdtype="fences" cid="n81" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">https://www.elastic.co/guide/en/elasticsearch/reference/7.17/rpm.html#rpm-repo</pre>
传输到目标服务器
<pre mdtype="fences" cid="n66" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">scp elasticsearch-7.17.10-linux-x86_64.tar.gz elastic@node151:~/</pre>
修改服务器 max file descriptors
Linux中修改文件描述符(file descriptors)的方法有多种。下面列举了一些常见的方法:
命令行参数:在运行程序时,可以使用命令行参数来设置文件描述符的数量。例如,通过
ulimit
命令可以设置进程的最大文件描述符数量,如ulimit -n 1024
表示将最大文件描述符数量设置为1024。修改配置文件:可以通过修改系统的配置文件来调整文件描述符的数量限制。在Linux系统中,有两个主要的配置文件与文件描述符相关:
/etc/security/limits.conf
和/etc/sysctl.conf
。在这些配置文件中,可以设置文件描述符的软限制和硬限制,以及其他相关参数。编程方式:通过编程语言(如C、C++、Java等)的API,可以在程序中设置文件描述符的数量。这通常涉及到使用系统调用(如
setrlimit
)或编程语言提供的相关函数。
文件描述符的修改与登录用户相关。在Linux系统中,每个用户都有自己的文件描述符限制。用户可以通过上述方法中的命令行参数或配置文件来调整自己的文件描述符数量限制。登录用户可以根据自身需求灵活地调整文件描述符的数量。
** 不同用户之间的文件描述符配置是相互独立的,即每个用户可以根据自己的需要设置文件描述符的数量。因此,文件描述符的修改配置是与用户相关的,不同用户之间的文件描述符是独立的,不会相互影响。每个用户可以根据自己的需求进行个性化的配置,而不会对其他用户产生影响。**
修改limits.conf和sysctl.conf区别
/etc/security/limits.conf
和/etc/sysctl.conf
是两个不同的配置文件,用于在Linux系统中进行不同类型的系统配置调整。
-
/etc/security/limits.conf
:作用:
limits.conf
文件用于设置用户级别的资源限制,包括文件描述符限制。针对对象:
limits.conf
文件的配置是针对用户级别的,可以为每个用户或用户组设置不同的限制。文件描述符限制:通过
nofile
或nofiles
参数可以设置每个用户(或用户组)可以打开的文件描述符的最大数量。该参数的值表示用户(或用户组)可以同时打开的文件或网络套接字的数量。-
示例配置:
<pre mdtype="fences" cid="n292" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sudo vim /etc/security/limits.conf
to add
hard nofile 65535</pre>
生效范围:配置
limits.conf
文件后,对应的限制会应用于特定的用户或用户组,而不会影响整个系统的其他用户。
-
/etc/sysctl.conf
:作用:
sysctl.conf
文件用于设置内核参数,包括系统级别的配置。针对对象:
sysctl.conf
文件的配置是针对整个系统的内核参数,适用于所有用户。文件描述符限制:通过修改
fs.file-max
参数可以调整系统级别的文件描述符限制,指定整个系统可以打开的文件描述符的最大数量。-
示例配置:
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n276" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sudo vim /etc/sysctl.conf
to add
fs.file-max = 190000</pre>
生效范围:配置
sysctl.conf
文件后,对应的内核参数会影响整个系统的所有用户,即所有用户共享相同的文件描述符限制。
总结:
/etc/security/limits.conf
用于设置用户级别的资源限制,包括文件描述符限制,配置针对用户或用户组,并且对应的限制仅适用于指定的用户或用户组。/etc/sysctl.conf
用于设置系统级别的内核参数,包括文件描述符限制,配置对整个系统生效,所有用户共享相同的限制。这句话指的是所有用户加起来不能超过这个值。所以就要注意limits和sysctl之间的数量关系,sysctl中的配置一定要大于limit
elastic is not in the sudoers file
<pre mdtype="fences" cid="n234" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">su
visudo -f /etc/sudoers
Then add the user below admin user like below syntax.
elastic ALL=(ALL) ALL</pre>
visudo
修改 max virtual memory areas vm.max_map_count [65530] is too low
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n158" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sysctl -w vm.max_map_count=262144</pre>
当出现"max virtual memory areas vm.max_map_count [65530] is too low"的错误消息时,这表示操作系统的虚拟内存区域数量限制太低,无法满足当前的需求。解决此问题的常见方法是增加vm.max_map_count
的值。
在Linux系统中,有几种配置方法可以调整vm.max_map_count
的值:
-
临时修改:可以使用以下命令临时修改
vm.max_map_count
的值:<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n193" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sysctl -w vm.max_map_count=362144
the setting will only last for the duration of the session. If the host reboots, the setting will be reset to the original value.
If you want to set this permanently, you need to edit /etc/sysctl.conf and set vm.max_map_count
sudo vim /etc/sysctl.conf</pre>
其中,xxxx是所需的新值。这种方法只在当前会话中生效,重启后会恢复为默认值。
-
持久修改:可以通过修改配置文件来永久修改
vm.max_map_count
的值。具体的配置文件路径可能因Linux发行版而异,一般是/etc/sysctl.conf
、/etc/sysctl.d/
目录下的配置文件。在该文件中添加或修改以下行:<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n197" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">vm.max_map_count=362144</pre>
然后保存文件并执行以下命令使配置生效:
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n199" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sudo sysctl -p
Do not forget to reboot or: sysctl --system</pre>
-
通过限制文件:有时,操作系统可能对修改
vm.max_map_count
的值有限制。在这种情况下,可以尝试通过限制文件来修改该值。在/etc/security/limits.conf
文件中,<pre mdtype="fences" cid="n240" lang="shell " class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">vim /etc/security/limits.conf</pre>
添加以下行:
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n202" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">* hard memlock unlimited
- soft nofile 65536
- hard nofile 65536
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536</pre>
这些配置方法的区别与联系如下:
临时修改适用于临时调整配置,对当前会话有效,重启后会恢复为默认值。
持久修改适用于永久性调整配置,会将修改写入配置文件中,重启后仍然有效。
通过限制文件修改
vm.max_map_count
的值可以绕过一些操作系统对配置修改的限制。
对于这个问题,不同用户无需单独配置。一旦修改了vm.max_map_count
的值,对于所有用户来说都会生效。因为该配置是作用于整个操作系统的虚拟内存区域,所以不需要为每个用户单独配置。任何登录到系统的用户都将受到新的vm.max_map_count
值的影响。
如何验证max file descriptors配置是否生效
<pre mdtype="fences" cid="n326" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">ulimit -n</pre>
max file descriptors too low 没有生效原因
-
没有改配置文件
vim /etc/security/limits.conf
<pre mdtype="fences" cid="n310" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">* hard memlock unlimited
- soft nofile 65536
- hard nofile 65536
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536</pre>
sudo vim /etc/sysctl.conf
<pre mdtype="fences" cid="n314" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">vm.max_map_count=362144</pre>
-
没有执行
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" cid="n318" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">sudo sysctl -p</pre>
-
没有重启终端
通过执行
ulimit -n
如果仍然不行则:如果仍然不行,则退出当前shell session(terminal),重新登录。亲测有效。
解压文件
<pre mdtype="fences" cid="n165" lang="shell" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">tar -xzf you.tar.gz</pre>
z : gz
x:extract
f: file name?
修改配置文件
单集群单节点配置
<pre mdtype="fences" cid="n149" lang="xml" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">cluster.name: zgc-cluster
node.name: node151
network.host: 172.16.193.151
http.port: 9200
transport.port: 9300discovery.seed_hosts: ["172.16.193.152:9300","172.16.193.153:9300"]
cluster.initial_master_nodes: ["node151"]
cluster.initial_master_nodes: ["node152"]</pre>
启动后验证
<pre mdtype="fences" cid="n379" lang="html" class="md-fences md-end-block ty-contain-cm modeLoaded" spellcheck="false" style="box-sizing: border-box; overflow: visible; font-family: Monaco, Consolas, "Andale Mono", "DejaVu Sans Mono", monospace; margin-top: 0px; margin-bottom: 20px; font-size: 0.9rem; display: block; break-inside: avoid; text-align: left; white-space: normal; background-color: rgb(51, 51, 51); position: relative !important; padding: 10px 10px 10px 30px; width: inherit; background-position: initial initial; background-repeat: initial initial;">http://172.16.193.151:9200/</pre>
基于Docker搭建ES集群
-