Elasticsearch7.5.0安装

切换到root用户

sudo su -

配置elasticsearch用户

usermod -s /bin/bash elasticsearch

创建数据和日志目录

mkdir /var/log/elasticsearch7.5.0
mkdir /data/elasticsearch7.5.0

分配给elasticsearch用户

chown -R elasticsearch:elasticsearch /var/log/elasticsearch7.5.0
chown -R elasticsearch:elasticsearch /data/elasticsearch7.5.0

下载安装

cd /opt
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz
tar zxvf elasticsearch-7.5.0-linux-x86_64.tar.gz
chown -R elasticsearch elasticsearch-7.5.0

修改配置文件

1. GIS单节点 /opt/elasticsearch-7.5.0/config/elasticsearch.yml 供参考

   cluster.name: gis_es
   node.name: gis-es-1.novalocal
   node.attr.rack: rc-1
   path.data: /data/elasticsearch7.5.0
   path.logs: /var/log/elasticsearch7.5.0
   bootstrap.memory_lock: true
   network.host: ...
   http.port: 9201
   transport.tcp.port: 9301
   cluster.initial_master_nodes: ["gis_es"]
   reindex.remote.whitelist: ...:9200

2. Trip 集群es-traffic-mgmt-1节点 /opt/elasticsearch-7.5.0/config/elasticsearch.yml 供参考

   cluster.name: trip-traffic_es
   node.name: es-traffic-mgmt-1.novalocal
   node.attr.rack: rc-1
   path.data: /data/elasticsearch7.5.0
   path.logs: /var/log/elasticsearch7.5.0
   bootstrap.memory_lock: true
   network.host: ...
   http.port: 9201
   transport.tcp.port: 9301
   discovery.seed_hosts: ["...:9301", "...:9301"]
   cluster.initial_master_nodes: ["es-traffic-mgmt-1", "es-trip-mgmt-1"]
   reindex.remote.whitelist: ...:9200

3. Trip 集群es-trip-mgmt-1节点 /opt/elasticsearch-7.5.0/config/elasticsearch.yml 供参考

   cluster.name: trip-traffic_es
   node.name: es-trip-mgmt-1.novalocal
   node.attr.rack: rc-1
   path.data: /data/elasticsearch7.5.0
   path.logs: /var/log/elasticsearch7.5.0
   bootstrap.memory_lock: true
   network.host: ...
   http.port: 9201
   transport.tcp.port: 9301
   discovery.seed_hosts: ["...:9301", "...:9301"]
   cluster.initial_master_nodes: ["es-traffic-mgmt-1", "es-trip-mgmt-1"]
   reindex.remote.whitelist: ...:9200

启动

1. 切换到elasticsearch启动
   su - elasticsearch
   /opt/elasticsearch-7.5.0/bin/elasticsearch -d

   如果报错：
   ERROR: [2] bootstrap checks failed
   [1]: max number of threads [2048] for user [elasticsearch] is too low, increase to at least [4096]
   cp /etc/security/limits.d/elsearch-limit.conf /etc/security/limits.d/elsearch-limit.conf.bak
   则修改配置文件：vi /etc/security/limits.d/elsearch-limit.conf
   elasticsearch soft nproc 2048 改为 4096
   elasticsearch hard nproc 2048改为 4096

2. 重启命令
   找到进程kill ps -ef |grep elasticsearch-7.5.0
   启动 /opt/elasticsearch-7.5.0/bin/elasticsearch -d

数据迁移

1. 主要使用官方自带的Reindex from a remote cluster
   参考:https://www.elastic.co/guide/en/elasticsearch/reference/7.5/reindex-upgrade-remote.html

2. 但是迁移数据前需要手工在es7.5创建好索引，所以新建python3脚本来创建索引和迁移数据

3. 运行脚本时候，很有可能报错”maximum shards open“则修改max_shards_per_node

4. |

   curl -X PUT ``"127.0.0.1:9201/_cluster/settings" -H ``'Content-Type: application/json' -d'

   {

   "persistent" : {

   "cluster.max_shards_per_node" : ``"2000"

   }

   }

   '

   |

5. 修改脚本中的3行，替换脚本中的127.0.0.1 为具体的ip

   reindex_py3.py 展开源码

   <pre style="margin: 0px; padding: 0px; font-family: ConfluenceInstalledFont, monospace;">es_old ip port 必须与 /opt/elasticsearch-7.5.0/config/elasticsearch.yml 中的 reindex.remote.whitelist: <address>:<port> 保持一致，如必须同为 127.0.0.1:9200 或者localhost:9200， 不能一个用localhost 另一个用127.0.0.1</pre>

   ES7.5.0 默认内存是1G， 导入可能报错, 需要修改 /opt/elasticsearch-7.5.0/config/jvm.options, 根据vm 内存进行调整，内存过大可能导致机器无法使用

   |

   -Xms3g

   -Xmx3g

   |

   Error log

   |

   elasticsearch.exceptions.TransportError: TransportError(``429``, ``'circuit_breaking_exception'``, ``'[parent] Data too large, data for [<http_request>] would be [1023885146/976.4mb], which is larger than the limit of [1003493785/957mb], real usage: [1023865048/976.4mb], new bytes reserved: [20098/19.6kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=20098/19.6kb, accounting=20440594/19.4mb]'``)

   |

验证结果

1. 打开es7.5.0 查看索引“.tasks“数据，记录了Reindex 的结果。

配置为系统服务

1. 待补充

停止旧版本ES服务

1. root用户 systemctl stop elasticsearch

多类型索引更新
因为旧版本支持一个索引包含多个type，而新版本一个索引建议只包含一个type，所以旧版本包含多个type的所以需要拆分多个新索引再同步数据