什么是elasticsearch
- 基于apache lucene 构建的开源搜索引擎
- 采用java 编写,提供简单易用的RESTFul API
- 轻松的横向拓展,可支持PB级的结构化或非结构化数据处理
应用场景
- 海量数据分析引擎
- 站内搜索引擎
- 数据仓库
单实例安装
- 下载5.5.2版本
- ~~官方~~
- 百度云 链接:https://pan.baidu.com/s/1L-pws-7SET1Ms4xKhsFL-w 密码:7kmz
- 解压
tar -zxvf elasticsearch-5.5.2.gz
- 运行
sh ./elasticsearch-5.5.2/bin/elasticsearch
需要在jdk8+ 运行 - 查看是否运行成功
localhost:9200
-
运行结果 (
插件安装(为了更直观的使用es)
- 安装node
brew install -g node
插件需要在6.0以上的node环境运行 - 下载插件
git clone https://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head/
-
npm install
仅第一次需要安装 -
npm run start
运行head插件 - 插件会在9100启动,elasticsearch在9200启动 是跨域请求,需要修改一下配置
cd **/elasticsearch-5.5.2/config
-
vim elasticsearch.yml
添加以下配置http.cors.enabled: true
http.cors.allow-origin: "*"
-
sh ../bin/elasticsearch –d
后台启动es cd ../elasticsearch-head/
npm run start
-
访问 localhost:9200 效果image
多实例安装
mkdir es_cluster
cp ../elasticsearch-5.5.2.gz ./
tar -zxvf elasticsearch-5.5.2.gz
-
cp -r elasticsearch-5.5.2 es_master
主 -
cp -r elasticsearch-5.5.2 es_slave1
从1 -
cp -r elasticsearch-5.5.2 es_slave2
从2 -
rm -rf elasticsearch-5.5.2*
删除多余的包 -
vim es_master/config/elasticsearch.yml
修改主配置http.cors.enabled: true
http.cors.allow-origin: "*"
cluster.name: ibest #集群名称
node.name: master #结点名称
node.master: true #指定为主结点
- 关闭之前启动单实例启动多实例master
lsof -i :9200
-
kill *
* 对应9200的id ./es_master/bin/elasticsearch -d
-
vim es_slave1/config/elasticsearch.yml
修改从1配置cluster.name: ibest #集群名称,必须和主结点一致,否则无法加入集群
node.name: slave1 #结点名称
network.host: 127.0.0.1 #和其他结点通信的地址
http.port: 8200 #9200已经被主结点占用
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]#发现主结点
-
./es_slave1/bin/elasticsearch -d
启动从1 -
vim es_slave2/config/elasticsearch.yml
修改从2配置cluster.name: ibest #集群名称,必须和主结点一致,否则无法加入集群
node.name: slave2 #结点名称
network.host: 127.0.0.1 #和其他结点通信的地址
http.port: 7200 #9200已经被主结点占用
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]#发现主结点
-
./es_slave2/bin/elasticsearch -d
启动从2 -
最终效果 image
基础概念
- 将es与mysql 联系起来
es | mysql |
---|---|
索引 | database |
类型 | table |
文档id | 一条记录的id |
- API基本格式 http://<ip>:<port>/<索引>/<类型>/<文档id>。请求方式:Get、Post、Put、Delete。
- es索引为结构化索引 和 非结构化索引
- settings:修改,设置分片和副本数的
- number_of_shards:分片数
- number_of_replicas:备份数
- mappings:修改,设置字段和类型
- properties:定义type的属性值
- type: 数据类型定义
- analyzer: 定义分析器
- properties:定义type的属性值
创建索引
json数据如下
"settings":{
"number_of_shards":5,
"number_of_replicas":1
},
"mappings":{
"man":{
"properties": {
"name":{
"type": "text"
},
"country": {
"type": "keyword"
},
"age": {
"type": "integer"
},
"date": {
"type": "date",
"format":"yyyy-MM-dd HH:mm:ss||yyy-MM-dd||epoch_millis"
}
}
},
"woman": {
}
}
}
image
image
image
插入
- 指定ID插入 put请求,id自定义
{
"name": "jack",
"age": 18,
"country": "China",
"date": "2020-04-21"
}
image
- 自动产生文档ID post请求,id自动生成
{
"name": "tom",
"age": 11,
"country": "China",
"date": "2001-04-21"
}
[站外图片上传中...(image-907979-1587472414044)]
image
修改 关键词_update
- 直接修改文档
{
"doc": {
"name": "jak_bus"
}
}
image
- 脚本修改文档
- ctx._source引用源文档
- lang 脚本语言(默认painless es内置语言)
- params 参数
{
"script": {
"inline": "ctx._source.age = params.age",
"params": {
"age":29
}
}
}
image
删除 delete请求
删除
查询
先创建 book索引
{
"settings":{
"number_of_shards":5,
"number_of_replicas":1
},
"mappings":{
"novel":{
"properties": {
"word_count":{
"type": "integer"
},
"author": {
"type": "keyword"
},
"title": {
"type": "text"
},
"publish_date": {
"type": "date",
"format":"yyyy-MM-dd HH:mm:ss||yyy-MM-dd||epoch_millis"
}
}
}
}
}
插入测试数据
{
"author": "施耐庵",
"title": "水浒传",
"word_count" : "39880",
"public_date": "1370-04-21"
}
{
"author": "罗贯中",
"title": "三国演义",
"word_count" : "2234222",
"public_date": "1400-04-21"
}
{
"author": "吴承恩",
"title": "西游记",
"word_count" : "2233922",
"public_date": "1582-04-21"
}
{
"author": "曹雪芹",
"title": "红楼梦",
"word_count" : "833922",
"public_date": "1763-04-21"
}
简单查询
image.png
条件查询 关键词_search
- term,terms查询
- term 查询某个字段里含有某个关键词的文档
- terms 查询某个字段里含有多个关键词的文档
- term 和 terms 是 包含(contains) 操作,而非 等值(equals) (判断)
- 不知道分词器的存在,所以不会去分词
- from,size 控制查询返回的数量,类似 mysql 中 limit 操作
- fields query中的范围
{
"from":0,
"size":2,
"query":{
"terms":{
"author": ["罗","吴","曹"]
}
}
}
image.png
- match查询
- match_all 查询所有文档
- multi_match:可以指定多个字段
- match_phrase:短语匹配查询,ElasticSearch引擎首先分析(analyze)查询字符串,从分析后的文本中构建短语查询,这意味着必须匹配短语中的所有分词,并且保证各个分词的相对位置不变
post localhost:9200/book/_search
{
"query": {
"match": {
"author" : "罗中吴"
}
}
}
结果
{
"took": 20,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.219939,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 1.219939,
"_source": {
"author": "罗贯中",
"title": "三国演义",
"word_count": "2234222",
"public_date": "1400-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 0.25316024,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
post localhost:9200/book/_search
{
"query": {
"match_all": {
}
}
}
结果
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 1,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 1,
"_source": {
"author": "罗贯中",
"title": "三国演义",
"word_count": "2234222",
"public_date": "1400-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1,
"_source": {
"author": "曹雪芹",
"title": "红楼梦",
"word_count": "833922",
"public_date": "1763-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 1,
"_source": {
"author": "施耐庵",
"title": "水浒传",
"word_count": "39880",
"public_date": "1370-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 1,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
post localhost:9200/book/_search
{
"query": {
"multi_match":{
"query": "吴红",
"fields": ["author","title"] //查询的字段为 author,title
}
}
}
结果
{
"took": 16,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 0.6548752,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 0.6548752,
"_source": {
"author": "曹雪芹",
"title": "红楼梦",
"word_count": "833922",
"public_date": "1763-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 0.25316024,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
post localhost:9200/book/_search
{
"query": {
"match_phrase": {
"author":"曹雪"
}
}
}
结果
{
"took": 15,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.219939,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1.219939,
"_source": {
"author": "曹雪芹",
"title": "红楼梦",
"word_count": "833922",
"public_date": "1763-04-21"
}
}
]
}
}
- 指定返回的字段
- _source 设置返回字段
- 显示要的字段或去除不需要的字段、可以使用通配符*
- includes 包含
- excludes 排除
post localhost:9200/book/_search
{
"_source":["author","word_count"],
"query": {
"match_phrase": {
"author":"曹雪"
}
}
}
结果
{
"took": 31,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.219939,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1.219939,
"_source": {
"word_count": "833922",
"author": "曹雪芹"
}
}
]
}
}
post localhost:9200/book/_search
{
"_source":{
"includes":"*a*",
"excludes": "tit*"
},
"query": {
"match_phrase": {
"author":"曹雪"
}
}
}
结果
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.219939,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1.219939,
"_source": {
"author": "曹雪芹",
"public_date": "1763-04-21"
}
}
]
}
}
- 排序
post localhost:9200/book/_search
{
"query": {
"match_all": {
}
},
"sort":[
{
"public_date": "desc"
}
]
}
结果
{
"took": 25,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": null,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": null,
"_source": {
"author": "曹雪芹",
"title": "红楼梦",
"word_count": "833922",
"public_date": "1763-04-21"
},
"sort": [
-6522768000000
]
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": null,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
},
"sort": [
-12234585600000
]
},
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": null,
"_source": {
"author": "罗贯中",
"title": "三国演义",
"word_count": "2234222",
"public_date": "1400-04-21"
},
"sort": [
-17977939200000
]
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": null,
"_source": {
"author": "施耐庵",
"title": "水浒传",
"word_count": "39880",
"public_date": "1370-04-21"
},
"sort": [
-18924624000000
]
}
]
}
}
- 语法查询
- query_string 查询解析输入并在运算符周围分割文本
post localhost:9200/book/_search
{
"query":{
"query_string": {
"query": "吴 AND 恩"
}
}
}
结果
{
"took": 21,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.56977004,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 0.56977004,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
localhost:9200/book/_search
{
"query":{
"query_string": {
"query": "(吴 AND 恩) OR 水浒传"
}
}
}
结果
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 0.854655,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 0.854655,
"_source": {
"author": "施耐庵",
"title": "水浒传",
"word_count": "39880",
"public_date": "1370-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 0.56977004,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
post localhost:9200/book/_search
{
"query":{
"query_string": {
"query": "(吴 AND 恩) OR 水浒传",
"fields":["author"] //设定查询范围
}
}
}
结果
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.5063205,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 0.5063205,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
- range:实现范围查询
- 参数:from,to,include_lower,include_upper,boost
- include_lower:是否包含范围的左边界,默认是true
- include_upper:是否包含范围的右边界,默认是true
- gt大于,gte大于或等于,lt小于,lte小于或等于 与mybatisPlus语法一致不做演示
-
boost 搜索条件的权重,boost,可以将某个搜索条件的权重加大- 当匹配这个搜索条件和匹配另一个搜索条件的document知识点计算relevance score时,匹配权重更大的搜索条件的document,relevance score会更高,当然也就会优先被返回回来
- 默认情况下,搜索条件的权重都是一样的,都是1
post localhost:9200/book/_search
{
"query":{
"range": {
"word_count": {
"from": 223,
"to": 2233922,
"include_lower": true, // 包含223这个值
"include_upper": true //包含2233922这个值
}
}
}
}
结果
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 1,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
}
}
post localhost:9200/book/_search
{
"query":{
"range": {
"word_count": {
"from": 223,
"to": 2233922,
"include_lower": true,
"include_upper": false //不包含2233922
}
}
}
}
结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
- filter 简单过滤查询
post localhost:9200/book/_search
{
"query": {
"bool": {
"filter": {
"term": {
"public_date": "1400-04-21"
}
}
}
}
}
wildcard 查询fuzzy实现模糊查询高亮搜索结果-
符合查询
以上几种查询不一一列举
聚合查询
post localhost:9200/book/_search
{
"aggs":{ //聚合关键字
"group_by_word_count":{ //名称,自定义
"terms":{
"field":"word_count" //要统计的列
}
}
}
结果
{
"took": 28,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 1,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 1,
"_source": {
"author": "罗贯中",
"title": "三国演义",
"word_count": "2234222",
"public_date": "1400-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1,
"_source": {
"author": "曹雪芹",
"title": "红楼梦",
"word_count": "833922",
"public_date": "1763-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 1,
"_source": {
"author": "施耐庵",
"title": "水浒传",
"word_count": "39880",
"public_date": "1370-04-21"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 1,
"_source": {
"author": "吴承恩",
"title": "西游记",
"word_count": "2233922",
"public_date": "1582-04-21"
}
}
]
},
"aggregations": {
"group_by_word_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": -18924624000000,
"key_as_string": "1370-04-21T00:00:00.000Z",
"doc_count": 1
},
{
"key": -17977939200000,
"key_as_string": "1400-04-21T00:00:00.000Z",
"doc_count": 1
},
{
"key": -12234585600000,
"key_as_string": "1582-04-21T00:00:00.000Z",
"doc_count": 1
},
{
"key": -6522768000000,
"key_as_string": "1763-04-21T00:00:00.000Z",
"doc_count": 1
}
]
}
}
}