elasticsearch学习

什么是elasticsearch

  • 基于apache lucene 构建的开源搜索引擎
  • 采用java 编写,提供简单易用的RESTFul API
  • 轻松的横向拓展,可支持PB级的结构化或非结构化数据处理

应用场景

  • 海量数据分析引擎
  • 站内搜索引擎
  • 数据仓库

单实例安装

插件安装(为了更直观的使用es)

  • 安装node brew install -g node 插件需要在6.0以上的node环境运行
  • 下载插件 git clone https://github.com/mobz/elasticsearch-head.git
  • cd elasticsearch-head/
  • npm install 仅第一次需要安装
  • npm run start 运行head插件
  • 插件会在9100启动,elasticsearch在9200启动 是跨域请求,需要修改一下配置
  • cd **/elasticsearch-5.5.2/config
  • vim elasticsearch.yml添加以下配置
    • http.cors.enabled: true
    • http.cors.allow-origin: "*"
  • sh ../bin/elasticsearch –d 后台启动es
  • cd ../elasticsearch-head/
  • npm run start
  • 访问 localhost:9200 效果
    image

多实例安装

  • mkdir es_cluster
  • cp ../elasticsearch-5.5.2.gz ./
  • tar -zxvf elasticsearch-5.5.2.gz
  • cp -r elasticsearch-5.5.2 es_master
  • cp -r elasticsearch-5.5.2 es_slave1 从1
  • cp -r elasticsearch-5.5.2 es_slave2 从2
  • rm -rf elasticsearch-5.5.2* 删除多余的包
  • vim es_master/config/elasticsearch.yml修改主配置
    • http.cors.enabled: true
    • http.cors.allow-origin: "*"
    • cluster.name: ibest #集群名称
    • node.name: master #结点名称
    • node.master: true #指定为主结点
  • 关闭之前启动单实例启动多实例master
    • lsof -i :9200
    • kill * * 对应9200的id
    • ./es_master/bin/elasticsearch -d
  • vim es_slave1/config/elasticsearch.yml修改从1配置
    • cluster.name: ibest #集群名称,必须和主结点一致,否则无法加入集群
    • node.name: slave1 #结点名称
    • network.host: 127.0.0.1 #和其他结点通信的地址
    • http.port: 8200 #9200已经被主结点占用
    • discovery.zen.ping.unicast.hosts: ["127.0.0.1"]#发现主结点
  • ./es_slave1/bin/elasticsearch -d 启动从1
  • vim es_slave2/config/elasticsearch.yml修改从2配置
    • cluster.name: ibest #集群名称,必须和主结点一致,否则无法加入集群
    • node.name: slave2 #结点名称
    • network.host: 127.0.0.1 #和其他结点通信的地址
    • http.port: 7200 #9200已经被主结点占用
    • discovery.zen.ping.unicast.hosts: ["127.0.0.1"]#发现主结点
  • ./es_slave2/bin/elasticsearch -d 启动从2
  • 最终效果
    image

基础概念

  • 将es与mysql 联系起来
es mysql
索引 database
类型 table
文档id 一条记录的id
  • API基本格式 http://<ip>:<port>/<索引>/<类型>/<文档id>。请求方式:Get、Post、Put、Delete。
  • es索引为结构化索引非结构化索引
  • settings:修改,设置分片和副本数的
    • number_of_shards:分片数
    • number_of_replicas:备份数
  • mappings:修改,设置字段和类型
    • properties:定义type的属性值
      • type: 数据类型定义
      • analyzer: 定义分析器

创建索引

json数据如下

   "settings":{
       "number_of_shards":5,
       "number_of_replicas":1
   },
   "mappings":{
       "man":{
           "properties": {
               "name":{
                   "type": "text"
               },
               "country": {
                   "type": "keyword"
               },
               "age": {
                   "type": "integer"
               },
               "date": {
                   "type": "date",
                   "format":"yyyy-MM-dd HH:mm:ss||yyy-MM-dd||epoch_millis"
               }
           }
       },
       "woman": {
           
       }
   }
   
} 
image

image

image

插入

  • 指定ID插入 put请求,id自定义
{
    "name": "jack",
    "age": 18,
    "country": "China",
    "date": "2020-04-21"
    
}
image
  • 自动产生文档ID post请求,id自动生成
{
    "name": "tom",
    "age": 11,
    "country": "China",
    "date": "2001-04-21"
    
}

[站外图片上传中...(image-907979-1587472414044)]


image

修改 关键词_update

  • 直接修改文档
{
 "doc": {
    "name": "jak_bus"
 }
    
}
image
  • 脚本修改文档
    • ctx._source引用源文档
    • lang 脚本语言(默认painless es内置语言)
    • params 参数
{
  "script": {
    "inline":  "ctx._source.age = params.age",
  "params": {
    "age":29
  }
  }

}
image

删除 delete请求

删除

查询

先创建 book索引

{  
    "settings":{
       "number_of_shards":5,
       "number_of_replicas":1
   },
   "mappings":{
       "novel":{
           "properties": {
               "word_count":{
                   "type": "integer"
               },
               "author": {
                   "type": "keyword"
               },
               "title": {
                   "type": "text"
               },
               "publish_date": {
                   "type": "date",
                   "format":"yyyy-MM-dd HH:mm:ss||yyy-MM-dd||epoch_millis"
               }
           }
       }
   }
   
}

插入测试数据

{
    "author": "施耐庵",
    "title": "水浒传",
    "word_count" : "39880",
    "public_date": "1370-04-21"
}
{
    "author": "罗贯中",
    "title": "三国演义",
    "word_count" : "2234222",
    "public_date": "1400-04-21"
}
{
    "author": "吴承恩",
    "title": "西游记",
    "word_count" : "2233922",
    "public_date": "1582-04-21"
}
{
    "author": "曹雪芹",
    "title": "红楼梦",
    "word_count" : "833922",
    "public_date": "1763-04-21"
}

简单查询

image.png

条件查询 关键词_search

  • term,terms查询
    • \color{red}{term,terms 无视分词器,这种查询适合keyword 、numeric、date}
    • term 查询某个字段里含有某个关键词的文档
    • terms 查询某个字段里含有多个关键词的文档
      • term 和 terms 是 包含(contains) 操作,而非 等值(equals) (判断)
      • 不知道分词器的存在,所以不会去分词
    • from,size 控制查询返回的数量,类似 mysql 中 limit 操作
    • fields query中的范围
{
    "from":0,
    "size":2,
    "query":{
        "terms":{
            "author": ["罗","吴","曹"]
        }
    }
}
image.png
  • match查询
    • \color{red}{match 知道分词器的存在,会对field进行分词操作,然后再查询}
    • match_all 查询所有文档
    • multi_match:可以指定多个字段
    • match_phrase:短语匹配查询,ElasticSearch引擎首先分析(analyze)查询字符串,从分析后的文本中构建短语查询,这意味着必须匹配短语中的所有分词,并且保证各个分词的相对位置不变
post localhost:9200/book/_search
{
    "query": {
        "match": {
            "author" : "罗中吴"
        }
    }
}
结果 
{
    "took": 20,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.219939,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "2",
                "_score": 1.219939,
                "_source": {
                    "author": "罗贯中",
                    "title": "三国演义",
                    "word_count": "2234222",
                    "public_date": "1400-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 0.25316024,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "query": {
        "match_all": {
        
        }
    }
}
结果
{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 4,
        "max_score": 1,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "author": "罗贯中",
                    "title": "三国演义",
                    "word_count": "2234222",
                    "public_date": "1400-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 1,
                "_source": {
                    "author": "曹雪芹",
                    "title": "红楼梦",
                    "word_count": "833922",
                    "public_date": "1763-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "author": "施耐庵",
                    "title": "水浒传",
                    "word_count": "39880",
                    "public_date": "1370-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "query": {
        "multi_match":{
            "query":  "吴红",
            "fields": ["author","title"]  //查询的字段为  author,title
        }
        
    }
}
结果
{
    "took": 16,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 0.6548752,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 0.6548752,
                "_source": {
                    "author": "曹雪芹",
                    "title": "红楼梦",
                    "word_count": "833922",
                    "public_date": "1763-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 0.25316024,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "query": {
        "match_phrase": {
            "author":"曹雪"
        }
        
    }
}
结果
{
    "took": 15,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.219939,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 1.219939,
                "_source": {
                    "author": "曹雪芹",
                    "title": "红楼梦",
                    "word_count": "833922",
                    "public_date": "1763-04-21"
                }
            }
        ]
    }
}
  • 指定返回的字段
    • _source 设置返回字段
    • 显示要的字段或去除不需要的字段、可以使用通配符*
      • includes 包含
      • excludes 排除
post localhost:9200/book/_search
{
    "_source":["author","word_count"],
    "query": {
        "match_phrase": {
            "author":"曹雪"
        }
        
    }
}
结果
{
    "took": 31,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.219939,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 1.219939,
                "_source": {
                    "word_count": "833922",
                    "author": "曹雪芹"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "_source":{
        "includes":"*a*",
        "excludes": "tit*"
    },
    "query": {
        "match_phrase": {
            "author":"曹雪"
        }
        
    }
}
结果 
{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.219939,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 1.219939,
                "_source": {
                    "author": "曹雪芹",
                    "public_date": "1763-04-21"
                }
            }
        ]
    }
}
  • 排序
post localhost:9200/book/_search
{
    
    "query": {
        "match_all": {
            
        }
    },
    "sort":[
                {
                    "public_date": "desc"
                }
            ]
        
}
结果
{
    "took": 25,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 4,
        "max_score": null,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": null,
                "_source": {
                    "author": "曹雪芹",
                    "title": "红楼梦",
                    "word_count": "833922",
                    "public_date": "1763-04-21"
                },
                "sort": [
                    -6522768000000
                ]
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": null,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                },
                "sort": [
                    -12234585600000
                ]
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "2",
                "_score": null,
                "_source": {
                    "author": "罗贯中",
                    "title": "三国演义",
                    "word_count": "2234222",
                    "public_date": "1400-04-21"
                },
                "sort": [
                    -17977939200000
                ]
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "1",
                "_score": null,
                "_source": {
                    "author": "施耐庵",
                    "title": "水浒传",
                    "word_count": "39880",
                    "public_date": "1370-04-21"
                },
                "sort": [
                    -18924624000000
                ]
            }
        ]
    }
}
  • 语法查询
    • query_string 查询解析输入并在运算符周围分割文本
post localhost:9200/book/_search
{
    "query":{
        "query_string": {
            "query": "吴 AND 恩"
        }
    }
}
结果
{
    "took": 21,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 0.56977004,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 0.56977004,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
localhost:9200/book/_search
{
    "query":{
        "query_string": {
            "query": "(吴 AND 恩) OR 水浒传"
        }
    }
}
结果
{
    "took": 10,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 0.854655,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "1",
                "_score": 0.854655,
                "_source": {
                    "author": "施耐庵",
                    "title": "水浒传",
                    "word_count": "39880",
                    "public_date": "1370-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 0.56977004,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "query":{
        "query_string": {
            "query": "(吴 AND 恩) OR 水浒传",
            "fields":["author"] //设定查询范围
        
        }           
    }
}

结果
{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 0.5063205,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 0.5063205,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
  • range:实现范围查询
    • 参数:from,to,include_lower,include_upper,boost
    • include_lower:是否包含范围的左边界,默认是true
    • include_upper:是否包含范围的右边界,默认是true
    • gt大于,gte大于或等于,lt小于,lte小于或等于 与mybatisPlus语法一致不做演示
    • boost 搜索条件的权重,boost,可以将某个搜索条件的权重加大
      • 当匹配这个搜索条件和匹配另一个搜索条件的document知识点计算relevance score时,匹配权重更大的搜索条件的document,relevance score会更高,当然也就会优先被返回回来
      • 默认情况下,搜索条件的权重都是一样的,都是1
post localhost:9200/book/_search
{
    "query":{
        "range": {
            "word_count": {
                "from": 223,
                "to": 2233922,
                "include_lower": true, // 包含223这个值
                "include_upper": true //包含2233922这个值
            }
        }
    }
}

结果
{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    }
}
post localhost:9200/book/_search
{
    "query":{
        "range": {
            "word_count": {
                "from": 223,
                "to": 2233922,
                "include_lower": true,
                "include_upper": false //不包含2233922
            }
        }
    }
}
结果
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 0,
        "max_score": null,
        "hits": []
    }
}
  • filter 简单过滤查询
post localhost:9200/book/_search

{
  "query": {
    "bool": {
      "filter": {
        "term": {
            "public_date": "1400-04-21"
        }
      }
}
}
}

  • wildcard 查询
  • fuzzy实现模糊查询
  • 高亮搜索结果
  • 符合查询
    以上几种查询不一一列举

聚合查询

post localhost:9200/book/_search
{
    "aggs":{  //聚合关键字
        "group_by_word_count":{ //名称,自定义
            "terms":{
                "field":"word_count" //要统计的列
            }
        }
}
结果
{
    "took": 28,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 4,
        "max_score": 1,
        "hits": [
            {
                "_index": "book",
                "_type": "novel",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "author": "罗贯中",
                    "title": "三国演义",
                    "word_count": "2234222",
                    "public_date": "1400-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "4",
                "_score": 1,
                "_source": {
                    "author": "曹雪芹",
                    "title": "红楼梦",
                    "word_count": "833922",
                    "public_date": "1763-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "author": "施耐庵",
                    "title": "水浒传",
                    "word_count": "39880",
                    "public_date": "1370-04-21"
                }
            },
            {
                "_index": "book",
                "_type": "novel",
                "_id": "3",
                "_score": 1,
                "_source": {
                    "author": "吴承恩",
                    "title": "西游记",
                    "word_count": "2233922",
                    "public_date": "1582-04-21"
                }
            }
        ]
    },
    "aggregations": {
        "group_by_word_count": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": -18924624000000,
                    "key_as_string": "1370-04-21T00:00:00.000Z",
                    "doc_count": 1
                },
                {
                    "key": -17977939200000,
                    "key_as_string": "1400-04-21T00:00:00.000Z",
                    "doc_count": 1
                },
                {
                    "key": -12234585600000,
                    "key_as_string": "1582-04-21T00:00:00.000Z",
                    "doc_count": 1
                },
                {
                    "key": -6522768000000,
                    "key_as_string": "1763-04-21T00:00:00.000Z",
                    "doc_count": 1
                }
            ]
        }
    }
}
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 220,976评论 6 513
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 94,249评论 3 396
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 167,449评论 0 360
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 59,433评论 1 296
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 68,460评论 6 397
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 52,132评论 1 308
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 40,721评论 3 420
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 39,641评论 0 276
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 46,180评论 1 319
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 38,267评论 3 339
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 40,408评论 1 352
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 36,076评论 5 347
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 41,767评论 3 332
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 32,255评论 0 23
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 33,386评论 1 271
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 48,764评论 3 375
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 45,413评论 2 358

推荐阅读更多精彩内容