Elasticsearch笔记--聚合统计

记录Elasticsearch使用过程中印象深刻的语法结构

基础语法

查看索引

curl 'http://192.168.163.126:9200/_cat/indices?v'

查看安装的插件

curl 'http://192.168.163.126:9200/_cat/plugins'

创建索引指定分片

curl -XPUT 'http://192.168.163.126:9200/mytestindex?pretty' -H 'Content-Type: application/json' -d'{"settings" : {"number_of_shards":3,"number_of_replicas" : 1,"refresh_interval":"3s"}}'

创建索引指定分片和字段类型为keyword

curl -XPUT 'http://192.168.163.126:9200/mytestindex_test?pretty' -H 'Content-Type: application/json' -d 
'{
    "mappings": {
        "default": {
            "properties": {
                "id": {
                    "type": "keyword"
                },
                "name": {
                    "type": "keyword"
                },
                "createTime": {
                    "type": "date"
                }
            }
        }
    },
    "order": 0,
    "settings": {
        "number_of_replicas": 1,
        "number_of_shards": 3,
        "refresh_interval": "1s"
    }}'

分词

创建索引指定分片并指定IK分词器，并指定分词字段映射：contentdata字段值分词

curl -XPUT 'http://192.168.163.126:9200/mytestindex_test?pretty' -H 'Content-Type: application/json' -d '{"mappings":{"mytestindex_test":{"dynamic": true,"properties": {"contentdata": {"analyzer": "ik_max_word","type": "text"}}}},"settings" : {"number_of_shards":3,"number_of_replicas" : 0,"refresh_interval":"1s","analysis": {"analyzer": {"ik": {"tokenizer": "ik_max_word"}}}}}'

创建索引指定分片、指定自定义分词器

curl -XPUT 'http://192.168.163.126:9200/mytestindex_test?pretty' -H 'Content-Type: application/json' -d '{"settings" : {"number_of_shards":3,"number_of_replicas" : 0,"refresh_interval":"1s","analysis": {"analyzer": {"myelasticsearchplugin": {"type": "custom","tokenizer": "myelasticsearchplugin-word"}}}}}'

查看索引指定数据分词效果

curl -XGET 'http://192.168.163.126:9200/mytestindex_test/mytestindex_test/1/_termvectors?fields=contentdata&pretty'

集群状态

查询集群状态

curl -XGET 'http://192.168.163.126:9200/_stats?pretty'
curl -XGET 'http://192.168.163.126:9200/_cluster/state?pretty'

查看热线程

curl -XGET 'http://192.168.163.126:9200/_nodes/hot_threads?pretty'

模板

查询所有模板

curl -XGET  'http://192.168.163.126:9200/_template?pretty'

创建模板

curl -XPUT 'http://192.168.163.126:9200/_template/mytest_tpl' -H 'Content-Type: application/json' -d '{"mappings": {"default": {"properties": {"id": {"type": "keyword"},"JDMC": {"type": "keyword"}}}},"order": 0,"settings": {"max_result_window": "2000000000","number_of_replicas": 1,"number_of_shards": 3,"refresh_interval": "1s"},"template": "mytest*"}'

查询指定模板

curl -XGET  'http://192.168.163.126:9200/_template/mytest_tpl?pretty'

删除模板

curl -XDELETE 'http://192.168.163.126:9200/_template/mytest_tpl?pretty'

创建模板前缀索引

 curl -XPUT 'http://192.168.163.126:9200/mytest_mytesttest?pretty'

聚合统计

条件过滤多字段count聚合

http://192.168.163.129:9200/mytest20190418/
{
"query": {
        "range" : {
            "sessionStartTime" : {
                "gte" : "2019-04-18T15:00:00+08:00",
                "lt" :  "2019-04-18T16:00:00+08:00"
            }
        }
    },
    "aggs": {
        "srcIp": {
            "aggs": {
                "destIp": {
                    "aggs": {
                        "corpId": {
                            "aggs": {
                                "srcPort": {
                                    "aggs": {
                                        "destPort": {
                                            "terms": {
                                                "field": "destPort",
                                                "order" : {
                                                  "_count" : "desc"
                                                }
                                            }
                                        }
                                    },
                                    "terms": {
                                        "field": "srcPort"
                                    }
                                }
                            },
                            "terms": {
                                "field": "corpId"
                            }
                        }
                    },
                    "terms": {
                        "field": "destIp"
                    }
                }
            },
            "terms": {
                "field": "srcIp",
                "order" : {
                  "_count" : "desc"
                }
            }
        }
    },
    "size": 0
}

count聚合统计

{
    "aggs": {
        "srcIpCount": {
            "terms": {
                "field": "srcIp",
                "order": {
                    "_count": "desc"
                }
            }
        }
    },
    "query": {
        "range": {
            "sessionStartTime": {
                "gte": "2019-04-18T15:00:00+08:00",
                "lt": "2019-04-18T16:00:00+08:00"
            }
        }
    },
    "size": 0
}

聚合中的条件过滤:bucket_selector -- 按count值过滤

{
  "size": 0,
  "aggs": {
    "srcIpCount": {
      "terms": {
        "field": "srcIp"
      },
      "aggs": {
        "having": {
          "bucket_selector": {
            "buckets_path": {
              "srcIpCount": "_count"
            },
            "script": {
              "source": "params.srcIpCount > 520026"
            }
          }
        },
        "destIp": {
                    "terms": {
                        "field": "destIp"
                    }
                }
      }
    }
  }
}

sum聚合

{
  "size": 1,
  "aggs": {
    "destPortSum": {
      "sum": {
        "field": "destPort"
      }
      }
    }
}

sum统计，按聚合结果排序过滤

{
    "aggs": {
        "srcIpCount": {
            "aggs": {
                "srcPortSum_filter": {
                    "bucket_selector": {
                        "buckets_path": {
                            "totalSales": "srcPortSum"
                        },
                        "script": "params.totalSales > 2792119599L"
                    }
                },
                "srcPortSum": {
                    "sum": {
                        "field": "srcPort"
                        
                    }
                }
            },
            "terms": {
                "field": "srcIp",
                "order": {
                            "srcPortSum": "desc"
                        }
            }
        }
    },
    "size": 0
}

都是项目中实际应用到的Elasticsearch统计语法，放这里看看哪位有缘的道友也入这个坑
原创不易，转载请注明出处