ELK中常见的命令整理如下：
ES
说明：REST API调用方式为<REST Verb> /<Index>/<Type>/<ID>
查看ES中的index信息： GET /_cat/indices?v
查看集群的健康状态： GET /_cat/health?v
查看集群的节点信息： GET /_cat/nodes?v
创建index：
PUT /customer?pretty
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "customer"
}
PUT /customer/_doc/1?pretty
{
"name": "xi dada"
}
- 删除index：
DELETE /customer?pretty
DELETE /customer/_doc/1?pretty
- 更新index：
POST /customer/_doc/1/_update?pretty
{
"doc": {"name":"wang FengFeng", "age":28}
}
POST /customer/_doc/1/_update?pretty
{
"script": "ctx._source.age += 3"
}
- 查询index信息：
GET /customer/_doc/2?pretty
GET /kafka-snail/_search?pretty&q=* 某个index中所有的记录 - 复杂查询index信息+条件：
GET /bank/_search?q=*&sort=account_number:asc&pretty
GET /bank/_search
{
"query": { "match_all": {} },
"sort": [
{ "account_number": "asc" }
]
}
// 指定查询结果的大小和起始位置
GET /bank/_search
{
"query": {"match_all": {}},
"_source": ["account_number","balance"],
"sort": [
{ "account_number": "asc" },
{"balance": "desc"}
],
"from":10,
"size": 5
}
// 指定多个查询条件，包含与、或、非
GET /bank/_search
{
"query": {
"bool": {
"must":[
{"match":{"address":"mill"}},
{"match":{"gender":"M"}}
],
"must_not":[
{"match":{"state":"IL"}}
],
"should": [
{"match": {"city": "Lopezo"}},
{"match": {"city": "Urie"}}
]
}
}
}
//使用where条件，限定字段的范围
GET /bank/_search
{
"query": {
"bool": {
"must": { "match": {"gender":"M"} },
"filter": {
"range": {
"balance": {
"gte": 20000,
"lte": 23000
}
}
}
}
}
}
- 批量导入或者更新index：
参见官网 https://www.elastic.co/guide/en/elasticsearch/reference/6.4/docs-bulk.html
// 批量操作
POST /customer/_doc/_bulk?pretty
{"index":{"_id":"1"}}
{"name":"John legend"}
{"index":{"_id":"2"}}
{"name":"wang jun"}
POST /customer/_doc/_bulk?pretty
{"update":{"_id":"1"}}
{"doc":{"name":"John legend become mengmeng"}}
{"delete":{"_id":"2"}}
- 按indices查看缓存 GET /_stats/request_cache?human
- 按节点查看缓存 GET /_nodes/stats/indices/request_cache?human
- 清理多个index的缓存 POST /index1,inex2/_cache/clear?request=true
Logstash:
- 查看logstash使用的插件信息和版本
./bin/logstash-plugin list --verbose - 卸载logstash使用的某个xx插件
./bin/logstash-plugin remove xx - logstash安装指定版本的插件
./bin/logstash-plugin install --version 8.0.6 xx 例如：./bin/logstash-plugin install --version 8.0.6 logstash-input-kafka - 简单控制台输入输出
./bin/logstash -e 'input { stdin {}} output { stdout { } }' - 配置好Logstash.conf，检查其配置项
./bin/logstash -f logstash.conf --config.test_and_exit - 配置好Logstash.conf，检查其配置项
./bin/logstash -f logstash.conf --config.test_and_exit - 启动logstash，当配置文件修改时，可自动加载
./bin/logstash -f logstash.conf --config.reload.automatic - logstash读取kafka数据存入到ES中的配置kafka-logstash.conf
input {
kafka {
bootstrap_servers => "10.194.xxx.yyy:9092,10.xxx.yyy.18:9092,10.xxx.yyy.180:9092"
auto_offset_reset => "earliest"
group_id => "logstash23"
id => "8.0.6"
client_id => "logstash-5"
check_crcs => "false"
topics => ["mda.online"]
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
geoip {
source => "clientip"
}
}
output {
stdout { } //会打印出信息
elasticsearch {
hosts => "esIp:9200"
index => "kafka-snail"
template_overwrite => true
}
}
参考文献
