日志格式示例
- tomcat access log示例
18.18.18.18 - - [04/May/2017:15:37:48 +0800] "POST /api/cashie/module/delivery HTTP/1.1" 200 244 75
配置logstash
logstash 配置文件
input {
beats {
port => 5044
}
}
filter {
grok {
patterns_dir => ["/etc/logstash/patterns"]
match => {
"message" => "%{COMMONAPACHELOG}"
}
}
#将日志中的时间替换成@timestamp
date {
match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
}
}
添加pattern_dir /etc/logstash/patterns/httpd 文件
HTTPDUSER %{EMAILADDRESS}|%{USER}
HTTPDERROR_DATE %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}
HTTPD_COMMONLOG %{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:%{NUMBER:res_time}|-)
HTTPD_COMBINEDLOG %{HTTPD_COMMONLOG} %{QS:referrer} %{QS:agent}
HTTPD20_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:message}
HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{POSINT:pid}:tid %{NUMBER:tid}\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_message}:)?( \[client %{IPORHOST:clientip}:%{POSINT:clientport}\])? %{DATA:errorcode}: %{GREEDYDATA:message}
HTTPD_ERRORLOG %{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}
COMMONAPACHELOG %{HTTPD_COMMONLOG}
COMBINEDAPACHELOG %{HTTPD_COMBINEDLOG}
配置filebeat
tomcat日志多行日志合并
filebeat.prospectors:
- input_type: log
#修改为tomcat配置的日志路径
paths:
- /opt/tomcat7/logs/access.log
output.logstash:
hosts: ["18.18.18.18:5046"]
配置完成后可以分析类似格式的tomcat日志。
