EFK集群:elasticsearch + filebeat + kibana, 集群主要指的是elasticsearch(之后用es代指)需要做个集群,这里采用三台es服务器做集群。
安装:
efk需要用同一版本:
es安装:需要在三台机器上都装下。
mkdir -p /opt/software && cd /opt/software
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.2-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz
mv elasticsearch-7.3.2 /opt/elasticsearch
useradd elasticsearch
mkdir -p /opt/logs/elasticsearch
chown elasticsearch.elasticsearch /opt/elasticsearch -R
chown elasticsearch.elasticsearch /opt/logs/elasticsearch -R
echo " * soft nofile 65536 " >> /etc/security/limits.conf
echo " * hard nofile 65536 " >> /etc/security/limits.conf
echo "vm.max_map_count = 655350" >> /etc/sysctl.conf
sysctl -p
filebeat 安装:
cd /opt/software
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.3.2-linux-x86_64.tar.gz
mkdir -p /opt/logs/filebeat/
tar -zxvf filebeat-7.3.2-linux-x86_64.tar.gz
mv filebeat-7.3.2-linux-x86_64 /opt/filebeat
kibana 安装:
cd /opt/software
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.3.2-linux-x86_64.tar.gz
tar -zxvf kibana-7.3.2-linux-x86_64.tar.gz
mv kibana-7.3.2-linux-x86_64 /opt/kibana
useradd kibana
chown kibana.kibana /opt/kibana -R
配置:
es 配置文件修改:
vim config/elasticsearch.yml
cluster.name: my-application #集群名称,三台es必须一致
node.name: node-2 #节点名称
path.data: /opt/logs/elasticsearch #日志位置
network.host: 0.0.0.0 #访问地址
http.port: 9200 #端口号
discovery.zen.ping.unicast.hosts: ["192.168.0.100", "192.168.0.5", "192.168.0.9"] #三台es服务器的ip地址
kibana配置文件修改:
vi config/kinaba.yml :
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://localhost:9200"
kibana.index: ".kibana"
filebeat配置文件修改:
filebeat.prospectors:
- type: log
enabled: true
paths:
- /var/log/*.log
- /var/log/*.out
multiline.pattern: ^\[
multiline.negate: true
multiline.match: after
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
setup.kibana:
host: "192.168.0.9:5601"
output.elasticsearch:
hosts: ["localhost:9200", "192.168.0.9:9200", "192.168.0.100"] #三台es服务器地址
启动:
(1)、es服务器启动(三台都要启):
需要切换到普通用户:su elasticsearch
切换到elasticsearch 目录: nohup ./bin/elasticsearch &
(2)、kibana启动:
切换到kibana目录: nohup ./bin/kibana &
(3)、filebeat启动:
切换到filebeat 目录:
nohup ./filebeat -c -e filebeat.yml &