1. 数据采集
Telegraf 是一个用 Go 编写的代理程序,可收集系统和服务的统计数据,并写入到 InfluxDB 数据库,Telegraf 具有内存占用小的特点,通过插件系统开发人员可轻松添加支持其他服务的扩展。
下载:
Telegraf 可以从这里下载,也可以到开源软件清华大学镜像站下载。
安装
[root@m1 ~]# rpm -ivh telegraf-1.2.1.x86_64.rpm
配置
[root@m01 ~]# vi /etc/telegraf/telegraf.conf
配置输入源
cpu,硬盘,内核,内存等基本信息
[[inputs.cpu]]
## Whether to report per-cpu stats or not
percpu = true
## Whether to report total system cpu stats or not
totalcpu = true
## If true, collect raw CPU time metrics.
collect_cpu_time = false
[[inputs.disk]]
## By default, telegraf gather stats for all mountpoints.
## Setting mountpoints will restrict the stats to the specified mountpoints.
# mount_points = ["/"]
## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually
## present on /run, /var/run, /dev/shm or /dev).
ignore_fs = ["tmpfs", "devtmpfs"]
[[inputs.diskio]]
## By default, telegraf will gather stats for all devices including
## disk partitions.
## Setting devices will restrict the stats to the specified devices.
# devices = ["sda", "sdb"]
## Uncomment the following line if you need disk serial numbers.
# skip_serial_number = false
[[inputs.kernel]]
# no configuration
[[inputs.mem]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]
配置其他输入源
如:zookeeper、kafka、MQTT、redis等,这里通过jolokia
连接SpringBoot应用
# [[inputs.redis]]
# ## specify servers via a url matching:
# ## [protocol://][:password]@address[:port]
# ## e.g.
# ## tcp://localhost:6379
# ## tcp://:password@192.168.99.100
# ## unix:///var/run/redis.sock
# ##
# ## If no servers are specified, then localhost is used as the host.
# ## If no port is specified, 6379 is used
# servers = ["tcp://localhost:6379"]
[[inputs.jolokia]]
#SpringBoot内配置的jolokia管理地址
context = "/jolokia/"
#服务器配置可以有多个
[[inputs.jolokia.servers]]
#应用名称
name = "web60"
#应用主机
host = "192.168.1.60"
#应用主机端口
port = "9090"
#如果SpringBoot内配置的jolokia管理地址需要授权访问需配置用户密码
#username = "myuser"
#password = "mypassword"
[[inputs.jolokia.servers]]
name = "web61"
host = "192.168.1.61"
port = "9090"
#username = "myuser"
#password = "mypassword"
[[inputs.jolokia.metrics]]
#监控堆内存
name = "heap_memory_usage"
mbean = "java.lang:type=Memory"
attribute = "HeapMemoryUsage"
[[inputs.jolokia.metrics]]
#监控应用线程
name = "thread_count"
mbean = "java.lang:type=Threading"
attribute = "TotalStartedThreadCount,ThreadCount,DaemonThreadCount,PeakThreadCount"
[[inputs.jolokia.metrics]]
#监控类加载情况
name = "class_count"
mbean = "java.lang:type=ClassLoading"
attribute = "LoadedClassCount,UnloadedClassCount,TotalLoadedClassCount"
[[inputs.jolokia.metrics]]
#监控应用的指标,参照SpringBoot文档
name = "endpoint"
mbean = "org.springframework.boot:name=metricsEndpoint,type=Endpoint"
attribute = "Data"
配置输出源
Telegraf 支持多种输出源:influxdb,cloudwatch,datadog,file,graphite等,这里使用influxdb
[[outputs.influxdb]]
# 数据库地址
urls = ["http://localhost:8086"] # required
# 数据库名称
database = "telegraf" # required
## Retention policy to write to. Empty string writes to the default rp.
retention_policy = ""
## Write consistency (clusters only), can be: "any", "one", "quorum", "all"
write_consistency = "any"
## Write timeout (for the InfluxDB client), formatted as a string.
## If not provided, will default to 5s. 0s means no timeout (not recommended).
timeout = "5s"
# username = "telegraf"
# password = "metricsmetricsmetricsmetrics"
## Set the user agent for HTTP POSTs (can be useful for log differentiation)
# user_agent = "telegraf"
## Set UDP payload size, defaults to InfluxDB UDP Client default (512 bytes)
# udp_payload = 512
## Optional SSL Config
# ssl_ca = "/etc/telegraf/ca.pem"
# ssl_cert = "/etc/telegraf/cert.pem"
# ssl_key = "/etc/telegraf/key.pem"
## Use SSL but skip chain & host verification
# insecure_skip_verify = false
启动/停止
[root@m01 ~]# service telegraf start|stop|restart|status
2. 数据存储
InfluxDB 是一个开源分布式时序、事件和指标数据库。使用 Go 语言编写,无需外部依赖。其设计目标是实现分布式和水平伸缩扩展。
三大特性:
- Time Series (时间序列):你可以使用与时间有关的相关函数(如最大,最小,求和等)
- Metrics(度量):你可以实时对大量数据进行计算
- Eevents(事件):它支持任意的事件数据
特点:
- schemaless(无结构),可以是任意数量的列
- Scalable
- min, max, sum, count, mean, median 一系列函数,方便统计
- Native HTTP API, 内置http支持,使用http读写
- Powerful Query Language 类似sql
- Built-in Explorer 自带管理工具
下载
InfluxDB可以从这里下载,也可以到开源软件清华大学镜像站下载。
安装
[root@m1 ~]# rpm -ivh influxdb-1.2.2.x86_64.rpm
配置
数据库默认占用8088,8086端口,其中8086为HTTP API端口,可以更改为其他端口。默认Web管理页面没打开,Web管理页面默认占用8083端口。
[root@m01 ~]# vi /etc/influxdb/influxdb.conf
启动
root@m02:~# service influxdb start|stop|restart|status
3. 数据展示
Grafana是 Graphite 和 InfluxDB 仪表盘和图形编辑器。Grafana 是开源的,功能齐全的度量仪表盘和图形编辑器,支持 Graphite,InfluxDB 和 OpenTSDB。
Grafana 主要特性:灵活丰富的图形化选项;可以混合多种风格;支持白天和夜间模式;多个数据源;Graphite 和 InfluxDB 查询编辑器等等。
安装
Ubuntu & Debian(64 Bit)
root@m02:~# wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_4.2.0_amd64.deb
root@m02:~# dpkg -i grafana_4.2.0_amd64.deb
Standalone Linux Binaries(64 Bit)
root@m02:~# wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.2.0.linux-x64.tar.gz
root@m02:~# tar -zxvf grafana-4.2.0.linux-x64.tar.gz
Redhat & Centos(64 Bit)
root@m02:~# wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.2.0-1.x86_64.rpm
root@m02:~# yum localinstall grafana-4.2.0-1.x86_64.rpm
Docker
root@m02:~# docker run -d --name=grafana -p 3000:3000 grafana/grafana
4. 数据来源
在SpringBoot中引入如下依赖
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-core</artifactId>
<version>1.3.5</version>
</dependency>
SpringBoot1.5.x默认打开应用安全管理,可以手动关闭,可以指定默认管理路径。如果指定了默认管理路径/mgr
,则jolokia
的访问路径变成/mgr/jolokia
,配置Telegraf
时需注意。Telegraf
下jolokia
的context= /${web-context}/mgr/jolokia
。
management:
security:
enabled: false
context-path: /mgr