搭建Prometheus Grafana Go监控平台

一、前言

监控指标

Go监控指标有好多库，监控指标大致一样。
1）https://github.com/bmhatfield/go-runtime-metrics
2）https://github.com/tevjef/go-runtime-metrics
3）Prometheus SDK

指标输出

1）golang expvar
2）Prometheus

二、Prometheus搭建

镜像设置

docker pull mirrorgooglecontainers/addon-resizer:1.8.3
docker tag mirrorgooglecontainers/addon-resizer:1.8.3 k8s.gcr.io/addon-resizer:1.8.3

docker pull mirrorgooglecontainers/heapster-grafana-amd64:v5.0.4
docker tag mirrorgooglecontainers/heapster-grafana-amd64:v5.0.4 k8s.gcr.io/heapster-grafana-amd64:v5.0.4

prometheus部署

kubectl apply -f alertmanager
kubectl apply -f prometheus
kubectl apply -f kube-state-metrics
kubectl apply -f node-exporter
kubectl apply -f grafana

检查

kubectl get pod -n monitoring
kubectl get svc -n monitoring

三、Grafana搭建

数据源配置

image.png

Dashboard配置

https://grafana.com/grafana/dashboards/6671

四、效果展示

自定义job

image.png

查看指标

image.png

dashboard展示pod信息

image.png

实现

prometheus参考Thakurvaibhav的作品，并进行改造。
demo项目使用prometheus-client

代码见：
https://github.com/huiwq1990/OpenShare/tree/master/monitoring

参考

https://medium.com/faun/production-grade-kubernetes-monitoring-using-prometheus-78144b835b60

https://github.com/Thakurvaibhav/k8s/blob/master/monitoring/README.md

https://jeremyxu2010.github.io/2018/08/%E7%A0%94%E7%A9%B6%E7%9B%91%E6%8E%A7%E7%B3%BB%E7%BB%9F%E4%B9%8Bprometheus/

https://github.com/giantswarm/prometheus

https://o-my-chenjian.com/2018/01/27/Deploy-Prometheus-And-Grafana-On-Kubernetes/