生态框架

1. Exporter部署

1.1 node_exporter部署

# NOTE: Replace the URL with one from the above mentioned "downloads" page.
# <VERSION>, <OS>, and <ARCH> are placeholders.
wget https://github.com/prometheus/node_exporter/releases/download/v<VERSION>/node_exporter-<VERSION>.<OS>-<ARCH>.tar.gz
tar xvfz node_exporter-*.*-amd64.tar.gz
cd node_exporter-*.*-amd64
./node_exporter

https://github.com/prometheus/node_exporter

1.2 mysqld_exporter部署

1.创建账号并赋权
CREATE USER 'exporter'@'localhost' IDENTIFIED BY 'XXXXXXXX' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
2. 避免泄露敏感信息如账号密码在url，采用配置的方式
Sample config file：
    [client]
    user = foo
    password = foo123
3.Running using .my.cnf
./mysqld_exporter --config.my-cnf=<path_to_cnf>

https://github.com/prometheus/mysqld_exporter

1.3 elasticsearch_exporter部署

./elasticsearch_exporter --es.all --es.indices --collector.clustersettings   --es.indices_settings --es.indices_mappings --es.shards --collector.snapshots --es.timeout=10s --web.listen-address=":9114" --web.telemetry-path="/metrics" --es.uri <proto>://<user>:<password>@<host>:<port>

https://github.com/prometheus-community/elasticsearch_exporter

1.4 mongodb_exporter部署

1.创建用户
db.createUser({
    user: "username",
    pwd: "password",
    roles: [
        { role: "clusterMonitor", db: "admin" },
        { role: "read", db: "local" }
    ]
})
2.Example启动
 ./mongodb_exporter --mongodb.uri='mongodb://[user]:[password]@[ip]:[port]/admin' --compatible-mode --collect-all
–compatible-mode：兼容模式，当使用的dashboard是从grafana library上下载的模板，该模板很有可能仅支持老版本的mongodb_exporter，所以需要兼容模式才能正常显示图表等信息。
–collect-all：采集mongodb的所有信息，如果不需要所有信息，可以修改该参数为指定的采集对象，比如表，集合等

https://github.com/percona/mongodb_exporter

1.5 redis_exporter部署

1.避免泄露敏感信息如账号密码在url，采用配置的方式
sample-pwd-file.json:
{
  "redis://redis6:6379": "",
  "redis://pwd-redis5:6380": "redis-password"
}
2. Running using sample-pwd-file.json
./redis_exporter -redis.password-file=sample-pwd-file.json

https://github.com/oliver006/redis_exporter

1.6 cadvisor部署

VERSION=v0.36.0 # use the latest release version from https://github.com/google/cadvisor/releases
sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  --privileged \
  --device=/dev/kmsg \
  gcr.io/cadvisor/cadvisor:$VERSION

https://github.com/google/cadvisor

1.7 其他Exporters接入

https://cloud.tencent.com/document/product/1416/56035

2 Prometheus部署

2.1 prometheus.yml配置

global:
  scrape_interval: 15s

scrape_configs:
- job_name: node
  static_configs:
  - targets: ['localhost:9100']

- job_name: mysql
  static_configs:
  - targets: ['localhost:9104']

- job_name: es
  static_configs:
  - targets: ['localhost:9114']

- job_name: mongodb
  static_configs:
  - targets: ['localhost:9216']

- job_name: redis
  static_configs:
  - targets: ['localhost:9121']

- job_name: "cadvisor"
  static_configs:
  - targets: ['localhost:8080']

2.2 docker方式启动

docker run  -d  --restart=always  --name prometheus \
    -p 9090:9090 \
    -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus

3 Grafana部署

docker run -d -p 3000:3000 --restart=always  --name grafana grafana/grafana
默认密码admin/admin

https://grafana.com/docs/grafana/latest/setup-grafana/installation/

模版仓库
https://grafana.com/grafana/dashboards/?pg=hp&plcmt=lt-box-dashboards
node_exporter模版 8919、11559
mysql_exporter模版 7362
elasticsearch_exporter模版 14191
mongodb_exporter模版 2583、16490
redis_exporter模版 763
cadvisor模版 893、11558、8321

4 docke-compose部署

version: "3.7"
services:
  prometheus:
    restart: always
    image: prom/prometheus
    container_name: prometheus
    ports:
      - 9090:9090
    volumes:
      - /path/to/prometheus.yml:/etc/prometheus/prometheus.yml
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--web.enable-lifecycle'            
      - '--web.enable-admin-api'

  grafana:
    restart: always
    image: grafana/grafana
    container_name: grafana
    ports:
      - 3000:3000

  node_exporter:
    image: quay.io/prometheus/node-exporter:latest
    container_name: node_exporter
    command:
      - '--path.rootfs=/host'
    network_mode: host
    pid: host
    restart: unless-stopped
    volumes:
      - '/:/host:ro,rslave'

  mysql_exporter:
    image: prom/mysqld-exporter
    container_name: mysql_exporter
    restart: unless-stopped
    command:
     - "--mysqld.username=[user]:[password]"
     - "--mysqld.address=[ip]:[port]"
    ports:
      - 9104:9104
      
  mongodb_exporter:
    image: percona/mongodb_exporter:0.40.0
    container_name: mongodb_exporter
    restart: unless-stopped
    command: --mongodb.uri='mongodb://[user]:[password]@[ip]:[port]/admin' --compatible-mode --collect-all
    ports:
      - 9216:9216

  elasticsearch_exporter:
    image: quay.io/prometheuscommunity/elasticsearch-exporter:latest
    container_name: elasticsearch_exporter
    command: --es.all --es.indices --collector.clustersettings   --es.indices_settings --es.indices_mappings --es.shards --collector.snapshots --es.timeout=10s --web.listen-address=":9114" --web.telemetry-path="/metrics" --es.uri <proto>://<user>:<password>@<host>:<port>
    restart: always
    ports:
      - 9114:9114
      
  redis_exporter:
    image: oliver006/redis_exporter
    container_name: redis_exporter
    restart: unless-stopped
    command: --redis.addr=<REDIS_HOST>:<REDIS_PORT> --redis.password=<REDIS_POSSWORD>
    ports:
      - 9121:9121

  cadvisor:
    image: google/cadvisor
    container_name: cadvisor
    restart: unless-stopped
    ports:
      - 8080:8080
    privileged: true
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro

Prometheus+Grafana+Exporter搭建监控平台