简述

记录java服务运行时jvm的metric信息采集操作，本文使用的方案是通过jmx_prometheus采集服务数据。

相关依赖

• jmx_exporter
  git : https://github.com/prometheus/jmx_exporter
  站点：https://prometheus.github.io/jmx_exporter/

操作步骤

• 1. 下载相关jar包
     下载地址：https://github.com/prometheus/jmx_exporter/releases
• 2. 修改java OPTS
     java应用的启动参数中添加如下内容：

-javaagent:/opt/jmx_exporter/jmx_prometheus_javaagent-1.3.0.jar=jmx_port:/opt/jmx_exporter/conf/conf.yml

• 3. 数据验证
     浏览器打开地址：http://ip:jmx_port/metrics,查看相关的metric数据

相关配置

conf.yml(通用)

# 应用标识标签
labels:
  application: "test-metric"
  environment: "test"
# jmx_exporter_generic.yml
lowercaseOutputName: true
lowercaseOutputLabelNames: true
# 白名单，只采集指定内容
whitelistObjectNames: ["java.lang:*"]
# 禁用默认线程指标
#blacklistObjectNames: ["java.lang:type=Threading"]  

rules:
  # ======================== JVM 核心指标 ========================
  - pattern: 'java.lang<type=Memory><HeapMemoryUsage>committed'
    name: jvm_memory_heap_committed_bytes
    type: GAUGE
  - pattern: 'java.lang<type=Memory><HeapMemoryUsage>used'
    name: jvm_memory_heap_used_bytes
    type: GAUGE
  - pattern: 'java.lang<type=Memory><NonHeapMemoryUsage>committed'
    name: jvm_memory_nonheap_committed_bytes
    type: GAUGE
  - pattern: 'java.lang<type=Memory><NonHeapMemoryUsage>used'
    name: jvm_memory_nonheap_used_bytes
    type: GAUGE

  - pattern: 'java.lang<type=GarbageCollector, name=(.+)><*>CollectionCount'
    name: jvm_gc_collections_total
    labels:
      gc: "$1"
    type: COUNTER
  - pattern: 'java.lang<type=GarbageCollector, name=(.+)><*>CollectionTime'
    name: jvm_gc_collection_time_milliseconds_total
    labels:
      gc: "$1"
    type: COUNTER

  - pattern: 'java.lang<type=Threading><*>ThreadCount'
    name: jvm_threads_live
    type: GAUGE
  - pattern: 'java.lang<type=Threading><*>DaemonThreadCount'
    name: jvm_threads_daemon
    type: GAUGE
  - pattern: 'java.lang<type=Threading><*>PeakThreadCount'
    name: jvm_threads_peak
    type: GAUGE

  - pattern: 'java.lang<type=OperatingSystem><*>ProcessCpuLoad'
    name: jvm_cpu_load_process
    type: GAUGE
  - pattern: 'java.lang<type=OperatingSystem><*>SystemCpuLoad'
    name: jvm_cpu_load_system
    type: GAUGE
  - pattern: 'java.lang<type=OperatingSystem><*>OpenFileDescriptorCount'
    name: jvm_filedescriptors_open
    type: GAUGE

  # 默认规则：收集所有其他指标（谨慎使用），调试时可开启
  # - pattern: '.*'
  #   name: jmx_$1
  #   type: UNTYPED

问题处理

访问http://ip:jmx_port/metrics,如果出现如下提示：

An Exception occurred while scraping metrics: java.lang.IllegalStateException: jvm_threads_daemon: duplicate metric name.

根本原因是配置的规则出现重复，除了配置文件配置的规则有重合这个可能性外，还有一种情况即和默认的规则出现重复了；处理方法如下：

1. 删除配置文件中重复的规则（推荐）
2. 禁用默认的规则

# 禁用默认线程指标
blacklistObjectNames: ["java.lang:type=Threading"]