监控

目前大部分应用在听云上面监控

• 缺点: 不能监控日志

自己搭建elk日志监控平台

• 监控logback日志，抽取关键字 统一日志规范报警

spring-boot-admin

目前简单方案

基于foxmail邮件监控，绑定微信

• 基于特定的异常监控 买点在web-common中
• 对于dubbo服务，买点在dubbo异常处理的filter中

使用配置

• 引入

     <dependency>
            <groupId>com.groot.io.common</groupId>
            <artifactId>monitor-common</artifactId>
            <version>1.0-SNAPSHOT</version>
      </dependency>

mail:
    host: smtp.exmail.qq.com
    protocol: smtp
    password: xxx
    username: xxxx
    properties:
      mail:
          smtp:
            auth: true
            timeout: 25000
            connectiontimeout: 60000
            writetimeout: 60000
      from: xxx
      to: xxxx
      # 同一个uri或方式一分钟最发送的频率
      rate: 2

报警邮件

请求: xxx-/test/xxxx
参数: {}
机器: xxx // 服务器机器ip
环境: ["prod"] // 环境
堆栈:
com.netflix.zuul.exception.ZuulException: Filter threw Exception
    at com.netflix.zuul.FilterProcessor.processZuulFilter(FilterProcessor.java:227)
    at com.netflix.zuul.FilterProcessor.runFilters(FilterProcessor.java:157)
    at com.netflix.zuul.FilterProcessor.preRoute(FilterProcessor.java:133)
    at com.netflix.zuul.ZuulRunner.preRoute(ZuulRunner.java:105)
    at com.netflix.zuul.http.ZuulServlet.preRoute(ZuulServlet.java:125)
    at com.netflix.zuul.http.ZuulServlet.service(ZuulServlet.java:74)

实现

image.png

注册机制可以扩展其他的报警实现方式

• 实现Monitor接口

@Slf4j
@Order(1)
@Component
public class EmailMonitorHandler implements Monitor {

  @Autowired
  private EmailService emailService;

  @Value("${spring.application.name}")
  private String appName;

  @Value("${spring.mail.properties.rate}")
  private int rate;

  @Override
  public boolean support(MonitorContext context) {

    if (Strings.isNullOrEmpty(context.getUri())) {

      log.info("[support] uri is blank not to send mail");
      return false;
    }

    boolean enter = RateLimiterService.enter(context.getUri(), rate);

    if (!enter) {
      log.info("[support] send mail slack rate is limit uri={}", context.getUri());
    }
    return enter;
  }

  @Override
  public void handler(MonitorContext context) {

    Map<String, String> map = Maps.newLinkedHashMap();
    map.put("请求", context.getIp() + "-" + context.getUri());
    map.put("参数", context.getParams());
    map.put("机器", LocalHostUtils.getIp());
    map.put("环境", context.getProfile());
    map.put("堆栈", context.getException());

    emailService.sendHtmlMail(appName, map);
  }


}

频率控制简单实用guava cache

public class RateLimiterService {

  private static LoadingCache<String, AtomicInteger> rate = CacheBuilder.newBuilder()
      .expireAfterWrite(1, TimeUnit.MINUTES).build(new CacheLoader<String, AtomicInteger>() {
        @Override
        public AtomicInteger load(String key) throws Exception {
          return new AtomicInteger(0);
        }
      });

  /**
   * 递增
   * 
   * @param key
   * @return
   * @throws ExecutionException
   */
  public static boolean enter(String key, int rateLimit) {

    return rate.getUnchecked(key).incrementAndGet() <= rateLimit;
  }
}

接入

try {

      MonitorContext context = new MonitorContext();
      context.setUri(request.getRequestURI());
      context.setProfile(JsonUtils.toJSON(environment.getActiveProfiles()));
      context.setParams(JsonUtils.toJSON(WebUtils.getParametersStartingWith(request, "")));
      context.setException(ExceptionUtils.getStackTrace(e));
      monitorManager.slack(context);

    } catch (Exception ex) {
      log.error("[monitor] ", ex);

    }