原理

Cache 仿佛一直以来都是提高性能, 特别是I/O密集型服务提高性能的法宝

参见下面一组数字, 比较不同媒介的存取速度: 缓存 > 内存 > 网络 > 磁盘
(来自 Google Fellow Jeff Dean 的 PPT )

• 每个人都应知道的一组数字 :

从上面的数据, 我们知道数据缓存在 CPU cache 和内存中存取是最快的, 磁盘最慢, 顺序为

1. 本地内存中的缓存数据
2. 本地磁盘中的数据
3. 远程快速节点内存中的缓存数据
4. 远程快速节点磁盘中的数据
5. 远程慢速节点内存中的缓存数据
6. 远程慢速节点磁盘中的数据

第2, 第3取决于网络和磁盘的存取速度, 顺序可能会对调

远程节点分两种：
一种是在低延迟网络中的远程节点，或者说同一个局域网或城域网中的比较近的节点，又或虽然相隔距离远但是通过专用高速网络互连的节点，RTT一般在50ms 以内

一种是在高延迟网络中的远程节点，比如跨地区，跨网络，RTT大于100ms的网络，有时候，同一个城市不同的网络运营商之前的网络速度可能比不同城市同一个运营商之前速度更慢

Cache 的种类

• 本地 Cache: 放在本地内存中的缓存, 典型代表就是内存中的 map
• 远程 Cache: 放在远程服务器内存中的缓存, 比如 memcached, redis cache
• 分布式 Cache: 它其实有两种, 一种全部是远程分布式 cache , 还有一种是本地和远程cache 有同步机制, 存取都是在本地内存中

其实缓存无处不在, CPU 有L1/L2 缓存, 硬盘有读写缓存, 这里仅提及微服务常用的缓存

对于Cache 我们最关心的就是 Cache 的使用效率, 也就是命中率, 提高命中率的关键因素是

1. Cache key size 缓存的键值数量
2. Cache capacity 缓存的容量
3. Cache lifetime 缓存的生命周期

Cache 不可能无限增长, 不可能永远有效, 所以对于 Cache 的清除策略和失效策略要细细考量.
对于放在 Cache 中的数据也最好是读写比较高的, 即读得多, 写得少, 不会频繁地更新.

Cache的常用操作通常有

• get： 获取
• put： 设置
• remove： 移除
• clear： 清除
• expire: 过期

对于Cluster Cache来说，读操作（get）肯定是本地方法，只需要从本台计算机内存中获取数据。

Remove/clear 这两个操作，一般是本地方法加上远程方法，需要和Cluster其他计算机进行同步。

Put这个写方法，可以是本地操作，也可以是远程操作的。 Remote Put方法的场景是这样，一台计算机把数据放到Cache里面，这个数据就会被传播到Cluster其他计算机上。这个做法的好处是Cluster各台计算机的Cache数据可以及时得到补充，坏处是传播的数据量比较大，这个代价比较大。
本地 Local Put方法的场景是这样，一台计算机把数据放到Cache里面，这个数据不会被传播到Cluster其他计算机上。这个做法的好处是不需要传播数据，坏处是Cluster各台计算机的Cache数据不能及时得到补充，这个不是很明显的问题，从Cache中得不到数据，从数据库获取数据是很正常的现象。 Local Put比起Remote Put的优势很明显，所以，通常的Cluster Cache都采用Local Put的策略。各Cache一般都提供了Local Put的配置选项，如果你没有看到这个支持，那么请换一个Cache。

通常有很多不同的 cache 实现，例如

• Guava Cache
• EhCache
• Memcached
• Redis

这里我们先重点讲讲 Guava Cache

最简单的cache 实现莫过于在内存中维护一张 map 了, 按 key 检索或存储 , 不过内存毕竟有限, 要自己实现 Cache 和各种清除和失效策略, Guava Cache 是不错的选择, 可以很方便地设置最大容量, 清除和失效策略等等.

它的主要特性有:

• 自动加载数据项到缓存中
• 当最大限度到达时应用最近最少策略驱逐数据项
• 从上次的访问或修改时间开始计算的数据项过期
• 数据项的键自动封装于弱引用中
• 数据项的值自动封装于弱引用或软引用中
• 当数据项被驱逐或清除时通知
• 聚合缓存访问统计

我们来举一个天气预报数据的缓存例子, 天气预报是那种比较适合缓存的数据, 它在一定的时间范围内变化不大, 读写比很高.

各大厂商纷纷推出各种 open API , 免费的大多有调用次数限制，例如百度的天气API, 一天之内的限制是 5000条/天，并发上限是 200QPS， 一般使用用也够了，可是如果不是一般情况，使用缓存就可以轻松搞定

百度应用控制台 http://lbs.baidu.com/apiconsole/key#/home

baidu application console

接口示例

接口参数说明

返回结果

完整代码参见 https://github.com/walterfan/helloworld/tree/master/hellocache

主要界面如下：

主要代码如下：

• WeatherApplication

package com.github.walterfan.hellocache;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class WeatherApplication {

    public static void main(String[] args) {
        SpringApplication.run(WeatherApplication.class, args);
    }
}

• WeatherController

package com.github.walterfan.hellocache;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Controller;
import org.springframework.stereotype.Service;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;

/**
 * Created by yafan on 8/10/2017.
 */
@Slf4j
@Controller
public class WeatherController {
    @Autowired
    private WeatherService weatherService;
    // inject via application.properties
    @Value("${hello.message:test}")
    private String message = "Hello World";

    @RequestMapping("/")
    public String index(Model model) {
        model.addAttribute("cityName", "hefei");
        model.addAttribute("currentTime", Instant.now().toString());
        return "index";
    }



    @GetMapping("/weather")
    protected String queryWeather(Model model, HttpServletRequest req, HttpServletResponse resp) {


        String cityName = req.getParameter("cityName");
        model.addAttribute("cityName", cityName);
        Optional<CityWeather> cityWeather =  weatherService.getWeather(cityName);
        if(cityWeather.isPresent()) {
            log.info("weather: {}", cityWeather.get());
            model.addAttribute("cityWeather", cityWeather.get());
            model.addAttribute("weatherString", cityWeather.map(x->x.toString()).orElse(""));
        }
        return "index";
    }



}

• WeatherService.java

package com.github.walterfan.hellocache;

import com.codahale.metrics.MetricRegistry;
import com.google.common.cache.LoadingCache;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.Optional;
import java.util.concurrent.ExecutionException;


@Slf4j
@Service
public class WeatherService {
    @Autowired
    private LoadingCache<String, CityWeather> cityWeatherCache;

    @Autowired
    private MetricRegistry metricRegistry;


    public Optional<CityWeather> getWeather(String city) {
        try {
            return Optional.ofNullable(cityWeatherCache.get(city));
        } catch (ExecutionException e) {
            log.error("getWeather error", e);
            return Optional.empty();
        }
    }
}

• WeatherCacheConfig.java 构建 Weather Cache 和相关依赖

package com.github.walterfan.hellocache;


import com.codahale.metrics.MetricRegistry;
import com.fasterxml.jackson.databind.ObjectMapper;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.context.annotation.Lazy;
import org.springframework.core.env.Environment;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.client.RestTemplate;

import java.util.List;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 14/10/2017.
 */
@EnableAspectJAutoProxy
@ComponentScan
@Configuration
@Slf4j
public class WeatherCacheConfig {//implements EnvironmentAware

    @Autowired
    private Environment environment;

    @Bean
    public WeatherCacheLoader weatherCacheLoader() {
        return new WeatherCacheLoader();
    }

    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate();

        List<HttpMessageConverter<?>> converters = restTemplate.getMessageConverters();
        for (HttpMessageConverter<?> converter : converters) {
            if (converter instanceof MappingJackson2HttpMessageConverter) {
                MappingJackson2HttpMessageConverter jsonConverter = (MappingJackson2HttpMessageConverter) converter;
                jsonConverter.setObjectMapper(new ObjectMapper());
                jsonConverter.setSupportedMediaTypes(ImmutableList.of(
                        new MediaType("application", "json", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET),
                        new MediaType("text", "javascript", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET)));
            }
        }
        return restTemplate;
    }

    @Bean
    public String appToken() {
        return this.environment.getProperty("BAIDU_AK");
    }


    @Bean
    public LoadingCache<String, CityWeather> cityWeatherCache() {
        LoadingCache<String, CityWeather> cache = CacheBuilder.newBuilder()
                .recordStats()
                .maximumSize(1000)
                .expireAfterWrite(60, TimeUnit.MINUTES)
                .build(weatherCacheLoader());

        recordCacheMetrics("cityWeatherCache", cache);
        return cache;
    }

    public void recordCacheMetrics(String cacheName, Cache cache) {
        MetricRegistry metricRegistry = metricRegistry();
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "hitCount"), () -> () -> cache.stats().hitCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "hitRate"), () -> () -> cache.stats().hitRate());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "missCount"), () -> () -> cache.stats().missCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "missRate"), () -> () -> cache.stats().missRate());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "requestCount"), () -> () -> cache.stats().requestCount());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadCount"), () -> () -> cache.stats().loadCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadSuccessCount"), () -> () -> cache.stats().loadSuccessCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadExceptionCount"), () -> () -> cache.stats().loadExceptionCount());
    }

    public String generateMetricsKeyForCache(String cacheName, String keyName) {
        String metricKey = MetricRegistry.name("cache", cacheName, keyName);
        log.info("metric key generated for cache: {}", metricKey);
        return metricKey;
    }

    @Bean
    public DurationTimerAspect durationTimerAspect() {
        return new DurationTimerAspect();
    }

    @Bean
    @Lazy
    public MetricRegistry metricRegistry() {
        return new MetricRegistry();
    }
}

• WeatherCacheLoader 从远程的 Baidu API 获取天气预报数据放入缓存中
  同时把 main 函数放在这里, 也就是构建 Spring Application Context, 并从cache 中获取天气预报数据
  第一次花的时间比较长, 用了352 毫秒, 之后从cache 中获取数据, 都只了几十到几百微秒
  注意对于百度天气API, 你需要自己申请一个apptoken, 我申请了一个, 5000次/天的调用都免费的, 由于我用了一小时的缓存, 所以无需付出任何费用

package com.github.walterfan.hellocache;

import com.codahale.metrics.Histogram;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Snapshot;


import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;


import java.util.Map;
import java.util.SortedMap;
import java.util.concurrent.ExecutionException;

/**
 * Created by yafan on 11/10/2017.
 *
 * refer to api http://lbsyun.baidu.com/index.php?title=car/api/weather
 */

@Slf4j
@Component
public class WeatherCacheLoader extends CacheLoader<String, CityWeather> {


    @Autowired
    private RestTemplate restTemplate;

    @Autowired
    private String appToken;

    @Autowired
    private LoadingCache<String, CityWeather> cityWeatherCache;

    @Autowired
    private MetricRegistry metricRegistry;

    @DurationTimer(name="getCityWeather")
    public CityWeather getCityWeather(String city) throws ExecutionException {
        return this.cityWeatherCache.get(city);
    }


    @Override
    public CityWeather load(String city) throws Exception {

        String url = "http://api.map.baidu.com/telematics/v3/weather";
        UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url)
                .queryParam("location", city)
                .queryParam("output", "json")
                .queryParam("ak", appToken);

        ResponseEntity<CityWeather> resp = restTemplate.getForEntity(builder.toUriString(), CityWeather.class);

        log.debug("response status: " + resp.getStatusCode());
        return resp.getBody();
    }

    public static void main(String[] args) throws ExecutionException {
        try (AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(WeatherCacheConfig.class)) {
            WeatherCacheLoader helloCacheLoder = (WeatherCacheLoader) context.getBean("WeatherCacheLoader");

            CityWeather cityWeather = helloCacheLoder.getCityWeather("hefei");
            for(int i=0;i<10;++i) {
                cityWeather = helloCacheLoder.getCityWeather("hefei");
            }

            log.info("----- weather -----");
            log.info(cityWeather.toString());

            MetricRegistry metricRegistry = (MetricRegistry) context.getBean("metricRegistry");


            SortedMap<String, Histogram> histograms =  metricRegistry.getHistograms();

            for(Map.Entry<String, Histogram> entry: histograms.entrySet()) {
                Snapshot snapshot = entry.getValue().getSnapshot();
                log.info("{}: size={},values: {}",  entry.getKey(), snapshot.size(), snapshot.getValues());
                log.info(" max={}, min={}, mean={}, median={}",
                        snapshot.getMax(), snapshot.getMin(), snapshot.getMean(), snapshot.getMedian());
            }
        }
    }


}

• output

13:02:03.273 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 352404 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 251 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 127 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 87 MICROSECONDS, threshold: 0 MICROSECONDS
...
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader - getCityWeather: size=11, count=[34, 36, 40, 45, 60, 63, 85, 87, 127, 251, 352404],values: {}
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader -  max=352404, min=34, mean=32112.0, median=63.0

这里顺手写了两个 Metrics 相关的辅助类, 利用 AOP 和 Metrics 来记录调用时间

• class DurationTimer

ackage com.github.walterfan.hello;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Target({ElementType.METHOD})
@Retention(value = RetentionPolicy.RUNTIME)
@Documented
public @interface DurationTimer {

    String name() default "";

    long logThreshold() default 0;

    //default timeunit μs
    TimeUnit thresholdTimeUnit() default TimeUnit.MICROSECONDS;
}

• class DurationTimerAspect

package com.github.walterfan.hello;

import com.codahale.metrics.MetricRegistry;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.aop.aspectj.annotation.AspectJProxyFactory;
import org.springframework.beans.factory.annotation.Autowired;

import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Aspect
public class DurationTimerAspect {


    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private MetricRegistry metricRegistry;

    public <T> T proxy(T o) {
        final AspectJProxyFactory factory = new AspectJProxyFactory(o);
        factory.setProxyTargetClass(true);
        factory.addAspect(this);
        return factory.getProxy();
    }

    @Around("@annotation( durationAnnotation ) ")
    public Object measureTimeRequest(final ProceedingJoinPoint pjp, DurationTimer durationAnnotation) throws Throwable {
        final long start = System.nanoTime();
        final Object retVal = pjp.proceed();

        String timerName = durationAnnotation.name();
        if("".equals(timerName)) {
            timerName = pjp.getSignature().toShortString();
        }
        TimeUnit timeUnit = durationAnnotation.thresholdTimeUnit();
        long threshold = durationAnnotation.logThreshold();
        //System.out.println("timerName=" + timerName);
        try {
            long difference = timeUnit.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);

            if(difference > threshold) {
                metricRegistry.histogram(timerName).update(difference);
                logger.info("Duration of {}: {} {}, threshold: {} {}", timerName, difference, timeUnit.name(), threshold, timeUnit.name());
            }

        } catch (Exception ex) {
            logger.error("Cannot measure api timing.... :" + ex.getMessage(), ex);
        }
        return retVal;
    }

}

相关的 DTO 如下

package com.github.walterfan.dto;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.util.Date;
import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
@JsonIgnoreProperties(ignoreUnknown = true)
public class CityWeather  {
    private int error;

    private String status;

    private Date date;

    private List<WeatherResult> results;

    public int getError() {
        return error;
    }

    public void setError(int error) {
        this.error = error;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public Date getDate() {
        return date;
    }

    public void setDate(Date date) {
        this.date = date;
    }

    public List<WeatherResult> getResults() {
        return results;
    }

    public void setResults(List<WeatherResult> results) {
        this.results = results;
    }

    @Override
    public String toString() {
        try {
            return new ObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(this);
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }
        return null;
    }
}


//---------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherData {

    private String date;
    private String dayPictureUrl;
    private String nightPictureUrl;
    private String weather;
    private String wind;
    private String temperature;

    public String getDate() {
        return date;
    }

    public void setDate(String date) {
        this.date = date;
    }

    public String getDayPictureUrl() {
        return dayPictureUrl;
    }

    public void setDayPictureUrl(String dayPictureUrl) {
        this.dayPictureUrl = dayPictureUrl;
    }

    public String getNightPictureUrl() {
        return nightPictureUrl;
    }

    public void setNightPictureUrl(String nightPictureUrl) {
        this.nightPictureUrl = nightPictureUrl;
    }

    public String getWeather() {
        return weather;
    }

    public void setWeather(String weather) {
        this.weather = weather;
    }

    public String getWind() {
        return wind;
    }

    public void setWind(String wind) {
        this.wind = wind;
    }

    public String getTemperature() {
        return temperature;
    }

    public void setTemperature(String temperature) {
        this.temperature = temperature;
    }
}
//------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherIndex {
    private String title;
    private String zs;
    private String tipt;
    private String des;

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getZs() {
        return zs;
    }

    public void setZs(String zs) {
        this.zs = zs;
    }

    public String getTipt() {
        return tipt;
    }

    public void setTipt(String tipt) {
        this.tipt = tipt;
    }

    public String getDes() {
        return des;
    }

    public void setDes(String des) {
        this.des = des;
    }
}
//---------------------
package com.github.walterfan.dto;

import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherResult {
    private String currentCity;
    private String pm25;

    private List<WeatherIndex> index;

    private List<WeatherData> weather_data;

    public String getCurrentCity() {
        return currentCity;
    }

    public void setCurrentCity(String currentCity) {
        this.currentCity = currentCity;
    }

    public String getPm25() {
        return pm25;
    }

    public void setPm25(String pm25) {
        this.pm25 = pm25;
    }

    public List<WeatherIndex> getIndex() {
        return index;
    }

    public void setIndex(List<WeatherIndex> index) {
        this.index = index;
    }

    public List<WeatherData> getWeather_data() {
        return weather_data;
    }

    public void setWeather_data(List<WeatherData> weather_data) {
        this.weather_data = weather_data;
    }
}

参考资料

• https://github.com/google/guava/wiki/CachesExplained