有这样一种场景，公司的各个链路服务要进行压测，为了尽量得出准确的压测结果，直接在生产的服务上进行压测，但是有几个条件
- 压测的请求必须要标记识别，数据库，中间件这些数据的流转存储也必须作区分
- mysql 有影子库，判断是压测的请求就直接操作影子库，除了 mysql, 可能还有 mongodb, es, hbase, redis这些数据都要落到影子库，压测完影子库删掉，不对生产库造成影响
- 重点是压测请求的识别与数据源的动态切换
- 请求的识别还包括在一个链路调用中，请求标记的识别跟透传
- 为了减少原有服务压测时候的改动，可以把这些影子库配置，动态切换的逻辑做成一个SDK，压测项目引入 SDK依赖就可以了
根据上面场景，现在来说明mongodb的多数据源的影子库，生产库 动态切换怎么完成，mysql 已经有很多成熟的方案了，redis，mq这些有机会再研究.
1 压测请求的标记识别
这个应该说是相当简单了，在压测的请求加个header 标记, 过滤去里面判断到 header 标记是否存在，然后把标记放到 ThreadLocal 里面，后面的动态切换需要依赖这个 副本变量
public class RequestTagFilter implements Filter {
@Override
public void init(FilterConfig filterConfig) throws ServletException {
}
@Override
public void doFilter(ServletRequest request,
ServletResponse response,
FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
String header = httpServletRequest.getHeader("test");
HeaderThreadLocal.setIsTestRequest(!StringUtils.isEmpty(header));
chain.doFilter(request, response);
}
@Override
public void destroy() {
}
}
2 mongodb 多数据源配置
模拟生产的mongo配置.png
生产和影子库的配置文件是分开的
mongo影子库的配置.png
3 多数据源的切换逻辑的bean注入，只要加入这2个bean 就可以了
public class DynamicMongoDbFactory extends SimpleMongoDbFactory {
//生产库的客户端与数据库
private MongoClient mongoClientPro;
private String databaseNamePro;
//影子库的客户端与数据库
private MongoClient mongoClientShodow;
private String databaseNameShodow;
private final boolean mongoInstanceCreated = false;
private final PersistenceExceptionTranslator exceptionTranslator = null;
private @Nullable WriteConcern writeConcern;
public DynamicMongoDbFactory(MongoClient mongoClientPro, MongoClient mongoClientShodow,
String databaseNamePro, String databaseNameShodow) {
super(mongoClientPro, databaseNamePro);
this.mongoClientPro = mongoClientPro;
this.mongoClientShodow = mongoClientShodow;
this.databaseNamePro = databaseNamePro;
this.databaseNameShodow = databaseNameShodow;
}
@Override
public void destroy() throws Exception {
super.destroy();
}
@Override
public MongoDatabase getDb() throws DataAccessException {
//实现数据源动态切换的逻辑，判断是否是压测请求
if (HeaderThreadLocal.isTestRequest()){
return mongoClientShodow.getDatabase(databaseNameShodow);
}
return mongoClientPro.getDatabase(databaseNamePro);
}
@Override
public MongoDatabase getDb(String dbName) throws DataAccessException {
MongoDatabase mongoDatabase = mongoClientPro.getDatabase(dbName);
if (mongoDatabase != null){
return mongoDatabase;
}
return mongoClientPro.getDatabase(dbName);
}
@Override
public PersistenceExceptionTranslator getExceptionTranslator() {
return new MongoExceptionTranslator();
}
@SuppressWarnings("deprecation")
@Override
public DB getLegacyDb() {
return mongoClientPro.getDB(databaseNamePro);
}
}
@Configuration
@EnableAutoConfiguration
public class DynamicMongoRegister implements EnvironmentAware{
private Environment environment;
@Override
public void setEnvironment(Environment environment) {
this.environment = environment;
}
@Bean
public DynamicMongoDbFactory simpleMongoDbFactory(){
String mongoUrlPro = environment.getProperty("spring.data.mongodb.uri");
String mongoUrlShadow = environment.getProperty("spring.data.mongodb.uri.shadow");
//这里获取2个数据源的配置，生成2个数据源的客户端实例
MongoClient mongoClientPro = new MongoClient(new MongoClientURI(mongoUrlPro, MongoClientOptions.builder()));
MongoClient mongoClientShadow = new MongoClient(new MongoClientURI(mongoUrlShadow, MongoClientOptions.builder()));
return new DynamicMongoDbFactory(mongoClientPro, mongoClientShadow, "produce", "shadow");
}
@Bean
public MongoTemplate mongoTemplate(){
return new MongoTemplate(simpleMongoDbFactory());
}
}
4mongo数据库新建2个库，对应好库名，建好表
image.png
5 进行测试
使用springboot-data-mongodb的操作方式.png
测试用例调用.png
