现象
- 修改了nacos配置，重新发布
- 过了一段时间发现，数据库偶尔会连接失败报异常
- create connection SQLException, url: jdbc:mysql://*******:3306/settlement?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&zeroDateTimeBehavior=convertToNull&serverTimezone=Asia/Shanghai, errorCode 1045, state 28000java.sql.SQLException: Access denied for user '****'@'ip' (using password: YES)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)
at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:156)
at com.alibaba.druid.filter.FilterAdapter.connection_connect(FilterAdapter.java:787)
at com.alibaba.druid.filter.FilterEventAdapter.connection_connect(FilterEventAdapter.java:38)
at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
at com.alibaba.druid.filter.stat.StatFilter.connection_connect(StatFilter.java:218)
at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
at com.alibaba.druid.filter.FilterAdapter.connection_connect(FilterAdapter.java:787)
at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1646)
at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1710)
at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2753)
环境
- mysql 5.7.28-log
- maven pom.xml
<dependency>
<groupId>com.github.ulisesbocchio</groupId>
<artifactId>jasypt-spring-boot-starter</artifactId>
<version>2.1.2</version>
</dependency>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<version>2.2.0.RELEASE</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>1.1.21</version>
</dependency>
- bootstrap.yml 配置
spring:
application:
name: test
cloud:
nacos:
serveraddr: ***
namespace: loc
config:
server-addr: ${spring.cloud.nacos.serveraddr}
namespace: ${spring.cloud.nacos.namespace}
prefix: ${spring.application.name}
file-extension: yml
discovery:
server-addr: ${spring.cloud.nacos.serveraddr}
namespace: ${spring.cloud.nacos.namespace}
- nacos配置
test.yml
jasypt:
encryptor:
password: hello
分析
发布nacos配置会导致配置刷新吗？
如下代码可以清楚看到和bootstrap.yml对应的NacosConfigProperties ，默认是开启刷新的。
com.alibaba.cloud.nacos.NacosConfigProperties
/**
* the master switch for refresh configuration, it default opened(true).
*/
private boolean refreshEnabled = true;
数据库为啥连接失败？
启动时数据库创建是成功的，为啥现在偶尔失败。
打个断点看一看
com.alibaba.druid.pool.DruidAbstractDataSource#createPhysicalConnection()
// 断点
String password = getPassword();
PasswordCallback passwordCallback = getPasswordCallback();
断点处发现 密码变成了 ENC()加密数据 ，密码根本未解密才导致连接失败。
什么时候密码被变成未解密的了？
修改nacos上的配置，重新发布
com.alibaba.druid.pool.DruidAbstractDataSource#setPassword
打个断点看一看
public void setPassword(String password) {
// 断点
if (StringUtils.equals(this.password, password)) {
return;
}
if (inited) {
LOG.info("password changed");
}
this.password = password;
}
堆栈如下，收到NacosContextRefresher事件后，动态刷新触发了密码的修改
setPassword:1134, DruidAbstractDataSource (com.alibaba.druid.pool)
invoke:-1, GeneratedMethodAccessor502 (sun.reflect)
invoke:43, DelegatingMethodAccessorImpl (sun.reflect)
invoke:498, Method (java.lang.reflect)
setValue:346, JavaBeanBinder$BeanProperty (org.springframework.boot.context.properties.bind)
bind:96, JavaBeanBinder (org.springframework.boot.context.properties.bind)
bind:79, JavaBeanBinder (org.springframework.boot.context.properties.bind)
bind:56, JavaBeanBinder (org.springframework.boot.context.properties.bind)
lambda$bindDataObject$5:452, Binder (org.springframework.boot.context.properties.bind)
get:-1, 540325452 (org.springframework.boot.context.properties.bind.Binder$$Lambda$42)
withIncreasedDepth:570, Binder$Context (org.springframework.boot.context.properties.bind)
withDataObject:556, Binder$Context (org.springframework.boot.context.properties.bind)
access$400:513, Binder$Context (org.springframework.boot.context.properties.bind)
bindDataObject:450, Binder (org.springframework.boot.context.properties.bind)
bindObject:391, Binder (org.springframework.boot.context.properties.bind)
bind:320, Binder (org.springframework.boot.context.properties.bind)
bind:308, Binder (org.springframework.boot.context.properties.bind)
bind:238, Binder (org.springframework.boot.context.properties.bind)
bind:225, Binder (org.springframework.boot.context.properties.bind)
bind:89, ConfigurationPropertiesBinder (org.springframework.boot.context.properties)
bind:107, ConfigurationPropertiesBindingPostProcessor (org.springframework.boot.context.properties)
postProcessBeforeInitialization:96, ConfigurationPropertiesBindingPostProcessor (org.springframework.boot.context.properties)
applyBeanPostProcessorsBeforeInitialization:416, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
initializeBean:1795, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
initializeBean:407, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
rebind:108, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
rebind:84, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
onApplicationEvent:142, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
onApplicationEvent:51, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
doInvokeListener:172, SimpleApplicationEventMulticaster (org.springframework.context.event)
invokeListener:165, SimpleApplicationEventMulticaster (org.springframework.context.event)
multicastEvent:139, SimpleApplicationEventMulticaster (org.springframework.context.event)
publishEvent:403, AbstractApplicationContext (org.springframework.context.support)
publishEvent:360, AbstractApplicationContext (org.springframework.context.support)
refreshEnvironment:96, ContextRefresher (org.springframework.cloud.context.refresh)
refresh:85, ContextRefresher (org.springframework.cloud.context.refresh)
handle:72, RefreshEventListener (org.springframework.cloud.endpoint.event)
onApplicationEvent:61, RefreshEventListener (org.springframework.cloud.endpoint.event)
doInvokeListener:172, SimpleApplicationEventMulticaster (org.springframework.context.event)
invokeListener:165, SimpleApplicationEventMulticaster (org.springframework.context.event)
multicastEvent:139, SimpleApplicationEventMulticaster (org.springframework.context.event)
publishEvent:403, AbstractApplicationContext (org.springframework.context.support)
publishEvent:360, AbstractApplicationContext (org.springframework.context.support)
innerReceive:133, NacosContextRefresher$1 (com.alibaba.cloud.nacos.refresh)
receiveConfigInfo:38, AbstractSharedListener (com.alibaba.nacos.api.config.listener)
run:203, CacheData$1 (com.alibaba.nacos.client.config.impl)
safeNotifyListener:233, CacheData (com.alibaba.nacos.client.config.impl)
checkListenerMd5:174, CacheData (com.alibaba.nacos.client.config.impl)
run:552, ClientWorker$LongPollingRunnable (com.alibaba.nacos.client.config.impl)
call:511, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:266, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
如何解决？
1. 不刷新
bootstrap.yml 配置
spring.cloud.nacos.config.refreshEnabled 为false
spring:
application:
name: test
cloud:
nacos:
serveraddr: ***
namespace: loc
config:
server-addr: ${spring.cloud.nacos.serveraddr}
namespace: ${spring.cloud.nacos.namespace}
prefix: ${spring.application.name}
file-extension: yml
refreshEnabled: false
discovery:
server-addr: ${spring.cloud.nacos.serveraddr}
namespace: ${spring.cloud.nacos.namespace}
2. 让获取的property是解密后
梳理一下事件和动作点
| 时刻 | 动作 |
|---|---|
| T1 | 我们点击发布新的nacos配置 |
| T2 | 触发ContextRefresh，更新本地的属性 |
| T3 | 连接池重新连接数据库 |
复现
在T1，T2时间之间，我kill掉了当前的connection。
因为是连接池，可能原来connection继续使用，这样无法看到报错，我需要kill掉之前的connection。
为了方便kill，我将druid初始化连接数，最小连接数都改为1。
mysql kill connection
// 查看当前connection
show processlist;
// 杀掉当前连接
kill pid;
refresh() 刷新
org.springframework.cloud.context.refresh.ContextRefresher
public synchronized Set<String> refresh() {
Set<String> keys = refreshEnvironment();
this.scope.refreshAll();
return keys;
}
public synchronized Set<String> refreshEnvironment() {
Map<String, Object> before = extract(
this.context.getEnvironment().getPropertySources());
addConfigFilesToEnvironment();
// 哪些属性修改了，哪些Bean需要重新创建
Set<String> keys = changes(before,
extract(this.context.getEnvironment().getPropertySources())).keySet();
this.context.publishEvent(new EnvironmentChangeEvent(this.context, keys));
return keys;
}
debug看一下,可以看到是加密的PropertySource
image.png
在调用 addConfigFilesToEnvironment 之后，可以看到此时是不加密的PropertySource了。
image.png
nacos Property 如何刷新
com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosDataIfPresent
private void loadNacosDataIfPresent(final CompositePropertySource composite,
final String dataId, final String group, String fileExtension,
boolean isRefreshable) {
if (null == dataId || dataId.trim().length() < 1) {
return;
}
if (null == group || group.trim().length() < 1) {
return;
}
NacosPropertySource propertySource = this.loadNacosPropertySource(dataId, group,
fileExtension, isRefreshable);
this.addFirstPropertySource(composite, propertySource, false);
}
可以从图上看到核心的问题是NacosPropertySource没有被
EncryptableEnumerablePropertySourceWrapper 装饰，导致了获取到属性是未加密的。
可以从
org.springframework.cloud.context.refresh.ContextRefresher
看到触发了 EnvironmentChangeEvent 事件，所以解决方法是我们先处理EnvironmentChangeEvent事件，将NacosPropertySource装饰为EncryptableEnumerablePropertySourceWrapper
官方解决方案
jasypt-spring-boot-parent-3.0.3
image.png
1. 升级版本到v3.0.3
<dependency>
<groupId>com.github.ulisesbocchio</groupId>
<artifactId>jasypt-spring-boot-starter</artifactId>
<version>3.0.3<version>
</dependency>
2. nacos test.yml 新增配置
jasypt:
encryptor:
password: hello
# 新增配置
algorithm: PBEWithMD5AndDES
iv-generator-classname: org.jasypt.iv.NoIvGenerator
源码解析
我拉了v3.0.2和v3.0.3对比，
RefreshScopeRefreshedEventListener 是一个处理ApplicationEvent的Listener，
v3.0.3 新增了对
org.springframework.cloud.context.environment.EnvironmentChangeEvent的处理
image.png
V3.0.3 RefreshScopeRefreshedEventListener 代码如下
package com.ulisesbocchio.jasyptspringboot.caching;
import com.ulisesbocchio.jasyptspringboot.EncryptablePropertySource;
import com.ulisesbocchio.jasyptspringboot.EncryptablePropertySourceConverter;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.ApplicationEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.core.env.*;
import org.springframework.util.ClassUtils;
@Order(Ordered.HIGHEST_PRECEDENCE)
@Slf4j
public class RefreshScopeRefreshedEventListener implements ApplicationListener<ApplicationEvent> {
public static final String REFRESHED_EVENT_CLASS = "org.springframework.cloud.context.scope.refresh.RefreshScopeRefreshedEvent";
public static final String ENVIRONMENT_EVENT_CLASS = "org.springframework.cloud.context.environment.EnvironmentChangeEvent";
private final ConfigurableEnvironment environment;
private final EncryptablePropertySourceConverter converter;
public RefreshScopeRefreshedEventListener(ConfigurableEnvironment environment, EncryptablePropertySourceConverter converter) {
this.environment = environment;
this.converter = converter;
}
@Override
@SneakyThrows
public void onApplicationEvent(ApplicationEvent event) {
if (isAssignable(ENVIRONMENT_EVENT_CLASS, event) || isAssignable(REFRESHED_EVENT_CLASS, event)) {
log.info("Refreshing cached encryptable property sources");
refreshCachedProperties();
decorateNewSources();
}
}
private void decorateNewSources() {
// 将新的PropertySource转为EncryptablePropertySource
MutablePropertySources propSources = environment.getPropertySources();
converter.convertPropertySources(propSources);
}
boolean isAssignable(String className, Object value) {
try {
return ClassUtils.isAssignableValue(ClassUtils.forName(className, null), value);
} catch (ClassNotFoundException e) {
return false;
}
}
private void refreshCachedProperties() {
PropertySources propertySources = environment.getPropertySources();
propertySources.forEach(this::refreshPropertySource);
}
@SuppressWarnings("rawtypes")
private void refreshPropertySource(PropertySource<?> propertySource) {
if (propertySource instanceof CompositePropertySource) {
CompositePropertySource cps = (CompositePropertySource) propertySource;
cps.getPropertySources().forEach(this::refreshPropertySource);
} else if (propertySource instanceof EncryptablePropertySource) {
EncryptablePropertySource eps = (EncryptablePropertySource) propertySource;
eps.refresh();
}
}
}
