spring cloud gateway [DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144]

问题背景

线上环境spring cloud gateway偶尔遇到如下异常:

DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144

问题的产生原因很容易查到,REST API接口请求数据超过256K,被网关拦截。正常我们加大缓冲区的配置即可,-1表示不限制。但是并没有解决问题,下面介绍我的分析过程和解决办法。

spring:
  codec:
    max-in-memory-size: 1048576

环境信息:

Spring Boot 2.2.1.RELEASE

异常信息:

2021-02-04 15:39:33.014 [reactor-http-epoll-2] ERROR org.springframework.boot.autoconfigure.web.reactive.error.AbstractErrorWebExceptionHandler - [1513c08e-1]  500 Server Error for HTTP POST "/x-service/path"
org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
        at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:101)
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP POST "/x-service/path" [ExceptionHandlingWebHandler]
Stack trace:
                at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:101)
                at org.springframework.core.io.buffer.LimitedDataBufferList.updateCount(LimitedDataBufferList.java:94)
                at org.springframework.core.io.buffer.LimitedDataBufferList.add(LimitedDataBufferList.java:59)
                at reactor.core.publisher.MonoCollect$CollectSubscriber.onNext(MonoCollect.java:124)
                at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
                at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:192)
                at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
                at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:331)
                at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:352)
                at reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:484)
...

分析过程

  1. 定位异常触发点
    org.springframework.core.io.buffer.LimitedDataBufferList.java


    image.png
  1. 查找maxByteCount的设置代码
    org.springframework.http.codec.support.BaseDefaultCodecs.java


    image.png
  1. 经过调试发现,启动时初始化是正常的,配置此参数有效spring.codec.max-in-memory-size
  2. 但业务调用的时候此参数接受值为null,配置并未生效
  3. 继续查找调用源头发现,我们的自定义拦截器获取body信息,代码如下
ServerRequest serverRequest = ServerRequest.create(exchange, HandlerStrategies.withDefaults().messageReaders());
  1. 因为HandlerStrategies.withDefaults()是重新创建的对象,并未使用Spring注入的对象,造成配置不生效
  2. 查看gateway的ModifyRequestBodyGatewayFilterFactory,使用注入的messageReaders即可
    org.springframework.cloud.gateway.filter.factory.rewrite.ModifyRequestBodyGatewayFilterFactory.java


    image.png

解决办法

参考ModifyRequestBodyGatewayFilterFactory调整自定义拦截器,使用注入的配置类。
代码如下:


@Component
@Slf4j
public class XXXFilter implements GlobalFilter, Ordered {
    @Autowired
    ServerCodecConfigurer codecConfigurer;

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        ServerHttpRequest request = exchange.getRequest();
        try {
            if (request.getHeaders().getContentLength() > 0) {
                ServerRequest serverRequest = ServerRequest.create(exchange, codecConfigurer.getReaders());
                Mono<String> bodyToMono = serverRequest.bodyToMono(String.class);
                return bodyToMono.flatMap(reqBody -> {
                    if (isJson(reqBody)) {
                        String perms = JSONObject.parseObject(reqBody).getString("perms");
                        if (StringUtils.isNotEmpty(perms)) {
                            exchange.getAttributes().put(Constants.XXX_REQUEST_PERMS_ATTR, perms);
                        }
                    }
                    ServerHttpRequestDecorator requestDecorator = new ServerHttpRequestDecorator(exchange.getRequest()) {

                        @Override
                        public Flux<DataBuffer> getBody() {
                            NettyDataBufferFactory nettyDataBufferFactory = new NettyDataBufferFactory(ByteBufAllocator.DEFAULT);
                            DataBuffer bodyDataBuffer = nettyDataBufferFactory.wrap(reqBody.getBytes());
                            return Flux.just(bodyDataBuffer);
                        }
                    };
                    return chain.filter(exchange.mutate().request(requestDecorator).build());
                });
            }
            return chain.filter(exchange);
        } catch (Exception e) {
            log.error("Exception[XXXFilter]:", e);
            return chain.filter(exchange);
        }
    }
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容

  • Spring Cloud为开发人员提供了快速构建分布式系统中一些常见模式的工具(例如配置管理,服务发现,断路器,智...
    卡卡罗2017阅读 135,292评论 19 139
  • 序 本文主要研究一下skywalking的spring-cloud-gateway-plugin NettyRou...
    go4it阅读 9,184评论 0 0
  • 引言 全局过滤器无疑是作用于所有经过网关转发的请求的,对于设计者来说被设计成全局过滤器实现的功能组件也即是设计者认...
    Lance_Xu阅读 18,029评论 2 11
  • 熔断主要保护的是调用方服务,如某A服务调用B服务的rpc接口,突然B服务接口不稳定,表现为接口延迟或者失败率变大。...
    小蘑菇_54c1阅读 11,852评论 0 1
  • 久违的晴天,家长会。 家长大会开好到教室时,离放学已经没多少时间了。班主任说已经安排了三个家长分享经验。 放学铃声...
    飘雪儿5阅读 12,182评论 16 22