背景
在web开发中, HTTP 400 Bad Request 是一种常见但却难以定位的问题, 通常是因为请求没有遵循 HTTP 标准. 原因看起来很直观, 但是在定位此类问题时, 往往需要花费极大的精力. 有以下几个难点:
- 一个请求在从客户端发出到应用服务器收到这个过程中, 中间经历了各种转发, 每个中间环节都有可能对请求进行改写, 例如代理、LB等. 所以通常需要与多个部门沟通协作, 定位改动发生的环节.
- 当不满足HTTP 标准的请求到达应用服务器之后, Tomcat 会在请求进入业务逻辑之前就返回400, 并且对于这种行为的默认的日志等级为debug. 这就意味着, 当此类问题发生时, 从业务逻辑的日志中发现不了任何异常. 通常需要抓包或暂时将Tomcat 的日志标准设置为debug.
- 即使抓包获取了请求信息, 也很难直接观察出请求出问题的地方.
- 甚至有些情况下, “相同”的请求, 一部分没有问题, 另一部分有问题. 这给定位问题带来了更多的干扰项.
在本片文章中, 会分享实际生产环境下的真实事例. 并且对Tomcat中所有400的情况给出了实际例子, 方便以后更直观的判断请求是否有问题.
本文基于 tomcat-embed-core-9.0.29
案例分析
Case 1. Web容器使用的协议版本不同
详见下一篇博文
Case 2. 请求头中含有非法字符
现象
由于对LB进行升级, 将流量从旧的LB切换到新的LB之后, 某些客户端的所有响应都变成了 400 BAD REQUEST.
分析
-
切换前后的网络拓扑结构发生了变化:
- 切换之前, 请求路线为 client -> 旧LB -> proxy -> server
- 切换之后, 请求路线为 client -> 新LB -> server
- proxy 做了一些额外的操作来确保请求的正确性. 在这里, 它将请求头中冒号前的空格给删除了.
请求头中冒号前的空格属于非法字符, Tomcat对于这种情况会直接返回400.
客户端发送的请求头示例 "APPLICATION-VERSION : 519", 冒号前的空格即为非法字符.
Tomcat相关源码
Tomcat调用逻辑如下. 在parseHeader()方法中, 会使用isToken()方法检查请求头中是否有非法字符.
org.apache.coyote.http11.Http11Processor#service
org.apache.coyote.http11.Http11InputBuffer#parseHeaders
org.apache.coyote.http11.Http11InputBuffer#parseHeader
org.apache.tomcat.util.http.parser.HttpParser#isToken
org.apache.coyote.http11.Http11InputBuffer#skipLine
throw new IllegalArgumentException(message);
response.setStatus(400);
org.apache.coyote.http11.Http11Processor#service: parseHeaders and catch IllegalArgumentException
org.apache.coyote.http11.Http11InputBuffer#parseHeader: 检验请求头并在skipLine()中抛出llegalArgumentException. IS_TOKEN 是一个用来存储非法字符的位图, 非法字符包含 ' ' ',' '(' ')' '' 等.
org.apache.coyote.http11.Http11InputBuffer#skipLine: 抛出异常
Tomcat HTTP 400 Bad Request总结
* Guide
这里总结了所有tomcat-embed-core-9.0.29中会返回HTTP 400 Bad Request的情况, 并给出了样例.
以下例子均为HTTP报文格式, case 0 是一个可以正确返回200的请求, 后续的例子是在case 0 的基础上进行改变, 改变的部分会以橙色标出, 导致400的关键字会以红色标出, 便于直观的发现问题.
0. 正确的请求
200 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
200 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
1. 请求头Host中端口号不为数字
org.apache.coyote.AbstractProcessor#line 302
400 Sample
Host: localhost:abc
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
2. 请求头Host中含有非法字符
org.apache.coyote.AbstractProcessor#line 337
Illegal characters logic: org.apache.tomcat.util.http.parser.HttpParser.DomainParseState#next
400 Sample
Host: localho(st
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
3.Socket状态为CONNECT_FAIL(eg. TLS handshake FAIL)
org.apache.coyote.AbstractProcessor#line 982
No example
4. 请求头名称中含有非法字符或空格
org.apache.coyote.http11.Http11Processor#line 311
Character set: org.apache.tomcat.util.http.parser.HttpParser#IS_TOKEN
400 Sample for illegal character
Host: localhost
Content-T(ype: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
400 Sample for blank
Host: localhost
Content-Type : application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
5. 存在多个Host请求头
org.apache.coyote.http11.Http11Processor#line 609
400 Sample
Host: localhost
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
6. 不存在Host请求头
org.apache.coyote.http11.Http11Processor#line 612
400 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
7. 授权中的用户信息含有非法字符
org.apache.coyote.http11.Http11Processor#line 660
Character set: org.apache.tomcat.util.http.parser.HttpParser#IS_USERINFO
400 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
8. URL中的Host与请求头中的Host不一致
org.apache.coyote.http11.Http11Processor#line 686
400 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
9. Start line中协议名称非法
org.apache.coyote.http11.Http11Processor#line 705
400 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
10. URI 中含有非法字符
org.apache.coyote.http11.Http11Processor#line 713
Character set: org.apache.tomcat.util.http.parser.HttpParser#IS_ABSOLUTEPATH_RELAXED
400 Sample
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
11. 请求头中Content-Length值不为数字
org.apache.coyote.http11.Http11Processor#line 739
400 Sample
POST /ws/spf HTTP/1.1
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: abc
12. 请求头中含有多个Content-Length
org.apache.coyote.http11.Http11Processor#line 741
400 Sample
POST /ws/spf HTTP/1.1
Host: localhost
Content-Type: application/xml
X-SOA-OPERATION-NAME: getVersion\r\n\tsecond line\r\n\tthird line
X-SOA-SERVICE-NAME: CoreShippingService
Content-length: {requestBodyLength}
Content-length: {requestBodyLength}
