缘起
Mobile App/SDK开发中会经常调用后端REST接口进行CRUD操作，但RESTful API 可能定义好但尚未开发完成，或有实现缺陷，这就要求client能mock接口数据或直接修改API返回数据，以保证APP UI或Mobile SDK API能按设计工作。
为实现该目的且最好平台（Android & iOS）无关，将探讨基于mitmproxy的抓包和数据拦截。
开发环境（MacOS）
- 安装mitmproxy
brew install mitmproxy
- 证书管理
首次运行mitmproxy时，会自动生成CA证书并存储到mitmproxy配置文件~/.mitmproxy中。
➜ ~ mitmproxy
➜ ~ ll .mitmproxy/
total 48
-rw-r--r-- 1 nling staff 1.3K Dec 11 12:00 mitmproxy-ca-cert.cer
-rw-r--r-- 1 nling staff 1.1K Dec 11 12:00 mitmproxy-ca-cert.p12
-rw-r--r-- 1 nling staff 1.3K Dec 11 12:00 mitmproxy-ca-cert.pem
-rw------- 1 nling staff 2.5K Dec 11 12:00 mitmproxy-ca.p12
-rw------- 1 nling staff 3.0K Dec 11 12:00 mitmproxy-ca.pem
-rw-r--r-- 1 nling staff 770B Dec 11 12:00 mitmproxy-dhparam.pem
不同客户端需要配置相应CA证书已进行数据的加解密，否则无法与服务端建立安全的连接。例如
curl --proxy 127.0.0.1:8080 https://www.baidu.com
会报如下错误
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
不预置和信任证书的情况下可使用--insecure模式或--cacert指定证书路径来解决。
curl --proxy 127.0.0.1:8080 --insecure https://www.baidu.com
或
curl --proxy 127.0.0.1:8080 --cacert ~/.mitmproxy/mitmproxy-ca-cert.cer https://www.baidu.com
配置证书
最快的方式是在客户端浏览器地址栏中访问mitm.it，然后根据向导安装和配置证书。若该地址不能正常访问，如显示为“If you can see this, traffic is not passing through mitmproxy.“，此时需要手动配置证书，否则HTTPS网页或Apps的请求不能正常加解密。
-
MacOS
- 双击
.mitmproxy/mitmproxy-ca-cert.cer添加证书到login或system keychains - 找到新增加的证书并选择【信任】（导入后默认为不信任）
- 双击
-
Android设备或模拟器 (以Android 11模拟器为例）
- 拷贝证书文件到
Downloads文件夹
adb push /Users/nling/.mitmproxy/mitmproxy-ca-cert.cer /sdcard/Download/- 从系统界面安装证书
Security -> Encryption & credentials -> Install a certificate -> CA certificate- 添加网络配置
配置文件：AndroidManifest.xml
<application> android:networkSecurityConfig="@xml/network_security_config" </application>配置内容：network_security_config.xml
<?xml version="1.0" encoding="utf-8"?> <network-security-config> <debug-overrides> <trust-anchors> <!-- Trust user added CAs while debuggable only --> <certificates src="user" /> </trust-anchors> </debug-overrides> </network-security-config>- 代理配置
连接设备到与主机相同到Wifi网络。模拟器默认为AndroidWifi，并选择编辑网络
Proxy: Manual Proxy hostname: 192.168.1.109 # 修改为运行mitmproxy的主机IP地址 Proxy port: 8080 - 拷贝证书文件到
使用自签名证书
openssl genrsa -out cert.key 2048
openssl req -new -x509 -key cert.key -out cert.crt
cat cert.key cert.crt > cert.pem
mitmproxy --certs cert.pem
测试
~ curl --proxy 127.0.0.1:8080 https://www.baidu.com
结果
Proxy server listening at http://*:8080
127.0.0.1:61054: clientconnect
127.0.0.1:61054: CONNECT www.baidu.com:443
<< Cannot establish TLS with client (sni: www.baidu.com): TlsException("SSL handshake error: Error([('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')])")
https://discourse.mitmproxy.org/t/self-created-ca-client-certificates/605/4
https://docs.mitmproxy.org/stable/concepts-certificates/
常用操作
任务拦截
- 快捷键
i进入拦截模式 - 配置拦截规则并保存退出
set intercept '~u /Dunedin & ~q' // 拦截URLpath “/Dunedin”
- 测试拦截
curl --proxy 127.0.0.1:8080 --cacert ~/.mitmproxy/mitmproxy-ca-cert.cer "http://wttr.in/Dunedin?0"
- 选中中断的目标请求
- 拦截操作
- 快捷键
a继续请求 - 快捷键
X终止请求 - [Enter] 查看拦截请求
-
e修改请求
| 1) cookies
| 2) urlencoded form
| 3) multipart form
| 4) path
| 5) method
| 6) query
| 7) reason
| 8) request-headers
| 9) response-headers
| a) request-body
| b) response-body
| c) status_code
| d) set-cookies
| e) url -
~保存并退出修改 -
q返回拦截请求详情界面 -
a继续修改后的请求 -
r重新请求
-
- 快捷键
拦截修改
- 快捷键
i进入拦截模式 - 配置拦截规则并保存退出
set intercept '~u /Dunedin & ~q' // 拦截URLpath “/Dunedin”
- 测试拦截
curl --proxy 127.0.0.1:8080 --cacert ~/.mitmproxy/mitmproxy-ca-cert.cer "http://wttr.in/Dunedin?0"
Group & Scene测试记录
- 新建网络配置文件
library/src/main/res/xml/network_security_config.xml
- 配置文件:
library/src/androidTest/AndroidManifest.xml<application> android:networkSecurityConfig="@xml/network_security_config" </application> - 配置模拟器使用主机代理
- 启动mitmproxy
- 设置地址拦截(
i):~u groupsceneservice/v1/collections.json & ~q - 运行测试case
CollectionsTest#testCreateCollection - mock返回数据
{
"collection_uuid": "62871755-3c72-4757-9af2-4e76569sfs54",
"name": "Good Morning",
"type": "SCENE",
"is_active": true,
"devices": [{
"dsn": "AC000W0000001165",
"states": [{
"property_name": "prop_name1",
"property_value": 1
},
{
"property_name": "prop_name2",
"property_value": 2
}
]
}],
"child_collections": [{
"collection_uuid": "62871755-3c72-4757-9af2-4e76569e6b94",
"devices": [{
"dsn": "AC000W0000001144"
}, {
"dsn": "AC000W0000001145"
}],
"states": [{
"property_name": "prop_name1",
"property_value": 1
}]
}],
"custom_attributes": [{
"key": "color",
"value": "#F08080"
},
{
"key": "sort_order",
"value": "1"
}
],
"trigger_expression": "30 17 * * 1-7",
"shared_users": []
}
FAQ
参考资料
- Kimmo Kulovesi, Scott Lyttle. mitmproxy-mock. 11/14/2020, 12/11/2020.
