什么是NTLM

NTLM是NT LAN Manager的缩写，即问询/应答(Challenge/Response)身份验证协议，是 Windows NT早期版本的标准安全协议。

认证流程

• 客户端向服务器发送HTTP请求，请求获得服务器资源,如访问邮件服务器资源
• 服务器由于开启了NTLM认证，所以返回401的错误码(未授权)，提示需要NTLM认证
• 客户端发起NTLM认证，向服务器发送协商消息
• 服务器收到消息后，生成一个随机数Challenge，明文发送回客户端
• 客户端接收到Challenge后，使用密码hash对Challenge加密，生成Response并发送给服务器
• 服务器接收到Response后，会向DC(Domain Controller)发送针对客户端的验证请求，该请求主要包含以下三方面的内容：客户端用户名，客户端密码哈希值加密的Challenge和原始的Challenge
• DC根据用户名获取该帐号的密码哈希值，对原始的Challenge进行加密。如果加密后的Challenge和服务器发送的一致，则意味着用户拥有正确的密码，验证通过，否则验证失败。DC将验证结果发给服务器，并最终反馈给客户端

image.png

下面以调用exchange邮件服务https://xxx.com/EWS/Exchange.asmx
为例进行分析,使用的网络库为httpclient 4.5.1

implementation 'org.apache.httpcomponents:httpmime:4.5.1'
implementation 'org.apache.httpcomponents:httpclient:4.5.1'

public static void main(String[] args) throws IOException {
    System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
    System.setProperty("org.apache.commons.logging.simplelog.defaultlog", "all");
    
    HttpPost httpPost = new HttpPost("https://xxx.com/EWS/Exchange.asmx");
    httpPost.addHeader("Content-type", "text/xml; charset=utf-8");
    httpPost.addHeader("User-Agent", "EWS");
    httpPost.addHeader("Accept", "text/xml");
    httpPost.addHeader("Keep-Alive", "300");
    httpPost.addHeader("Connection", "Keep-Alive");
    httpPost.addHeader("Accept-Encoding", "gzip,deflate");

    RequestConfig.Builder requestConfigBuilder = RequestConfig.custom()
            .setAuthenticationEnabled(true)
            .setRedirectsEnabled(true)
            .setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM));
    CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    NTCredentials webServiceCredentials = new NTCredentials("username", "password", "", "domain");
    credentialsProvider.setCredentials(new AuthScope(AuthScope.ANY), webServiceCredentials);
    HttpClientContext httpContext = HttpClientContext.create();
    httpContext.setCredentialsProvider(credentialsProvider);
    httpPost.setConfig(requestConfigBuilder.build());

    StringEntity entity = new StringEntity("<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><soap:Header><t:RequestServerVersion Version="Exchange2010_SP2"></t:RequestServerVersion></soap:Header><soap:Body><m:GetFolder><m:FolderShape><t:BaseShape>IdOnly</t:BaseShape></m:FolderShape><m:FolderIds><t:DistinguishedFolderId Id="inbox"></t:DistinguishedFolderId></m:FolderIds></m:GetFolder></soap:Body></soap:Envelope>\n");
    httpPost.setEntity(entity);
    CloseableHttpClient httpClient = HttpClients.custom().build();
    HttpResponse httpResponse = httpClient.execute(httpPost, httpContext);
    System.out.println(httpResponse.getStatusLine().getProtocolVersion() + " " + httpResponse.getStatusLine().getStatusCode());
    System.out.println(EntityUtils.toString(httpResponse.getEntity()));
}

请求资源(客户端<=>服务器)

• 请求报文

POST /EWS/Exchange.asmx HTTP/1.1
Content-type: text/xml; charset=utf-8
User-Agent: ExchangeServicesClient/0.0.0.0
Accept: text/xml
Keep-Alive: 300
Connection: Keep-Alive
Accept-Encoding: gzip,deflate
Content-Length: 630
Host: xxx.com

• 响应报文

HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/8.5
request-id: 10ca999c-84bf-4860-9248-283176dc502a
X-OWA-Version: 15.1.2375.17
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="xxx.com"
X-Powered-By: ASP.NET
X-FEServer: EX-1
Date: Sun, 06 Feb 2022 10:14:41 GMT
Content-Length: 0

Type 1(客户端->服务器)

POST /EWS/Exchange.asmx HTTP/1.1
Content-type: text/xml; charset=utf-8
User-Agent: ExchangeServicesClient/0.0.0.0
Accept: text/xml
Keep-Alive: 300
Connection: Keep-Alive
Accept-Encoding: gzip,deflate
Content-Length: 630
Host: xxx.com
Authorization: NTLM TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==

Authorization中的认证信息TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==为固定值，生成规则为对固定字节数组进行base64编码，然后转换为String

byte[]bytes=new byte[]{78, 84, 76, 77, 83, 83, 80, 0, 1, 0, 0, 0, 1, -126, 8, -94, 0, 0, 0, 0, 40, 0, 0, 0, 0, 0, 0, 0, 40, 0, 0, 0, 5, 1, 40, 10, 0, 0, 0, 15};
byte[]base64=Base64.encodeBase64(bytes);
System.out.println(new String(base64,0,base64.length, StandardCharsets.US_ASCII));
//output: TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==

字节数组bytes的组成如下：

Type 2(服务器->客户端)

HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/8.5
request-id: 7ec66b3a-b31e-4569-aea5-76edca6dd97b
WWW-Authenticate: NTLM TlRMTVNTUAACAAAADAAMADgxxxxxxxxxxxxxxxEAAAABgOAJQAAAA9aAEUATgBNAEUATgACAAwAWgBFAE4ATQBFAE4AAQAIAEUAWAAtADEABAAWAHoAZQBuAG0AZQBuAC4AYwBvAHIAcAADACAARQBYAC0AMQAuAHoAZQBuAG0AZQBuAC4AYwBvAHIAcAAFABYAegBlAG4AbQBlAG4ALgBjAG8AcgBwAAcACAD8JnFbQhvYAQAAAAA=
X-OWA-Version: 15.1.2375.17
WWW-Authenticate: Negotiate
WWW-Authenticate: Basic realm="xxx.com"
X-Powered-By: ASP.NET
X-FEServer: EX-1
Date: Sun, 06 Feb 2022 10:14:41 GMT
Content-Length: 0

Type 3(客户端->服务器)

POST /EWS/Exchange.asmx HTTP/1.1
Content-type: text/xml; charset=utf-8
User-Agent: ExchangeServicesClient/0.0.0.0
Accept: text/xml
Keep-Alive: 300
Connection: Keep-Alive
Accept-Encoding: gzip,deflate
Content-Length: 630
Host: xxx.com
Authorization: NTLM TlRMTVNTUAADxxxxxxxxxxxxxxxxxLQAYAAAAAwADAAUAQAADAAMACABAAAAAAAALAEAAAAAAAAsAQAABYKIogUBKAoAAAAPNXkU/KGJ902Hu30Coa3vMvKpcRDQ7F3Ax4fiNLAVa9htbnnjMBNx5wEBAAAAAAAAMEf6W0Ib2AFK8uwnZtst1wAAAAACAAwAWgBFAE4ATQBFAE4AAQAIAEUAWAAtADEABAAWAHoAZQBuAG0AZQBuAC4AYwBvAHIAcAADACAARQBYAC0AMQAuAHoAZQBuAG0AZQBuAC4AYwBvAHIAcAAFABYAegBlAG4AbQBlAG4ALgBjAG8AcgBwAAcACAD8JnFbQhvYAQAAAAAAAAAAWgBFAE4ATQBFAE4AawBvAG4AZwBwAGYA

最终响应报文(服务器->客户端)

HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: text/xml; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/8.5
request-id: bfe3362c-bacb-4c4a-8a40-dbf5cc5c2e2b
X-DiagInfo: EX-2
X-BEServer: EX-2
X-AspNet-Version: 4.0.30319
Set-Cookie: exchangecookie=a988e63f541647a592956c572e7b8de2; expires=Mon, 06-Feb-2023 10:14:42 GMT; path=/; HttpOnly
Set-Cookie: X-BackEndCookie=S-1-5-21-583588886-2180775141-1717425756-27500=u56Lnp2ejJqBzp3Hxs2dmc7Szs2bnNLLy53H0p7MzJvSxpqcyM6az8/JzZqagYHNz83N0s/M0s/Hq87Pxc7LxcvNgYWakZKakdGckI2Pgc8=; expires=Tue, 08-Mar-2022 10:14:42 GMT; path=/EWS; secure; HttpOnly
Persistent-Auth: true
X-Powered-By: ASP.NET
X-FEServer: EX-1
Date: Sun, 06 Feb 2022 10:14:41 GMT

密码hash

算法定义

1. 将password字符串转化为Unicode16进制小端序列字符串

2. 对Unicode字符串做MD4运算

如原始的密码为123456，则对应的密码hash为32ED87BDB5FDC5E9CBA88547376818D4

nodejs实现

var crypto = require('crypto');
function getNTLMHash(password){
  var buf = Buffer.from(password, 'utf16le');
  var md4 = crypto.createHash('md4');
  md4.update(buf);
  return Buffer.from(md4.digest());
}
var hash=getNTLMHash('123456')
console.log("ntlm hash:"+hash.toString('hex').toUpperCase())

java实现

implementation "com.sun.mail:javax.mail:1.6.2"

public static void main(String[] args) {
    String str="123456";
    byte[]bytes=str.getBytes(StandardCharsets.UTF_16LE);
    MD4 md4=new MD4();
    byte[]hash=md4.digest(bytes);
    System.out.println("ntlm hash:"+toHex(hash));
}

private static char[] hex = { '0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F' };
private static String toHex(byte[] bytes) {
    char[] result = new char[bytes.length * 2];
    for (int index = 0, i = 0; index < bytes.length; index++) {
        int temp = bytes[index] & 0xFF;
        result[i++] = hex[temp >> 4];
        result[i++] = hex[temp & 0xF];
    }
    return new String(result);
}

OkHttp支持NTLM验证

• 复制代码org.apache.http.impl.auth.NTLMEngineImpl到自己的项目目录并适当修改
• 在AndroidManifest.xml文件application节点下添加：

<uses-library
    android:name="org.apache.http.legacy"
    android:required="false" />

• 添加NTLMAuthenticator类，代码如下：

import android.text.TextUtils;

import org.apache.http.impl.auth.NTLMEngineException;

import java.util.List;
import java.util.Locale;

import okhttp3.Authenticator;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.Route;

public class NTLMAuthenticator implements Authenticator {

    private final String domain;
    private final String username;
    private final String password;
    private final String workstation;
    private final NTLMEngineImpl engine = new NTLMEngineImpl();

    public NTLMAuthenticator(String username, String password, String domain, String workstation) {
        this.username = username;
        this.password = password;
        this.domain = !TextUtils.isEmpty(domain) ? domain.toUpperCase(Locale.ROOT) : "";
        this.workstation = !TextUtils.isEmpty(workstation) ? workstation.toUpperCase(Locale.ROOT) : "";
    }

    @Override
    public Request authenticate(Route route, Response response) {
        final List<String> WWWAuthenticate = response.headers().values("WWW-Authenticate");
        if (WWWAuthenticate.contains("NTLM")) {
            try {
                String ntlmMsg1 = engine.generateType1Msg(null, null);
                return response.request().newBuilder().header("Authorization", "NTLM " + ntlmMsg1).build();
            } catch (NTLMEngineException e) {
                e.printStackTrace();
            }
        }
        String ntlmMsg3 = null;
        try {
            ntlmMsg3 = engine.generateType3Msg(username, password, domain, workstation, WWWAuthenticate.get(0).substring(5));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return response.request().newBuilder().header("Authorization", "NTLM " + ntlmMsg3).build();
    }
}

• 设置OkHttp

val client = OkHttpClient.Builder()
        .authenticator(NTLMAuthenticator("username", "password", "domain", ""))
        .build()

参考资源

• https://github.com/FrankyBoy/request-ntlm

• NTLM Hash在线生成网站

• JavaMail-https://javaee.github.io/javamail

• EwsEditor-https://github.com/dseph/EwsEditor

• Support NTLM auth-https://github.com/square/okhttp/issues/206