在默认的情况下，spring security oauth2 在发生异常情况下，返回的格式并不符合现实需要，其格式是：

{
    "error": "invalid_grant",
    "error_description": "Bad credentials"
}

我们一般需要是这种格式

{
    "code": "-1",
    "msg": "Bad credentials"
}

认证端

对于oauth2的异常，其类主要是 OAuth2Exception ，默认处理这些异常的是DefaultWebResponseExceptionTranslator,其错误主要是包含oauth2相关的错误，就类似以下的。

@SuppressWarnings("serial")
@org.codehaus.jackson.map.annotate.JsonSerialize(using = OAuth2ExceptionJackson1Serializer.class)
@org.codehaus.jackson.map.annotate.JsonDeserialize(using = OAuth2ExceptionJackson1Deserializer.class)
@com.fasterxml.jackson.databind.annotation.JsonSerialize(using = OAuth2ExceptionJackson2Serializer.class)
@com.fasterxml.jackson.databind.annotation.JsonDeserialize(using = OAuth2ExceptionJackson2Deserializer.class)
public class OAuth2Exception extends RuntimeException {

    public static final String ERROR = "error";
    public static final String DESCRIPTION = "error_description";
    public static final String URI = "error_uri";
    public static final String INVALID_REQUEST = "invalid_request";
    public static final String INVALID_CLIENT = "invalid_client";
    public static final String INVALID_GRANT = "invalid_grant";
    public static final String UNAUTHORIZED_CLIENT = "unauthorized_client";
    public static final String UNSUPPORTED_GRANT_TYPE = "unsupported_grant_type";
    public static final String INVALID_SCOPE = "invalid_scope";
    public static final String INSUFFICIENT_SCOPE = "insufficient_scope";
    public static final String INVALID_TOKEN = "invalid_token";
    public static final String REDIRECT_URI_MISMATCH ="redirect_uri_mismatch";
    public static final String UNSUPPORTED_RESPONSE_TYPE ="unsupported_response_type";
    public static final String ACCESS_DENIED = "access_denied";

DefaultWebResponseExceptionTranslator处理的OAuth2Exception，其类可见的序列化器如下，可以得出默认的返回格式：

DefaultWebResponseExceptionTranslator类中：

private ResponseEntity<OAuth2Exception> handleOAuth2Exception(OAuth2Exception e) throws IOException {

        int status = e.getHttpErrorCode();
        HttpHeaders headers = new HttpHeaders();
        headers.set("Cache-Control", "no-store");
        headers.set("Pragma", "no-cache");
        if (status == HttpStatus.UNAUTHORIZED.value() || (e instanceof InsufficientScopeException)) {
            headers.set("WWW-Authenticate", String.format("%s %s", OAuth2AccessToken.BEARER_TYPE, e.getSummary()));
        }

        ResponseEntity<OAuth2Exception> response = new ResponseEntity<OAuth2Exception>(e, headers,
                HttpStatus.valueOf(status));

        return response;

    }

·····························································································································································
OAuth2Exception的序列化器：

public class OAuth2ExceptionJackson2Serializer extends StdSerializer<OAuth2Exception> {

    public OAuth2ExceptionJackson2Serializer() {
        super(OAuth2Exception.class);
    }

    @Override
    public void serialize(OAuth2Exception value, JsonGenerator jgen, SerializerProvider provider) throws IOException,
            JsonProcessingException {
        jgen.writeStartObject();
        jgen.writeStringField("error", value.getOAuth2ErrorCode());
        String errorMessage = value.getMessage();
        if (errorMessage != null) {
            errorMessage = HtmlUtils.htmlEscape(errorMessage);
        }
        jgen.writeStringField("error_description", errorMessage);
        if (value.getAdditionalInformation()!=null) {
            for (Entry<String, String> entry : value.getAdditionalInformation().entrySet()) {
                String key = entry.getKey();
                String add = entry.getValue();
                jgen.writeStringField(key, add);                
            }
        }
        jgen.writeEndObject();
    }

}

所以我们需要重新定义自己的WebResponseExceptionTranslator，以及自己的oauth2异常以及其序列化器(也可以不定义，重写handleOauth2Exception即可)

自定义oauth2异常处理类，CustomWebResponseExceptionTranslator，模仿DefaultWebResponseExceptionTranslator即可，下面给出关键部分，注意result是自己定义返回格式的bean，含有msg和code。最后在认证服务器配置一下就可。

    private ResponseEntity handleOAuth2Exception(OAuth2Exception e) throws IOException {

        log.info("occur exception : " , e);

        int status = e.getHttpErrorCode();
        HttpHeaders headers = new HttpHeaders();
        headers.set("Cache-Control", "no-store");
        headers.set("Pragma", "no-cache");
        if (status == HttpStatus.UNAUTHORIZED.value() || (e instanceof InsufficientScopeException)) {
            headers.set("WWW-Authenticate", String.format("%s %s", OAuth2AccessToken.BEARER_TYPE, e.getSummary()));
        }

        Oauth2ExceptionCodeEnum oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.UNAUTHORIZED_CLIENT;

        if (e instanceof InvalidScopeException){
            oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.INVALID_SCOPE;
        }
        if (e instanceof InvalidGrantException){
            oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.INVALID_GRANT;
        }
        if (e instanceof InvalidClientException){
            oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.INVALID_CLIENT;
        }
        if (e instanceof UnsupportedGrantTypeException){
            oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.UNSUPPORTED_GRANT_TYPE;
        }

        if (e instanceof RedirectMismatchException){
            oauth2ExceptionCodeEnum = Oauth2ExceptionCodeEnum.REDIRECT_URI_MISMATCH;
        }

        Result r = new Result(oauth2ExceptionCodeEnum.getMsg(),oauth2ExceptionCodeEnum.getCode());

        return new ResponseEntity<>(r, headers, HttpStatus.valueOf(status));

    }

@Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
        //MUST：密码模式下需设置一个AuthenticationManager对象,获取 UserDetails信息
        //末确认点.userDetailsService(userDetailsService)

        /*
          使用 /pig4cloud/login 覆盖 原有的/oauth/token，注意这里是覆盖一旦配置 原有路径将失效
          endpoints.pathMapping("/oauth/token","/pig4cloud/login");
        */

        endpoints.allowedTokenEndpointRequestMethods(HttpMethod.GET, HttpMethod.POST);
        endpoints.approvalStore(jdbcApprovalStore());
        endpoints.authorizationCodeServices(jdbcAuthorizationCodeServices());
        /*
          这个是处理oauth2那些grant_type、invalid code 之类的异常返回，但是clientId、clientSecret这些错它不管的，
          要靠上面上面的security来配置security.authenticationEntryPoint()
         */
        endpoints.exceptionTranslator(new CustomWebResponseExceptionTranslator());
         /* 有了tokenServices就不设置了
        endpoints.authenticationManager(authenticationManager);

        endpoints.setClientDetailsService(jdbcClientDetailsService());
        endpoints.tokenStore(new JdbcTokenStore(dataSource));


        //token增强器，多加点信息在里面
        endpoints.tokenEnhancer(tokenEnhancerChain());
        */

        //tokenServices没有的话会自动创建一个默认的
        endpoints.tokenServices(customTokenService());
    }

这里说的是oauth2的异常处理，其中还有认证的异常，我们这里使用basic的认证方式，如果发生异常，是不会走这个自定义的异常处理器的，所以在securityConfig中配置,在httpBasic后面配置的那里处理了。

    @Override
    protected void configure(HttpSecurity http) throws Exception {
       http.addFilterBefore(customBasicAuthenticationFilter, BasicAuthenticationFilter.class);

        http.authorizeRequests()
                .antMatchers("/oauthClient/**").permitAll()
                .anyRequest().authenticated()
                .and()
                //授权码模式得加上httpBasic 不然报403,前后分离就用basic，不然用formLogin也开httpBasic在basic之前加filter
                /*
                设置好authentication认证好再SecurityContextHolder.getContext().setAuthentication(authResult);，不然触发
                .exceptionHandling().authenticationEntryPoint 产生 InsufficientAuthenticationException异常。说还没认证身份；

                要不就直接省功夫用basic认证，
                demoUri： http://kalos:123456789@127.0.0.1:8547/oauth/authorize?client_id=ncsf6sb4&response_type=code&scope=user_info&state=orion
                */
                .httpBasic()
                //密码错误返回在这里配置，用户名在自定义userDetail里配置
                .authenticationEntryPoint((request, response, authException) -> {
                    System.out.println("basic commence AuthenticationException：" + authException);
                    Result r = Oauth2ExceptionCodeEnum.UNAUTHORIZED_CLIENT.toResult();
                    ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                })
                .and()
                .exceptionHandling()
                .authenticationEntryPoint((request, response, authException) -> {
                    System.out.println("security config commence AuthenticationException:" + authException.getMessage());
                    Result r = CommonCodeEnum.COMMON_SERVER_ERROR.toResult();
                    ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                })
                .accessDeniedHandler((request, response, accessDeniedException) -> {
                    System.out.println("security config commence accessDeniedException:" + accessDeniedException.getMessage());
                    Result r = Oauth2ExceptionCodeEnum.ACCESS_DENIED.toResult();
                    ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                })
                .and().csrf().disable();

                /* session 设置为 IF_REQUIRED 有需要才生成 */
        http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);

这里加上 http.addFilterBefore(customBasicAuthenticationFilter, BasicAuthenticationFilter.class);

这个filter其实就是验证一些必要参数有无缺失，并且返回自定义的格式，

@Component
@Slf4j
public class CustomBasicAuthenticationFilter extends OncePerRequestFilter {

    private static final String OAUTH2_AUTHORIZATION_URL = "/oauth/authorize";

    @Resource
    private ClientDetailsService clientDetailsService;

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
            throws ServletException, IOException {

        if (!request.getRequestURI().equals(OAUTH2_AUTHORIZATION_URL)) {
            filterChain.doFilter(request, response);
            return;
        }

        if (!request.getMethod().equals(HttpMethod.POST.name())) {
            log.info("authorize request only post");
            Result r = Oauth2ExceptionCodeEnum.AUTHORIZE_UNSUPPORTED_REQUEST_METHOD.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        String scope = request.getParameter(OAuth2Utils.SCOPE);
        String clientId = request.getParameter(OAuth2Utils.CLIENT_ID);
        String responseType = request.getParameter(OAuth2Utils.RESPONSE_TYPE);


        if (StrUtil.isBlank(scope) || StrUtil.isBlank(clientId) || StrUtil.isBlank(responseType)) {
            log.info("missing authorize oauth2 param");
            Result r = Oauth2ExceptionCodeEnum.INVALID_REQUEST.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        Set<String> responseTypes = OAuth2Utils.parseParameterList(responseType);


        ClientDetails clientDetails;
        try {
            clientDetails = this.clientDetailsService.loadClientByClientId(clientId);
        } catch (ClientRegistrationException e) {
            log.info("clientId [{}] not found ",clientId);
            Result r = Oauth2ExceptionCodeEnum.INVALID_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        Set<String> scopes = clientDetails.getScope();
        if (!scopes.contains(scope)){
            log.info("invalid scope [{}]",scope);
            Result r = Oauth2ExceptionCodeEnum.INVALID_SCOPE.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        if (!responseTypes.contains("token") && !responseTypes.contains("code")) {
            log.info("Unsupported response types: {}" , responseTypes);
            Result r = Oauth2ExceptionCodeEnum.INVALID_GRANT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        filterChain.doFilter(request,response);
    }
}

还有需要注意是：因为是授权码模式，在获取token的时候，需要传clientId和clientSecret，这是获取token的验证我们需要先于一步再到后面，所以我们会定义个过滤器处理,因为在测试过程，如果不先于验证clientId和clientSecret，返回的错误可能不是预期之中的，这里还不清楚为啥会出现那些特殊情况，所以还是先于一步先校验了、

认证服务器配置上自定义的filter，这个filter会先执行再到后面/oauth2/token链接，还有这里不开启allowFormAuthenticationForClients，还是只限于basic，如果开启了，有可能返回不正确的异常格式，具体原因不明，但后续肯定是不走basicFilter的,自定义的filter还是验证参数完整性和认证clientId和clientSecret身份是否正确。

@Override
    public void configure(AuthorizationServerSecurityConfigurer security) throws Exception {
        security.passwordEncoder(passwordEncoder);
        // 开启/oauth/check_token验证端口认证权限访问
        security.checkTokenAccess("isAuthenticated()");
        // 开启/oauth/token_key验证端口无权限访问
        security.tokenKeyAccess("permitAll()");
        /*
         *
         * 主要是让/oauth/token支持client_id和client_secret做登陆认证
         * 如果开启了allowFormAuthenticationForClients，那么就在BasicAuthenticationFilter之前
         * 添加ClientCredentialsTokenEndpointFilter,使用ClientDetailsUserDetailsService来进行登陆认证
         *
         * 前后分离返回统一json不开启
         */
        //security.allowFormAuthenticationForClients();
        security.addTokenEndpointAuthenticationFilter(customBasicTokenFilter);
    }

@Slf4j
@Component
public class CustomBasicTokenFilter extends OncePerRequestFilter {


    /**
     * accessToken 和refreshToken 都使用该链接,只有三种模式走这个链接，隐藏式implicit只走authorize，其responseType=token
     */
    private static final String OAUTH2_AUTHORIZATION_URL = "/oauth/token";
    private static final String AUTHENTICATION_SCHEME_BASIC = "Basic";

    private static final String AUTHORIZATION_CODE = "authorization_code";
    private static final String PASSWORD = "password";
    private static final String REFRESH_TOKEN = "refresh_token";
    private static final String CLIENT_CREDENTIALS = "client_credentials";

    private static final Set<String> grantTypeSet = CollectionUtil.newHashSet(
            AUTHORIZATION_CODE,
            PASSWORD,
            REFRESH_TOKEN,
            CLIENT_CREDENTIALS);

    @Resource
    private ClientDetailsService clientDetailsService;

    @Resource
    private PasswordEncoder passwordEncoder;

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain filterChain) throws ServletException, IOException {

        if (!request.getRequestURI().equals(OAUTH2_AUTHORIZATION_URL)) {
            filterChain.doFilter(request, response);
            return;
        }

        String grantType = request.getParameter("grant_type");
        if (StrUtil.isBlank(grantType)) {
            log.info("missing grant type");
            Result r = CommonCodeEnum.COMMON_MISSING_PARAM.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        if (!grantTypeSet.contains(grantType)) {
            log.info("invalid grant type");
            Result r = Oauth2ExceptionCodeEnum.INVALID_GRANT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        //授权码模式需要
        /*
           http://ncsf6sb4:37bc1v7l8c@127.0.0.1:8547/oauth/token?
           scope=user_info&redirect_uri=http://127.0.0.1:8549/client/receiveCode&grant_type=authorization_code&code=Vky6Ie
          //请求头放clientId和clientSecret
         */
        String scope = request.getParameter("scope");
        String redirectUri = request.getParameter("redirect_uri");
        String code = request.getParameter("code");

        //密码模式需要 password
        // http://localhost:8080/oauth/token?username=hellxz&password=xyz&scope=read_scope&grant_type=password
        //请求头放clientId和clientSecret
        String username = request.getParameter("username");
        String password = request.getParameter("password");

        //客户端(凭证)模式只需要grantType
        // http://localhost:8080/oauth/token?grant_type=client_credentials
        //请求头放clientId和clientSecret
        String refreshToken = request.getParameter("refresh_token");

        //刷新token模式只需要grantType和refresh_token
        // http://ncsf6sb4:37bc1v7l8c@127.0.0.1:8547/oauth/token?grant_type=refresh_token&refresh_token=a9d43c25-c181-45a2-ba11-3f8cd5d6f3f4
        //请求头放clientId和clientSecret
        if (grantType.equals(AUTHORIZATION_CODE)) {
            if (StrUtil.isBlank(scope) || StrUtil.isBlank(redirectUri) || StrUtil.isBlank(code)) {
                log.info("{} mode : missing request token param", AUTHORIZATION_CODE);
                Result r = CommonCodeEnum.COMMON_MISSING_PARAM.toResult();
                ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                return;
            }
        }

        if (grantType.equals(PASSWORD)) {
            if (StrUtil.isBlank(username) || StrUtil.isBlank(password)) {
                log.info("{} mode : missing request token param", PASSWORD);
                Result r = CommonCodeEnum.COMMON_MISSING_PARAM.toResult();
                ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                return;
            }
        }

        if (grantType.equals(REFRESH_TOKEN)) {
            if (StrUtil.isBlank(refreshToken)) {
                log.info("{} mode : missing request token param", REFRESH_TOKEN);
                Result r = CommonCodeEnum.COMMON_MISSING_PARAM.toResult();
                ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
                return;
            }
        }

        Authentication authentication = SecurityContextHolder.getContext().getAuthentication();

        if (authentication != null && authentication.isAuthenticated()) {
            filterChain.doFilter(request, response);
            return;
        }

        //不开启form，因为其不走该filter，采用basic

        String header = request.getHeader(HttpHeaders.AUTHORIZATION);
        if (header == null) {
            log.info("Basic Authentication Authorization header not found");
            Result r = Oauth2ExceptionCodeEnum.INVALID_REQUEST.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        header = header.trim();
        if (!StringUtils.startsWithIgnoreCase(header, AUTHENTICATION_SCHEME_BASIC)) {
            log.info("Basic Authentication Authorization header not found");
            Result r = Oauth2ExceptionCodeEnum.INVALID_REQUEST.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        if (header.equalsIgnoreCase(AUTHENTICATION_SCHEME_BASIC)) {
            log.info("Empty basic authentication token");
            Result r = Oauth2ExceptionCodeEnum.UNAUTHORIZED_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        byte[] base64Token = header.substring(6).getBytes(StandardCharsets.UTF_8);
        byte[] decoded;
        try {
            decoded = Base64.getDecoder().decode(base64Token);
        } catch (IllegalArgumentException e) {
            log.info("Failed to decode basic authentication token");
            Result r = Oauth2ExceptionCodeEnum.UNAUTHORIZED_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        String token = new String(decoded, StandardCharsets.UTF_8);

        int delimit = token.indexOf(":");

        if (delimit == -1) {
            log.info("Invalid basic authentication token");
            Result r = Oauth2ExceptionCodeEnum.UNAUTHORIZED_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        String clientId = token.substring(0, delimit);
        String clientSecret = token.substring(delimit + 1);

        if (StrUtil.isBlank(clientId) || StrUtil.isBlank(clientSecret)) {
            log.info("client_id or client_secret not found in form or basic");
            Result r = Oauth2ExceptionCodeEnum.INVALID_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        ClientDetails clientDetails;
        try {
            clientDetails = this.clientDetailsService.loadClientByClientId(clientId);
        } catch (ClientRegistrationException e) {
            log.info("clientId [{}] not exist ", clientId);
            Result r = Oauth2ExceptionCodeEnum.INVALID_CLIENT.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        String bcryptSecret = clientDetails.getClientSecret();
        if (!passwordEncoder.matches(clientSecret, bcryptSecret)) {
            log.info("invalid clientSecret [}{]", clientSecret);
            Result r = Oauth2ExceptionCodeEnum.INVALID_CLIENT_SECRET.toResult();
            ResponseUtil.jsonResp(response).getWriter().write(JSONUtil.toJsonStr(r));
            return;
        }

        Collection<GrantedAuthority> authorities = clientDetails.getAuthorities();

        UsernamePasswordAuthenticationToken resultToken = new UsernamePasswordAuthenticationToken(clientId
                , bcryptSecret, authorities);

        SecurityContextHolder.getContext().setAuthentication(resultToken);

        filterChain.doFilter(request, response);
    }
}

例示参考：
https://blog.csdn.net/qq_31063463/article/details/83752459