Java→gRPC→Python业务功能调用实例

  写完这个Java→gRPC→Python业务功能调用实例,实验服务器的架构图就全部完成,加上最后一块砖封顶了。

  本篇在gRPC自带的HellowWorld例子上增加一个功能调用,调用前面文章中建立的墨尔本房价分析模型,服务器部分用Python实现,客户端是Java的Tomcat Web App,用浏览器访问。

一、运行效果

1、浏览器输入。

选择回归算法及输入异常值阀值

2、服务器处理请求。

服务器打印收到的请求信息以便调试

3、客户端接收服务器返回结果。

客户端接收异常房价列表并查询原始数据

  这里为了在一个页面上显示返回的异常数据及原始数据,将异常值阀值设为偏差60%,返回了6条记录。训练集有7212条数据,验证集有1803条数据,回归算法中准确率较好的已超过90%,非常高了。

二、Python部分

1、proto接口原型文件helloworld.proto。

  增加了一个RPC调用GetOutliers(),以及它的调用参数message MelbourneRequest{}和返回结果message MelbourneReply{},注意返回结果是不定长的列表,所以前面加stream定义返回结果为一个流,gRPC的一个优点是支持流操作,非常方便。

// Copyright 2015 The gRPC Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
  
  // Get outliers of Melbourne house pricing for a given algo and threshold.
  rpc GetOutliers (MelbourneRequest) returns (stream MelbourneReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}


// Added for the Melbourne Example by Jean, 2022/12/13.
// The request message containing the algo's name and threshhold for outliers.
message MelbourneRequest {
  string algo = 1;
  double  threshold = 2;
}

// The response message containing the outliers.
message MelbourneReply {
  int64 row = 1;
  double origin =2;
  double predict =3;
  double se =4;
}

  Protocol Buffers项目文档中有各种语言之间的数据类型转换对照表,要按对照表来定义参数的类型,Java中用double比float要方便一点。

Protocol Buffers数据类型对照表

  定义好接口原型文件后要先编译它,以便在Python服务器程序中引用新增加的接口类。

[jean@VM-4-12-centos grpcTest]$ pwd
/home/jean/grpcTest
[jean@VM-4-12-centos grpcTest]$ python -m grpc_tools.protoc  -I.  --python_out=. --pyi_out=. --grpc_python_out=. helloworld.proto

2、Python服务器程序greeter_server_SSL_Auth.py。

  增加了三个变量,各个回归算法在训练集的拟合结果train、验证集的拟合结果valid及算法性能perf,服务器启动时在主程序中直接执行与前面Shiny APP共享的Melbourne_Regress.py脚本加载预处理好的数据与各个算法最优参数的回归模型,然后初始化上述三个变量。具体脚本及参数优化请参阅前面的文章

  在服务类Greeter()中增加了提供服务的方法GetOutliers(),根据请求中的算法与阀值从验证集拟合结果valid中选出异常值,每个异常值一行,生成一个接口原型中定义的MelbourneReply对象outlier,用yield outlier以stream的形式返回给客户端。gRPC 官方文档中文版中有Python stream用法的说明。

# Copyright 2015 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Python implementation of the GRPC helloworld.Greeter server."""

from concurrent import futures
import logging

import grpc
import helloworld_pb2
import helloworld_pb2_grpc

import time
import pandas as pd
import numpy as np

train = None
valid = None
perf  = None


# The Class to serve the client with a Function defined by tha proto file.
class Greeter(helloworld_pb2_grpc.GreeterServicer):

    def SayHello(self, request, context):
        print(request.name)
        return helloworld_pb2.HelloReply(message='Hello, %s!' % request.name)


    def GetOutliers(self, request, context):
        print(request.algo)
        print(request.threshold)
        
        test = valid[["origin",request.algo]]
        test.index.name = 'row'
        test.reset_index(inplace=True)
        test.rename(columns={request.algo:"predict"},inplace=True)
        test["origin2"] = np.exp(test["origin"])
        test["predict2"] = np.exp(test["predict"])
        test["se"] = np.round((test["predict2"]- test["origin2"])/test["origin2"]*100,1)
        outliers =test.loc[np.abs(test["se"])>= request.threshold]
        outliers = outliers[["row","origin2","predict2","se"]]
        outliers.rename(columns={"origin2":"origin","predict2":"predict"},inplace=True)
        outliers.sort_values(["se"],ascending = True, inplace=True)

        for index, onerow in outliers.iterrows():
          outlier = helloworld_pb2.MelbourneReply(\
            row = int(onerow["row"]),\
            origin = onerow["origin"],\
            predict = onerow["predict"],\
            se = onerow["se"]
            )
          # print(outlier)
          yield outlier
        

# For authenticating with a password.
class AuthInterceptor(grpc.ServerInterceptor):
    def __init__(self, key):
        # 'rpc-auth-header' is the header data containing the password.
        self._valid_metadata = ('rpc-auth-header', key)

        def deny(_, context):
            context.abort(grpc.StatusCode.UNAUTHENTICATED, 'Invalid key')

        self._deny = grpc.unary_unary_rpc_method_handler(deny)

    def intercept_service(self, continuation, handler_call_details):
        meta = handler_call_details.invocation_metadata
        print(meta)
        # https://grpc.io/docs/guides/auth/#extending-grpc-to-support-other-authentication-mechanisms
        # There's a bug in the document, meta[0] should be meta[1] for newer versions.
        if meta and meta[1] == self._valid_metadata:
            return continuation(handler_call_details)
        else:
            return self._deny


# The server procedure.
def serve():
    port = '50051'
    # Now requires authentication with a password by attaching an interceptor.
    # 'access_key' is the password.
    server = grpc.server(
                futures.ThreadPoolExecutor(max_workers=10),
                interceptors=(AuthInterceptor('access_key'),)
             )
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    # Load the server's certificate chain & private key.
    # So the server should be run as root.
    with open('/root/cert/server.key', 'rb') as f:
        private_key = f.read()
    with open('/root/cert/server.crt', 'rb') as f:
        certificate_chain = f.read()
    with open('/root/cert/ca.crt', 'rb') as f:
        root_certificates = f.read()
    # False means client cert is not required.
    server_credentials = grpc.ssl_server_credentials(((private_key, certificate_chain),), root_certificates,False)
    # '[::]:' means may be accessed from any IP.
    server.add_secure_port('[::]:' + port, server_credentials)    
    server.start()
    print("Server started, listening on " + port)
    # May be interrupted by Ctrl+C.
    server.wait_for_termination()


if __name__ == '__main__':
  
    print("Loading Melbourne models......")
    t_started = time.time()
    exec(open("/home/jean/scripts/Melbourne_Regress.py").read())
    train = PredictTrain()
    valid = PredictValid()
    perf = performance()
    t_finished = time.time()
    print("Melbourne models loaded, "+str(round(t_finished-t_started,1))+" seconds.")
  
    logging.basicConfig()
    serve()

3、Python客户端程序greeter_client_SSL_Auth.py。

  Python客户端程序是测试用途,以验证服务器程序工作正常。先构造一个接口原型中定义的MelbourneRequest对象,然后调用GetOutliers()函数,返回的stream在Python客户端中是一个Iterator,用for语句遍历,具体可参阅gRPC 官方文档中文版

    print("Outlier client received: \n")
    for outlier in stub.GetOutliers(helloworld_pb2.MelbourneRequest(algo='cat', threshold=65)):
        print(outlier)

  完整的程序。

# Copyright 2015 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Python implementation of the GRPC helloworld.Greeter client."""

from __future__ import print_function

import logging

import grpc
import helloworld_pb2
import helloworld_pb2_grpc

# Class to add authentication header to the meta data of every gRPC call.
class GrpcAuth(grpc.AuthMetadataPlugin):
    def __init__(self, key):
        self._key = key

    # 'rpc-auth-header' is the authentication header defined by the server.
    def __call__(self, context, callback):
        callback((('rpc-auth-header', self._key),), None)


def run():
    # NOTE(gRPC Python Team): .close() is possible on a channel and should be
    # used in circumstances in which the with statement does not fit the needs
    # of the code.
    print("Will try to greet world ...")
    # Will fail for enpty credentials.
    # creds = grpc.ssl_channel_credentials()
    # Need to include root certificate in credentials.
    # https://stackoverflow.com/questions/72230151/how-to-open-a-secure-channel-in-python-grpc-client-without-a-client-ssl-certific
    # Copy the selfsigned CA's root cert to the client directory, so that the client doesn't need to be run as root.
    with open('ca.crt', 'rb') as f:
        creds = grpc.ssl_channel_credentials(f.read())
    
    # A composite channel credentials with SSL and password.
    # Host name need to be the same as the server certificate. 
    channel = grpc.secure_channel(
        'jeanye.cn:50051',
        grpc.composite_channel_credentials(
            creds,
            grpc.metadata_call_credentials(
                GrpcAuth('access_key')
                # A worng key will fail then.
                # GrpcAuth('wrong_access_key')
            )
        )
    )
        
    stub = helloworld_pb2_grpc.GreeterStub(channel)
    response = stub.SayHello(helloworld_pb2.HelloRequest(name='Jean'))
    print("Greeter client received: " + response.message)
    
    print("Outlier client received: \n")
    for outlier in stub.GetOutliers(helloworld_pb2.MelbourneRequest(algo='cat', threshold=65)):
        print(outlier)
    


if __name__ == '__main__':
    logging.basicConfig()
    run()

  运行Python客户端程序。

服务器端打印请求的算法与阀值以便调试

Python客户端正确运行

三、Java客户端

  演示性质,暂时还是在Java客户端项目中生成存根stub类,然后拷贝到Tomcat Web项目中,用Java客户端项目先测试也要方便一点。

1、proto接口原型文件helloworld.proto。

  客户端项目同上篇文章配置,拷贝上面的接口原型文件到Java客户端项目中更新原来的文件。项目名grpcclient 上右键-> Run As -> Maven Build,重新生成Java的stub类。

2、HelloWorldClientSSLAuthMelbourne.java。

  Java客户端操作gRPC stream的例子可以参阅grpc-java自带的RouteGuide例子及其proto文件。gRPC stream在Java客户端也是一个java.util.Iterator类,可以通过for()循环去迭代,用hasNext()判断是否结束,用next()去获取下一个返回的MelbourneReply对象,存储为一个java.util.List以便后续处理。注意blockingStub.getOutliers(),与接口原型定义中相比,java方法的首字母按Java函数的惯例都改成了小写。

package io.grpc.examples.helloworld;

import io.grpc.Channel;
import io.grpc.ClientInterceptor;
import io.grpc.ClientInterceptors;
import io.grpc.ManagedChannel;
import io.grpc.StatusRuntimeException;
import io.grpc.examples.header.HeaderClientInterceptor;
import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts;
import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.io.File;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * A simple client that requests a greeting from the {@link HelloWorldServer}.
 */
public class HelloWorldClientSSLAuthMelbourne {
  private static final Logger logger = Logger.getLogger(HelloWorldClient.class.getName());

  private final GreeterGrpc.GreeterBlockingStub blockingStub;

  /** Construct client for accessing HelloWorld server using the existing channel. */
  public HelloWorldClientSSLAuthMelbourne(Channel channel) {
    // 'channel' here is a Channel, not a ManagedChannel, so it is not this code's responsibility to
    // shut it down.

    // Passing Channels to code makes code easier to test and makes it easier to reuse Channels.
    blockingStub = GreeterGrpc.newBlockingStub(channel);
  }

  
  /** Get outliers of Melbourne house price. */
  public List<MelbourneReply> getOutliers(String algo, double threshold) {
    logger.info("Will try to get outliers for algo " + algo +" with threshold "+ threshold + " ...");
    MelbourneRequest request = MelbourneRequest.newBuilder().setAlgo(algo).setThreshold(threshold).build();
    Iterator<MelbourneReply>  response;
   
    List<MelbourneReply> result = new ArrayList<>();
    try {
        response = blockingStub.getOutliers(request);
        for (int i = 1; response.hasNext(); i++) {
            MelbourneReply outlier = response.next();
            result.add(outlier);
            System.out.println("Result #" + i + ": "+outlier.getRow()+" | "+ Math.round(outlier.getOrigin()*100.0)/100.0+" | "+Math.round(outlier.getPredict()*100.0)/100.0+" | "+outlier.getSe());
        }
      } catch (StatusRuntimeException e) {
          logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
        return null;
      }
    logger.info("Get outliers of Melbourne house price.");
    return result;
  }  
  
  /**
   * Greet server. If provided, the first element of {@code args} is the name to use in the
   * greeting. The second argument is the target server.
   */
  public static void main(String[] args) throws Exception {
    /* 
     * Use NettyChannelBuilder to build SSL connection to server.
     * https://stackoverflow.com/questions/42700411/ssl-connection-for-grpc-java-client
     * Need to include selfsigned CA certificate to build a sslContext to verify the server.
     * And the server name must be same as the name in the server certificate.
     */
    
    ManagedChannel originChannel = NettyChannelBuilder.forAddress("jeanye.cn", 50051)
            .sslContext(GrpcSslContexts.forClient().trustManager(new File("d:/temp/ca.crt")).build())
            .build(); 
    
    ClientInterceptor interceptor = new HeaderClientInterceptor();
    Channel channel = ClientInterceptors.intercept(originChannel, interceptor);

    try {
        HelloWorldClientSSLAuthMelbourne client = new HelloWorldClientSSLAuthMelbourne(channel);
        List<MelbourneReply> res = client.getOutliers("cat",60.0);
    } finally {
      // ManagedChannels use resources like threads and TCP connections. To prevent leaking these
      // resources the channel should be shut down when it will no longer be used. If it may be used
      // again leave it running.
        originChannel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
    }
  }
}

  运行结果:

Java客户端运行正确

四、Tomcat Web App

1、拷贝上面Java客户端项目生成的stub类更新。

GreeterGrpc.java                        HelloReply.java                         
HelloReplyOrBuilder.java                HelloRequest.java                       
HelloRequestOrBuilder.java              HelloWorldProto.java                    
MelbourneReply.java                     MelbourneReplyOrBuilder.java
MelbourneRequest.java                   MelbourneRequestOrBuilder.java

2、HelloWorldClientSSLAuthMelbourne.java。

  参考上面的Java客户端编写,实现gRPC调用的帮助类。

package io.grpc.examples.helloworld;

import io.grpc.Channel;
import io.grpc.ClientInterceptor;
import io.grpc.ClientInterceptors;
import io.grpc.ManagedChannel;
import io.grpc.StatusRuntimeException;
import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts;
import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder;

import java.io.File;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * A simple client that requests a greeting from the {@link HelloWorldServer}.
 */
public class HelloWorldClientSSLAuthMelbourne {
    private static final Logger logger = Logger.getLogger(HelloWorldClient.class.getName());

    private final GreeterGrpc.GreeterBlockingStub blockingStub;

    private ManagedChannel originChannel = null;

    private Channel channel = null;

    /** Construct client for accessing HelloWorld server by creating a channel. */
    public HelloWorldClientSSLAuthMelbourne(String caFile, String server, Integer port) throws Exception {

        /*
         * Use NettyChannelBuilder to build SSL connection to server.
         * https://stackoverflow.com/questions/42700411/ssl-connection-for-grpc-java-
         * client Need to include selfsigned CA certificate to build a sslContext to
         * verify the server. And the server name must be same as the name in the server
         * certificate.
         */
        originChannel = NettyChannelBuilder.forAddress(server, port)
                .sslContext(GrpcSslContexts.forClient().trustManager(new File(caFile)).build()).build();
        // "access_key" is the password to call gRPC.
        ClientInterceptor interceptor = new HeaderClientInterceptor("access_key");
        // Channel with an interceptor to add an auth header for each gRPC call.
        channel = ClientInterceptors.intercept(originChannel, interceptor);

        // 'channel' here is a Channel, not a ManagedChannel, so it is not this code's
        // responsibility to shut it down.
        // Passing Channels to code makes code easier to test and makes it easier to
        // reuse Channels.
        blockingStub = GreeterGrpc.newBlockingStub(channel);
    }

    /** Say hello to server. */
    public String greet(String name) {
        logger.info("Will try to greet " + name + " ...");
        HelloRequest request = HelloRequest.newBuilder().setName(name).build();
        HelloReply response;
        try {
            response = blockingStub.sayHello(request);
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return (e.toString());
        }
        logger.info("Greeting: " + response.getMessage());
        return (response.getMessage());
    }

    /** Get ooutliers of Melbourne house price. */
    public List<MelbourneReply> getOutliers(String algo, double threshold) {
        logger.info("Will try to get outliers for algo " + algo + " with threshold " + threshold + " ...");
        MelbourneRequest request = MelbourneRequest.newBuilder().setAlgo(algo).setThreshold(threshold).build();
        Iterator<MelbourneReply> response;

        List<MelbourneReply> result = new ArrayList<>();
        try {
            response = blockingStub.getOutliers(request);
            for (int i = 1; response.hasNext(); i++) {
                MelbourneReply outlier = response.next();
                result.add(outlier);
                System.out.println("Result #" + i + ": " + outlier.getRow() + " | " + outlier.getOrigin() + " | "
                        + outlier.getPredict() + " | " + outlier.getSe());
            }
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return null;
        }
        logger.info("Get outliers of Melbourne house price.");
        return result;
    }

    /** Close the channel. */
    public void close() throws Exception {
        try {
            // ManagedChannels use resources like threads and TCP connections. To prevent
            // leaking these
            // resources the channel should be shut down when it will no longer be used. If
            // it may be used again leave it running.
            originChannel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

3、algo.jsp。

  输入页面,选择算法及输入异常值阀值。

<%@ page language="java" contentType="text/html; charset=GBK"
    pageEncoding="GBK"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=GBK">
<title>Java EE gRPC调用测试</title>
</head>
<body>
<center>
 <form action="melbourne.jsp" method="post">
 墨尔本房价gRPC调用示例 <br/>
  算法:<select name="algo" id="algo">
      <option value="svm">SVM</option>
      <option value="rf">RandomForest</option>
      <option value="gbr">GBR</option>
      <option value="xgb">XGB</option>
      <option value="lgbm">LigthGBM</option>
      <option value="cat" selected>CatBoost</option>
      <option value="blend">Blend</option>      
      </select> 
      <br/>
  异常值阀值:<input type="number" name="threshold" value=60><br/>
 <input type="submit" value="查看异常值">
 </form>
 </center>
</body>
</html>

4、melbourne.jsp。

  显示gRPC调用结果及匹配的原始数据,所以项目中打包了原始数据文件(预处理)Melbourne_housing_pre.csv。

<%@ page language="java" contentType="text/html; charset=GBK"
    pageEncoding="GBK"%>
<%@ page import="io.grpc.examples.helloworld.*,java.util.*,java.io.BufferedReader,java.io.FileReader"%>    
    
<%

List<MelbourneReply> resp =null;
Set<Long> set = new HashSet<Long>();
try{
    
    String algo = request.getParameter("algo");
    Double threshold = new Double(request.getParameter("threshold"));
    if (algo==null) algo = "cat";
    //String server = "localhost";
    String server = "jeanye.cn";
    Integer port = 50051;
    
    // Plain text without SSL & password.
    // HelloWorldClient client = new HelloWorldClient(server, port);
    String caFile =  application.getRealPath("/WEB-INF/ca.crt");
    // SSL without password.
    // HelloWorldClientSSL client = new HelloWorldClientSSL(caFile,server,port); 
    // SSL with password.
    //HelloWorldClientSSLAuth client = new HelloWorldClientSSLAuth(caFile,server,port);  
    HelloWorldClientSSLAuthMelbourne client = new HelloWorldClientSSLAuthMelbourne(caFile,server,port);    
   

    resp = client.getOutliers(algo, threshold);
    client.close();
    
}catch(Exception e){
    e.printStackTrace();
}

%>    
<!DOCTYPE html>
<html>
<head>
<meta charset="GBK">
<title>Java EE 调用gRPC 测试</title>
</head>
<body>
<h3>Java gRPC调用服务器端Python程序结果  <a href="algo.jsp">返回</a></h3>
<%
if (resp !=null){
%>
<table border="1" cellspacing="0" bordercolor="#D3D3D3">
<tr><td>行号</td><td>真实值</td><td>预测值</td><td>偏差%</td>
</tr>
<%
    for(Iterator it=resp.iterator();it.hasNext();){
        MelbourneReply outlier = (MelbourneReply)it.next();
        set.add(outlier.getRow());
%>
<tr>
<td><%= outlier.getRow()%></td>
<td><%=  Math.round(outlier.getOrigin()*100.0)/100.0%></td>
<td><%= Math.round(outlier.getPredict()*100.0)/100.0%></td>
<td><%= outlier.getSe()%></td>
</tr>
<%
    }
%>
</table>
<%
}
%>
<br>

<h3>匹配的原始数据</h3>
<table border="1" cellspacing="0" bordercolor="#D3D3D3">
<tr>
<% 
String path = request.getRealPath("/WEB-INF");
System.out.println(path);
BufferedReader reader = new BufferedReader(new FileReader(path+"/Melbourne_housing_pre.csv"));
// skip header
String header[] =reader.readLine().split(",");
header[0] = "Row";
for (int i=0; i<header.length;i++){
%>
<td><%=header[i]%></td>
<%}%></tr>
<%
String line = null;
long index=0; 
//读取每行,直到为空
while((line=reader.readLine())!=null){ 
    if (set.contains(index)){
        String items[] = line.split(",");
%><tr><%
        for (int i=0; i< items.length; i++){
%><td><%=items[i] %></td>
<%
        }%></tr><%
    }
    index++;
}
%>
</table>
<br>

<a href="algo.jsp">返回</a>
</body>
</html>

5、本机测试。

Tomcat本地测试选择算法与阀值

服务器发返回结果正确

6、部署到服务器上运行,见本篇开头的运行效果。

五、用Maven管理Tomcat项目

  参考资料。用Maven管理Tomcat项目的好处,一是可以在项目中根据proto接口原型定义生成Java stub类,不用从Java客户端项目中拷贝;二是开发时不用逐个导入依赖的包,Maven会自动解决依赖关系,下载导入相应的包;三是打包成war发布到服务器时,Maven会自动把依赖包放进去。这样就方便多了。程序与上面的Tomcat Web App是一样的,只是项目的组织结构有所不同。

1、建立Maven管理的Web APP项目。

Eclipse->File->New Maven Project,后面要选择archetype,不要选simple project。

建立一个Maven项目

Catalog选All Catalogs,Filter中输入maven-archetype-webapp,项目类型选Group Id: org.apache.maven.archetypes,Artifact Id: maven-archetype-webapp,注意它的版本是1.4。如果Catalog选了Internal,也可以列出来,但它会是内置的1.0版,二者生成的pom.xml是不一样的。All Catalog要联网到各个catalog去下载archetype列表(可以按configure按钮配置),取决于网络的状况,有时候可能会出不来,有时候可能要等几十秒。

选择正确的Group Id和 Artifact Id

Internal自带的1.0版生成不同的pom.xml,本篇用1.4版

为自己的项目定义Group Id与Artifact Id,因为已经建好了grpc项目,用grpcTest演示,所以前后截图的项目名可能会有所不同。

为自己的项目定义Group Id与Artifact Id

项目名上右键->Properties->Java Build Path->Add Library,Server Runtime把Tomcat Server Runtime加入,否则JSP编译报错。JRE System Library把适当的JDK加入,由于我的笔记本上JDK11的版本比服务器上JDK11的版本新,导致服务器上部署时报java.lang.unsupportedclassversionerror,我在笔记本上降为JDK8。此时Maven Dependencies只有基本的junit等,后面再加。

设置Java Build Path

此时可以在项目名上右键->Run As->Run on Server,选择前面实验中配好的Tomcat服务器,把新建的项目发布到Tomcat上,在Eclipse中浏览首页,验证项目正确创建,它会自动生成web.xml。

启动Maven管理的Tomcat项目

2、更新pom.xml。

1)、将java版本设为与build path一样的1.8。

2)、把前面gRPC Java客户端项目中pom.xml的dependency拷贝过来。

3)、因为不了解<pluginManagement>的用法,新建了一个<plugins>,把Java客户端项目中pom.xml的generate-sources一节的plugin拷贝过来,用来生成gRPC的Java stub类源码。参阅资料

4)、<build>里的<defaultGoal>也可以拷贝过来,generate-sources要排在clean后,其它任务的前面。
然后在项目名上右键->Maven->Update Project,更新Maven项目的Java Build Path引用。如果是本地第一次引用这些包,Maven联网下载可能需要一点时间。

完整的源码如下:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>jean.test</groupId>
    <artifactId>grpc</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>

    <name>grpc Maven Webapp</name>
    <!-- FIXME change it to the project's website -->
    <url>http://www.example.com</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.21.7</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-netty-shaded</artifactId>
            <version>1.51.0</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-protobuf</artifactId>
            <version>1.51.0</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-stub</artifactId>
            <version>1.51.0</version>
        </dependency>
        <!-- Removed from JDK11 and above, need to be added here -->
        <!-- https://blog.csdn.net/ml863606/article/details/109202246?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_baidulandingword~default-0-109202246-blog-120904477.pc_relevant_3mothn_strategy_recovery&spm=1001.2101.3001.4242.1&utm_relevant_index=3 -->
        <dependency>
            <groupId>javax.annotation</groupId>
            <artifactId>javax.annotation-api</artifactId>
            <version>1.3.2</version>
        </dependency>

    </dependencies>

    <build>
        <defaultGoal>clean generate-sources compile install</defaultGoal>

        <finalName>grpc</finalName>
        <pluginManagement><!-- lock down plugins versions to avoid using Maven 
                defaults (may be moved to parent pom) -->
            <plugins>
                <plugin>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>3.1.0</version>
                </plugin>
                <!-- see http://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_war_packaging -->
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>3.0.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.8.0</version>
                </plugin>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-war-plugin</artifactId>
                    <version>3.2.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-install-plugin</artifactId>
                    <version>2.5.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>2.8.2</version>
                </plugin>
            </plugins>
        </pluginManagement>
        
        <plugins>
                <!-- compile proto file into java files -->
                <plugin>
                    <groupId>com.github.os72</groupId>
                    <artifactId>protoc-jar-maven-plugin</artifactId>
                    <version>3.11.1</version>
                    <executions>
                        <execution>
                            <phase>generate-sources</phase>
                            <goals>
                                <goal>run</goal>
                            </goals>
                            <configuration>
                                <includeMavenTypes>direct</includeMavenTypes>
                                <inputDirectories>
                                    <include>src/main/resources</include>
                                </inputDirectories>
                                <outputTargets>
                                    <outputTarget>
                                        <type>java</type>
                                        <outputDirectory>src/main/java</outputDirectory>
                                    </outputTarget>
                                    <outputTarget>
                                        <type>grpc-java</type>
                                        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.51.0</pluginArtifact>
                                        <outputDirectory>src/main/java</outputDirectory>
                                    </outputTarget>
                                </outputTargets>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>       
        </plugins>
        
    </build>
</project>

更新好的包依赖关系

3、生成gRPC的Java stub类。

把前面Java客户端项目中的接口原型定义文件helloworld.proto拷贝过来,在项目名字上右键->Run As->Maven generate-sources,则会执行pom.xml中定义的generate-sources任务生成gRPC stub类。注意如果手工删除了生成的stub类,又没有更新proto原型文件,这个操作会直接跳过,什么都不做。更新一下proto原型文件,让它有个新的时间戳,重新运行即可。Maven有这个右键菜单是因为在pom.xml中定义了它。

生成gRPC的Java stub类

Eclipse中有两个可以编辑proto文件的插件,提供了语法高亮显示等功能,我安装了protobuf-dt,具体安装方法请参阅其项目主页上的说明,它有一个小问题,就是不认识proto文件中的option语句,原因是其打包的descriptor.proto文件比较旧,参考该帖子,通过WinRAR用一个比较新的替换(D:/eclipse/plugins/com.google.eclipse.protobuf_2.3.2.201609161849.jar内的)即可,我用了Anaconda3安装好的一个。

编辑proto文件的插件protobuf-dt

替换descriptor.proto

找到版本比较新的descriptor.proto

另外,Eclipse->Window->Preferences->Protocol Buffer->Compiler中,默认不勾选"Compile .proto files on save"选项,不要改它,因为我们用Run As->Maven gererate-sources来生成,勾选了每次更改保存都调Compiler,然后会报错,应该是还没有配好:

Errors occurred during the build.
Errors running builder 'Xtext Project Builder' on project 'grpc'.
'void com.google.common.io.Closeables.closeQuietly(java.io.Closeable)'
保存proto文件时不要编译

4、把前面Tomcat Web App项目中的程序文件拷贝到Maven Tomcat Web App项目中。包括前面图中的HelloWorldClient*.java,HeaderClient*.java,*.jsp,以及WEB-INF下的资源文件ca.crt、Melbourne_housing_pre.csv,详见前面图中的项目结构。

然后在algo.jsp上右键->Run As->Run on Server,发布到前面配好的Tomcat服务器,运行测试,这与平常是一样的。

测试Maven-Tomcat项目

测试Maven-Tomcat项目

5、打包发布到Linux服务器上。

项目名字上右键->Run As->Maven Build,因为pom.xml中定义了<defaultGoal>,直接按Run按钮就可以。

运行Maven Build打包

生成的war文件在项目的target目录下,可以在Eclipse的Console窗口看build过程的输出。


生成war文件发布到Linux服务器

在Linux服务器上测试

六、配置Python服务端开机自启动

# cd /etc/rc.d
# vi rc.local
# Startup Python gRPC server, added by Jean 2022/12/16.
cd /home/jean/grpcTest
/usr/lib64/anaconda3/bin/python  greeter_server_SSL_Auth.py
# reboot now

Done.

©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 212,686评论 6 492
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 90,668评论 3 385
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 158,160评论 0 348
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 56,736评论 1 284
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 65,847评论 6 386
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 50,043评论 1 291
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 39,129评论 3 410
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 37,872评论 0 268
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 44,318评论 1 303
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 36,645评论 2 327
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 38,777评论 1 341
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 34,470评论 4 333
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 40,126评论 3 317
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 30,861评论 0 21
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 32,095评论 1 267
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 46,589评论 2 362
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 43,687评论 2 351

推荐阅读更多精彩内容