写完这个Java→gRPC→Python业务功能调用实例,实验服务器的架构图就全部完成,加上最后一块砖封顶了。
本篇在gRPC自带的HellowWorld例子上增加一个功能调用,调用前面文章中建立的墨尔本房价分析模型,服务器部分用Python实现,客户端是Java的Tomcat Web App,用浏览器访问。
一、运行效果
1、浏览器输入。
2、服务器处理请求。
3、客户端接收服务器返回结果。
这里为了在一个页面上显示返回的异常数据及原始数据,将异常值阀值设为偏差60%,返回了6条记录。训练集有7212条数据,验证集有1803条数据,回归算法中准确率较好的已超过90%,非常高了。
二、Python部分
1、proto接口原型文件helloworld.proto。
增加了一个RPC调用GetOutliers(),以及它的调用参数message MelbourneRequest{}和返回结果message MelbourneReply{},注意返回结果是不定长的列表,所以前面加stream定义返回结果为一个流,gRPC的一个优点是支持流操作,非常方便。
// Copyright 2015 The gRPC Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";
package helloworld;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
// Get outliers of Melbourne house pricing for a given algo and threshold.
rpc GetOutliers (MelbourneRequest) returns (stream MelbourneReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
// Added for the Melbourne Example by Jean, 2022/12/13.
// The request message containing the algo's name and threshhold for outliers.
message MelbourneRequest {
string algo = 1;
double threshold = 2;
}
// The response message containing the outliers.
message MelbourneReply {
int64 row = 1;
double origin =2;
double predict =3;
double se =4;
}
Protocol Buffers项目文档中有各种语言之间的数据类型转换对照表,要按对照表来定义参数的类型,Java中用double比float要方便一点。
定义好接口原型文件后要先编译它,以便在Python服务器程序中引用新增加的接口类。
[jean@VM-4-12-centos grpcTest]$ pwd
/home/jean/grpcTest
[jean@VM-4-12-centos grpcTest]$ python -m grpc_tools.protoc -I. --python_out=. --pyi_out=. --grpc_python_out=. helloworld.proto
2、Python服务器程序greeter_server_SSL_Auth.py。
增加了三个变量,各个回归算法在训练集的拟合结果train、验证集的拟合结果valid及算法性能perf,服务器启动时在主程序中直接执行与前面Shiny APP共享的Melbourne_Regress.py脚本加载预处理好的数据与各个算法最优参数的回归模型,然后初始化上述三个变量。具体脚本及参数优化请参阅前面的文章。
在服务类Greeter()中增加了提供服务的方法GetOutliers(),根据请求中的算法与阀值从验证集拟合结果valid中选出异常值,每个异常值一行,生成一个接口原型中定义的MelbourneReply对象outlier,用yield outlier以stream的形式返回给客户端。gRPC 官方文档中文版中有Python stream用法的说明。
# Copyright 2015 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Python implementation of the GRPC helloworld.Greeter server."""
from concurrent import futures
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
import time
import pandas as pd
import numpy as np
train = None
valid = None
perf = None
# The Class to serve the client with a Function defined by tha proto file.
class Greeter(helloworld_pb2_grpc.GreeterServicer):
def SayHello(self, request, context):
print(request.name)
return helloworld_pb2.HelloReply(message='Hello, %s!' % request.name)
def GetOutliers(self, request, context):
print(request.algo)
print(request.threshold)
test = valid[["origin",request.algo]]
test.index.name = 'row'
test.reset_index(inplace=True)
test.rename(columns={request.algo:"predict"},inplace=True)
test["origin2"] = np.exp(test["origin"])
test["predict2"] = np.exp(test["predict"])
test["se"] = np.round((test["predict2"]- test["origin2"])/test["origin2"]*100,1)
outliers =test.loc[np.abs(test["se"])>= request.threshold]
outliers = outliers[["row","origin2","predict2","se"]]
outliers.rename(columns={"origin2":"origin","predict2":"predict"},inplace=True)
outliers.sort_values(["se"],ascending = True, inplace=True)
for index, onerow in outliers.iterrows():
outlier = helloworld_pb2.MelbourneReply(\
row = int(onerow["row"]),\
origin = onerow["origin"],\
predict = onerow["predict"],\
se = onerow["se"]
)
# print(outlier)
yield outlier
# For authenticating with a password.
class AuthInterceptor(grpc.ServerInterceptor):
def __init__(self, key):
# 'rpc-auth-header' is the header data containing the password.
self._valid_metadata = ('rpc-auth-header', key)
def deny(_, context):
context.abort(grpc.StatusCode.UNAUTHENTICATED, 'Invalid key')
self._deny = grpc.unary_unary_rpc_method_handler(deny)
def intercept_service(self, continuation, handler_call_details):
meta = handler_call_details.invocation_metadata
print(meta)
# https://grpc.io/docs/guides/auth/#extending-grpc-to-support-other-authentication-mechanisms
# There's a bug in the document, meta[0] should be meta[1] for newer versions.
if meta and meta[1] == self._valid_metadata:
return continuation(handler_call_details)
else:
return self._deny
# The server procedure.
def serve():
port = '50051'
# Now requires authentication with a password by attaching an interceptor.
# 'access_key' is the password.
server = grpc.server(
futures.ThreadPoolExecutor(max_workers=10),
interceptors=(AuthInterceptor('access_key'),)
)
helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
# Load the server's certificate chain & private key.
# So the server should be run as root.
with open('/root/cert/server.key', 'rb') as f:
private_key = f.read()
with open('/root/cert/server.crt', 'rb') as f:
certificate_chain = f.read()
with open('/root/cert/ca.crt', 'rb') as f:
root_certificates = f.read()
# False means client cert is not required.
server_credentials = grpc.ssl_server_credentials(((private_key, certificate_chain),), root_certificates,False)
# '[::]:' means may be accessed from any IP.
server.add_secure_port('[::]:' + port, server_credentials)
server.start()
print("Server started, listening on " + port)
# May be interrupted by Ctrl+C.
server.wait_for_termination()
if __name__ == '__main__':
print("Loading Melbourne models......")
t_started = time.time()
exec(open("/home/jean/scripts/Melbourne_Regress.py").read())
train = PredictTrain()
valid = PredictValid()
perf = performance()
t_finished = time.time()
print("Melbourne models loaded, "+str(round(t_finished-t_started,1))+" seconds.")
logging.basicConfig()
serve()
3、Python客户端程序greeter_client_SSL_Auth.py。
Python客户端程序是测试用途,以验证服务器程序工作正常。先构造一个接口原型中定义的MelbourneRequest对象,然后调用GetOutliers()函数,返回的stream在Python客户端中是一个Iterator,用for语句遍历,具体可参阅gRPC 官方文档中文版。
print("Outlier client received: \n")
for outlier in stub.GetOutliers(helloworld_pb2.MelbourneRequest(algo='cat', threshold=65)):
print(outlier)
完整的程序。
# Copyright 2015 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Python implementation of the GRPC helloworld.Greeter client."""
from __future__ import print_function
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
# Class to add authentication header to the meta data of every gRPC call.
class GrpcAuth(grpc.AuthMetadataPlugin):
def __init__(self, key):
self._key = key
# 'rpc-auth-header' is the authentication header defined by the server.
def __call__(self, context, callback):
callback((('rpc-auth-header', self._key),), None)
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
print("Will try to greet world ...")
# Will fail for enpty credentials.
# creds = grpc.ssl_channel_credentials()
# Need to include root certificate in credentials.
# https://stackoverflow.com/questions/72230151/how-to-open-a-secure-channel-in-python-grpc-client-without-a-client-ssl-certific
# Copy the selfsigned CA's root cert to the client directory, so that the client doesn't need to be run as root.
with open('ca.crt', 'rb') as f:
creds = grpc.ssl_channel_credentials(f.read())
# A composite channel credentials with SSL and password.
# Host name need to be the same as the server certificate.
channel = grpc.secure_channel(
'jeanye.cn:50051',
grpc.composite_channel_credentials(
creds,
grpc.metadata_call_credentials(
GrpcAuth('access_key')
# A worng key will fail then.
# GrpcAuth('wrong_access_key')
)
)
)
stub = helloworld_pb2_grpc.GreeterStub(channel)
response = stub.SayHello(helloworld_pb2.HelloRequest(name='Jean'))
print("Greeter client received: " + response.message)
print("Outlier client received: \n")
for outlier in stub.GetOutliers(helloworld_pb2.MelbourneRequest(algo='cat', threshold=65)):
print(outlier)
if __name__ == '__main__':
logging.basicConfig()
run()
运行Python客户端程序。
三、Java客户端
演示性质,暂时还是在Java客户端项目中生成存根stub类,然后拷贝到Tomcat Web项目中,用Java客户端项目先测试也要方便一点。
1、proto接口原型文件helloworld.proto。
客户端项目同上篇文章配置,拷贝上面的接口原型文件到Java客户端项目中更新原来的文件。项目名grpcclient 上右键-> Run As -> Maven Build,重新生成Java的stub类。
2、HelloWorldClientSSLAuthMelbourne.java。
Java客户端操作gRPC stream的例子可以参阅grpc-java自带的RouteGuide例子及其proto文件。gRPC stream在Java客户端也是一个java.util.Iterator类,可以通过for()循环去迭代,用hasNext()判断是否结束,用next()去获取下一个返回的MelbourneReply对象,存储为一个java.util.List以便后续处理。注意blockingStub.getOutliers(),与接口原型定义中相比,java方法的首字母按Java函数的惯例都改成了小写。
package io.grpc.examples.helloworld;
import io.grpc.Channel;
import io.grpc.ClientInterceptor;
import io.grpc.ClientInterceptors;
import io.grpc.ManagedChannel;
import io.grpc.StatusRuntimeException;
import io.grpc.examples.header.HeaderClientInterceptor;
import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts;
import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.io.File;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* A simple client that requests a greeting from the {@link HelloWorldServer}.
*/
public class HelloWorldClientSSLAuthMelbourne {
private static final Logger logger = Logger.getLogger(HelloWorldClient.class.getName());
private final GreeterGrpc.GreeterBlockingStub blockingStub;
/** Construct client for accessing HelloWorld server using the existing channel. */
public HelloWorldClientSSLAuthMelbourne(Channel channel) {
// 'channel' here is a Channel, not a ManagedChannel, so it is not this code's responsibility to
// shut it down.
// Passing Channels to code makes code easier to test and makes it easier to reuse Channels.
blockingStub = GreeterGrpc.newBlockingStub(channel);
}
/** Get outliers of Melbourne house price. */
public List<MelbourneReply> getOutliers(String algo, double threshold) {
logger.info("Will try to get outliers for algo " + algo +" with threshold "+ threshold + " ...");
MelbourneRequest request = MelbourneRequest.newBuilder().setAlgo(algo).setThreshold(threshold).build();
Iterator<MelbourneReply> response;
List<MelbourneReply> result = new ArrayList<>();
try {
response = blockingStub.getOutliers(request);
for (int i = 1; response.hasNext(); i++) {
MelbourneReply outlier = response.next();
result.add(outlier);
System.out.println("Result #" + i + ": "+outlier.getRow()+" | "+ Math.round(outlier.getOrigin()*100.0)/100.0+" | "+Math.round(outlier.getPredict()*100.0)/100.0+" | "+outlier.getSe());
}
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return null;
}
logger.info("Get outliers of Melbourne house price.");
return result;
}
/**
* Greet server. If provided, the first element of {@code args} is the name to use in the
* greeting. The second argument is the target server.
*/
public static void main(String[] args) throws Exception {
/*
* Use NettyChannelBuilder to build SSL connection to server.
* https://stackoverflow.com/questions/42700411/ssl-connection-for-grpc-java-client
* Need to include selfsigned CA certificate to build a sslContext to verify the server.
* And the server name must be same as the name in the server certificate.
*/
ManagedChannel originChannel = NettyChannelBuilder.forAddress("jeanye.cn", 50051)
.sslContext(GrpcSslContexts.forClient().trustManager(new File("d:/temp/ca.crt")).build())
.build();
ClientInterceptor interceptor = new HeaderClientInterceptor();
Channel channel = ClientInterceptors.intercept(originChannel, interceptor);
try {
HelloWorldClientSSLAuthMelbourne client = new HelloWorldClientSSLAuthMelbourne(channel);
List<MelbourneReply> res = client.getOutliers("cat",60.0);
} finally {
// ManagedChannels use resources like threads and TCP connections. To prevent leaking these
// resources the channel should be shut down when it will no longer be used. If it may be used
// again leave it running.
originChannel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
}
}
}
运行结果:
四、Tomcat Web App
1、拷贝上面Java客户端项目生成的stub类更新。
GreeterGrpc.java HelloReply.java
HelloReplyOrBuilder.java HelloRequest.java
HelloRequestOrBuilder.java HelloWorldProto.java
MelbourneReply.java MelbourneReplyOrBuilder.java
MelbourneRequest.java MelbourneRequestOrBuilder.java
2、HelloWorldClientSSLAuthMelbourne.java。
参考上面的Java客户端编写,实现gRPC调用的帮助类。
package io.grpc.examples.helloworld;
import io.grpc.Channel;
import io.grpc.ClientInterceptor;
import io.grpc.ClientInterceptors;
import io.grpc.ManagedChannel;
import io.grpc.StatusRuntimeException;
import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts;
import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder;
import java.io.File;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* A simple client that requests a greeting from the {@link HelloWorldServer}.
*/
public class HelloWorldClientSSLAuthMelbourne {
private static final Logger logger = Logger.getLogger(HelloWorldClient.class.getName());
private final GreeterGrpc.GreeterBlockingStub blockingStub;
private ManagedChannel originChannel = null;
private Channel channel = null;
/** Construct client for accessing HelloWorld server by creating a channel. */
public HelloWorldClientSSLAuthMelbourne(String caFile, String server, Integer port) throws Exception {
/*
* Use NettyChannelBuilder to build SSL connection to server.
* https://stackoverflow.com/questions/42700411/ssl-connection-for-grpc-java-
* client Need to include selfsigned CA certificate to build a sslContext to
* verify the server. And the server name must be same as the name in the server
* certificate.
*/
originChannel = NettyChannelBuilder.forAddress(server, port)
.sslContext(GrpcSslContexts.forClient().trustManager(new File(caFile)).build()).build();
// "access_key" is the password to call gRPC.
ClientInterceptor interceptor = new HeaderClientInterceptor("access_key");
// Channel with an interceptor to add an auth header for each gRPC call.
channel = ClientInterceptors.intercept(originChannel, interceptor);
// 'channel' here is a Channel, not a ManagedChannel, so it is not this code's
// responsibility to shut it down.
// Passing Channels to code makes code easier to test and makes it easier to
// reuse Channels.
blockingStub = GreeterGrpc.newBlockingStub(channel);
}
/** Say hello to server. */
public String greet(String name) {
logger.info("Will try to greet " + name + " ...");
HelloRequest request = HelloRequest.newBuilder().setName(name).build();
HelloReply response;
try {
response = blockingStub.sayHello(request);
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return (e.toString());
}
logger.info("Greeting: " + response.getMessage());
return (response.getMessage());
}
/** Get ooutliers of Melbourne house price. */
public List<MelbourneReply> getOutliers(String algo, double threshold) {
logger.info("Will try to get outliers for algo " + algo + " with threshold " + threshold + " ...");
MelbourneRequest request = MelbourneRequest.newBuilder().setAlgo(algo).setThreshold(threshold).build();
Iterator<MelbourneReply> response;
List<MelbourneReply> result = new ArrayList<>();
try {
response = blockingStub.getOutliers(request);
for (int i = 1; response.hasNext(); i++) {
MelbourneReply outlier = response.next();
result.add(outlier);
System.out.println("Result #" + i + ": " + outlier.getRow() + " | " + outlier.getOrigin() + " | "
+ outlier.getPredict() + " | " + outlier.getSe());
}
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return null;
}
logger.info("Get outliers of Melbourne house price.");
return result;
}
/** Close the channel. */
public void close() throws Exception {
try {
// ManagedChannels use resources like threads and TCP connections. To prevent
// leaking these
// resources the channel should be shut down when it will no longer be used. If
// it may be used again leave it running.
originChannel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
} catch (Exception e) {
e.printStackTrace();
}
}
}
3、algo.jsp。
输入页面,选择算法及输入异常值阀值。
<%@ page language="java" contentType="text/html; charset=GBK"
pageEncoding="GBK"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=GBK">
<title>Java EE gRPC调用测试</title>
</head>
<body>
<center>
<form action="melbourne.jsp" method="post">
墨尔本房价gRPC调用示例 <br/>
算法:<select name="algo" id="algo">
<option value="svm">SVM</option>
<option value="rf">RandomForest</option>
<option value="gbr">GBR</option>
<option value="xgb">XGB</option>
<option value="lgbm">LigthGBM</option>
<option value="cat" selected>CatBoost</option>
<option value="blend">Blend</option>
</select>
<br/>
异常值阀值:<input type="number" name="threshold" value=60><br/>
<input type="submit" value="查看异常值">
</form>
</center>
</body>
</html>
4、melbourne.jsp。
显示gRPC调用结果及匹配的原始数据,所以项目中打包了原始数据文件(预处理)Melbourne_housing_pre.csv。
<%@ page language="java" contentType="text/html; charset=GBK"
pageEncoding="GBK"%>
<%@ page import="io.grpc.examples.helloworld.*,java.util.*,java.io.BufferedReader,java.io.FileReader"%>
<%
List<MelbourneReply> resp =null;
Set<Long> set = new HashSet<Long>();
try{
String algo = request.getParameter("algo");
Double threshold = new Double(request.getParameter("threshold"));
if (algo==null) algo = "cat";
//String server = "localhost";
String server = "jeanye.cn";
Integer port = 50051;
// Plain text without SSL & password.
// HelloWorldClient client = new HelloWorldClient(server, port);
String caFile = application.getRealPath("/WEB-INF/ca.crt");
// SSL without password.
// HelloWorldClientSSL client = new HelloWorldClientSSL(caFile,server,port);
// SSL with password.
//HelloWorldClientSSLAuth client = new HelloWorldClientSSLAuth(caFile,server,port);
HelloWorldClientSSLAuthMelbourne client = new HelloWorldClientSSLAuthMelbourne(caFile,server,port);
resp = client.getOutliers(algo, threshold);
client.close();
}catch(Exception e){
e.printStackTrace();
}
%>
<!DOCTYPE html>
<html>
<head>
<meta charset="GBK">
<title>Java EE 调用gRPC 测试</title>
</head>
<body>
<h3>Java gRPC调用服务器端Python程序结果 <a href="algo.jsp">返回</a></h3>
<%
if (resp !=null){
%>
<table border="1" cellspacing="0" bordercolor="#D3D3D3">
<tr><td>行号</td><td>真实值</td><td>预测值</td><td>偏差%</td>
</tr>
<%
for(Iterator it=resp.iterator();it.hasNext();){
MelbourneReply outlier = (MelbourneReply)it.next();
set.add(outlier.getRow());
%>
<tr>
<td><%= outlier.getRow()%></td>
<td><%= Math.round(outlier.getOrigin()*100.0)/100.0%></td>
<td><%= Math.round(outlier.getPredict()*100.0)/100.0%></td>
<td><%= outlier.getSe()%></td>
</tr>
<%
}
%>
</table>
<%
}
%>
<br>
<h3>匹配的原始数据</h3>
<table border="1" cellspacing="0" bordercolor="#D3D3D3">
<tr>
<%
String path = request.getRealPath("/WEB-INF");
System.out.println(path);
BufferedReader reader = new BufferedReader(new FileReader(path+"/Melbourne_housing_pre.csv"));
// skip header
String header[] =reader.readLine().split(",");
header[0] = "Row";
for (int i=0; i<header.length;i++){
%>
<td><%=header[i]%></td>
<%}%></tr>
<%
String line = null;
long index=0;
//读取每行,直到为空
while((line=reader.readLine())!=null){
if (set.contains(index)){
String items[] = line.split(",");
%><tr><%
for (int i=0; i< items.length; i++){
%><td><%=items[i] %></td>
<%
}%></tr><%
}
index++;
}
%>
</table>
<br>
<a href="algo.jsp">返回</a>
</body>
</html>
5、本机测试。
6、部署到服务器上运行,见本篇开头的运行效果。
五、用Maven管理Tomcat项目
参考资料。用Maven管理Tomcat项目的好处,一是可以在项目中根据proto接口原型定义生成Java stub类,不用从Java客户端项目中拷贝;二是开发时不用逐个导入依赖的包,Maven会自动解决依赖关系,下载导入相应的包;三是打包成war发布到服务器时,Maven会自动把依赖包放进去。这样就方便多了。程序与上面的Tomcat Web App是一样的,只是项目的组织结构有所不同。
1、建立Maven管理的Web APP项目。
Eclipse->File->New Maven Project,后面要选择archetype,不要选simple project。
Catalog选All Catalogs,Filter中输入maven-archetype-webapp,项目类型选Group Id: org.apache.maven.archetypes,Artifact Id: maven-archetype-webapp,注意它的版本是1.4。如果Catalog选了Internal,也可以列出来,但它会是内置的1.0版,二者生成的pom.xml是不一样的。All Catalog要联网到各个catalog去下载archetype列表(可以按configure按钮配置),取决于网络的状况,有时候可能会出不来,有时候可能要等几十秒。
为自己的项目定义Group Id与Artifact Id,因为已经建好了grpc项目,用grpcTest演示,所以前后截图的项目名可能会有所不同。
项目名上右键->Properties->Java Build Path->Add Library,Server Runtime把Tomcat Server Runtime加入,否则JSP编译报错。JRE System Library把适当的JDK加入,由于我的笔记本上JDK11的版本比服务器上JDK11的版本新,导致服务器上部署时报java.lang.unsupportedclassversionerror,我在笔记本上降为JDK8。此时Maven Dependencies只有基本的junit等,后面再加。
此时可以在项目名上右键->Run As->Run on Server,选择前面实验中配好的Tomcat服务器,把新建的项目发布到Tomcat上,在Eclipse中浏览首页,验证项目正确创建,它会自动生成web.xml。
2、更新pom.xml。
1)、将java版本设为与build path一样的1.8。
2)、把前面gRPC Java客户端项目中pom.xml的dependency拷贝过来。
3)、因为不了解<pluginManagement>的用法,新建了一个<plugins>,把Java客户端项目中pom.xml的generate-sources一节的plugin拷贝过来,用来生成gRPC的Java stub类源码。参阅资料。
4)、<build>里的<defaultGoal>也可以拷贝过来,generate-sources要排在clean后,其它任务的前面。
然后在项目名上右键->Maven->Update Project,更新Maven项目的Java Build Path引用。如果是本地第一次引用这些包,Maven联网下载可能需要一点时间。
完整的源码如下:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>jean.test</groupId>
<artifactId>grpc</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>war</packaging>
<name>grpc Maven Webapp</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.21.7</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
<version>1.51.0</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>1.51.0</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>1.51.0</version>
</dependency>
<!-- Removed from JDK11 and above, need to be added here -->
<!-- https://blog.csdn.net/ml863606/article/details/109202246?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_baidulandingword~default-0-109202246-blog-120904477.pc_relevant_3mothn_strategy_recovery&spm=1001.2101.3001.4242.1&utm_relevant_index=3 -->
<dependency>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
<build>
<defaultGoal>clean generate-sources compile install</defaultGoal>
<finalName>grpc</finalName>
<pluginManagement><!-- lock down plugins versions to avoid using Maven
defaults (may be moved to parent pom) -->
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- see http://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_war_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<!-- compile proto file into java files -->
<plugin>
<groupId>com.github.os72</groupId>
<artifactId>protoc-jar-maven-plugin</artifactId>
<version>3.11.1</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<includeMavenTypes>direct</includeMavenTypes>
<inputDirectories>
<include>src/main/resources</include>
</inputDirectories>
<outputTargets>
<outputTarget>
<type>java</type>
<outputDirectory>src/main/java</outputDirectory>
</outputTarget>
<outputTarget>
<type>grpc-java</type>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.51.0</pluginArtifact>
<outputDirectory>src/main/java</outputDirectory>
</outputTarget>
</outputTargets>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
3、生成gRPC的Java stub类。
把前面Java客户端项目中的接口原型定义文件helloworld.proto拷贝过来,在项目名字上右键->Run As->Maven generate-sources,则会执行pom.xml中定义的generate-sources任务生成gRPC stub类。注意如果手工删除了生成的stub类,又没有更新proto原型文件,这个操作会直接跳过,什么都不做。更新一下proto原型文件,让它有个新的时间戳,重新运行即可。Maven有这个右键菜单是因为在pom.xml中定义了它。
Eclipse中有两个可以编辑proto文件的插件,提供了语法高亮显示等功能,我安装了protobuf-dt,具体安装方法请参阅其项目主页上的说明,它有一个小问题,就是不认识proto文件中的option语句,原因是其打包的descriptor.proto文件比较旧,参考该帖子,通过WinRAR用一个比较新的替换(D:/eclipse/plugins/com.google.eclipse.protobuf_2.3.2.201609161849.jar内的)即可,我用了Anaconda3安装好的一个。
另外,Eclipse->Window->Preferences->Protocol Buffer->Compiler中,默认不勾选"Compile .proto files on save"选项,不要改它,因为我们用Run As->Maven gererate-sources来生成,勾选了每次更改保存都调Compiler,然后会报错,应该是还没有配好:
Errors occurred during the build.
Errors running builder 'Xtext Project Builder' on project 'grpc'.
'void com.google.common.io.Closeables.closeQuietly(java.io.Closeable)'
4、把前面Tomcat Web App项目中的程序文件拷贝到Maven Tomcat Web App项目中。包括前面图中的HelloWorldClient*.java,HeaderClient*.java,*.jsp,以及WEB-INF下的资源文件ca.crt、Melbourne_housing_pre.csv,详见前面图中的项目结构。
然后在algo.jsp上右键->Run As->Run on Server,发布到前面配好的Tomcat服务器,运行测试,这与平常是一样的。
5、打包发布到Linux服务器上。
项目名字上右键->Run As->Maven Build,因为pom.xml中定义了<defaultGoal>,直接按Run按钮就可以。
生成的war文件在项目的target目录下,可以在Eclipse的Console窗口看build过程的输出。
六、配置Python服务端开机自启动
# cd /etc/rc.d
# vi rc.local
# Startup Python gRPC server, added by Jean 2022/12/16.
cd /home/jean/grpcTest
/usr/lib64/anaconda3/bin/python greeter_server_SSL_Auth.py
# reboot now
Done.