本文是通过ffmpeg实现视频流截图后继工作内容探索
根据后继安排,想是将功能做成服务的形式。将这个想法和老大沟通一次,老大说:微服务涉及到服务的编排和后期的运维,目前团队还不具备这样的能力(当初有点心灰意冷,但是言之有理,作为一个项目负责人更多的要考虑团队成员的实际战斗力,在技术上不能一味的冒险)。但是,这不能阻止我探索的脚步,利用闲暇之余我开始后继工作。(由于网络摄像头故障,本篇文章改为文件上传服务)
搭建框架
在做这个微服务之前,需要先搭建个服务框架(个人理解的这个词),该框架应该包含服务注册发现、服务网关、服务熔断等。
IDEA 创建多个模块工程
由于服务框架有多个模块,可以利用IDEA新建一个空的Maven工程,并在该工程内添加模块,并把模块内共同的依赖抽出来放在Parent Module中。其项目及构图如下:
其pom.xml如下:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dhl</groupId>
<artifactId>***</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>discovery-microservice</module>
<module>config-microservice</module>
<module>api-gateway-microservice</module>
</modules>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.7.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spring-cloud.version>Dalston.SR4</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
需要注意的是,Parent Module中pom.xml里packaging为pom,而各子模块中pom.xml里packaging为jar或war。
服务注册发现
运用eureka作为服务注册发现中心(推荐程序员DD博客、)
服务网关
运用Zuul作为服务网关([Kong] (https://getkong.org/)也值得研究)
服务熔断
熔断能够保证某些服务不可用时,其被依赖的服务也将置为不可用,减少无谓的请求等待时间
服务提供者
该处服务提供者是向外提供图片上传服务的RestController,其定义如下:
package com.dhl.controller;
import com.dhl.service.UploadPictureService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
/**
* Created by daihl on 2017/10/18.
*/
@RestController
public class UploadPictureController {
@Autowired
private DiscoveryClient discoveryClient;
@Autowired
private UploadPictureService uploadPictureService;
@RequestMapping(method = RequestMethod.POST, value = "/uploadpicture")
public String uploadPictureCmd(@PathVariable("files") MultipartFile[] files) throws IOException {
String result = "success";
for(int i=0; i<files.length; i++) {
uploadPictureService.upload(files[i].getOriginalFilename(), files[i].getBytes());
}
return result;
}
/**
* 本地服务实例的信息
* @return
*/
@GetMapping("/instance-info")
public ServiceInstance showInfo() {
ServiceInstance localServiceInstance = this.discoveryClient.getLocalServiceInstance();
return localServiceInstance;
}
}
而Service及实现定义如下:
package com.dhl.service;
/**
* UploadPictureService class
*
* @author daihongliang
* @date 2017/10/17
*/
public interface UploadPictureService {
public void upload(String filename, byte[] data);
}
package com.dhl.service;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import java.io.*;
/**
* UploadPictureServiceImpl class
*
* @author daihongliang
* @date 2017/10/17
*/
@Service
public class UploadPictureServiceImpl implements UploadPictureService {
@Value("${image.uploadpath}")
private String imageUploadPath;
@Override
public void upload(String filename, byte[] data){
try {
FileOutputStream fos = new FileOutputStream(imageUploadPath + filename);
fos.write(data);
fos.close();
System.out.println("-------文件上传成功!-------------");
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
最后将该服务注册到注册中心
package com.dhl;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.web.MultipartAutoConfiguration;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.context.annotation.Import;
@SpringBootApplication
@EnableDiscoveryClient
@Import(value = MultipartAutoConfiguration.class)
public class UploadpictureMicroserviceApplication {
public static void main(String[] args) {
SpringApplication.run(UploadpictureMicroserviceApplication.class, args);
}
}
服务消费者
这边的消费者是从前端获取提交的请求(请求中包括图片文件MultipartFile)(注意:MultipartFile只能被使用一次,也就是说如果通过这种服务提供和消费模式,MultipartFile只在消费的时候有用,将请求再发送给服务提供者时将失效。为此我们可以自定义一个编码器FeignSpringFormEncoder,用以封装请求再进行转发)
package com.dhl.uploadpicture.config;
import feign.RequestTemplate;
import feign.codec.EncodeException;
import feign.codec.Encoder;
import org.springframework.core.io.InputStreamResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.multipart.MultipartFile;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.reflect.Type;
import java.nio.charset.Charset;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
/**
* A custom {@link Encoder} that supports Multipart requests. It uses
* {@link HttpMessageConverter}s like {@link RestTemplate} does.
*
* @author Pierantonio Cangianiello
*/
public class FeignSpringFormEncoder implements Encoder {
private final List<HttpMessageConverter<?>> converters = new RestTemplate().getMessageConverters();
private final HttpHeaders multipartHeaders = new HttpHeaders();
private final HttpHeaders jsonHeaders = new HttpHeaders();
public static final Charset UTF_8 = Charset.forName("UTF-8");
public FeignSpringFormEncoder() {
multipartHeaders.setContentType(MediaType.MULTIPART_FORM_DATA);
jsonHeaders.setContentType(MediaType.APPLICATION_JSON);
}
/**
* {@inheritDoc }
*/
@Override
public void encode(Object object, Type bodyType, RequestTemplate template) throws EncodeException {
if (isFormRequest(bodyType)) {
encodeMultipartFormRequest((Map<String, ?>) object, template);
} else {
encodeRequest(object, jsonHeaders, template);
}
}
/**
* Encodes the request as a multipart form. It can detect a single {@link MultipartFile}, an
* array of {@link MultipartFile}s, or POJOs (that are converted to JSON).
*
* @param formMap
* @param template
* @throws EncodeException
*/
private void encodeMultipartFormRequest(Map<String, ?> formMap, RequestTemplate template) throws EncodeException {
if (formMap == null) {
throw new EncodeException("Cannot encode request with null form.");
}
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
for (Entry<String, ?> entry : formMap.entrySet()) {
Object value = entry.getValue();
if (isMultipartFile(value)) {
map.add(entry.getKey(), encodeMultipartFile((MultipartFile) value));
} else if (isMultipartFileArray(value)) {
encodeMultipartFiles(map, entry.getKey(), Arrays.asList((MultipartFile[]) value));
} else {
map.add(entry.getKey(), encodeJsonObject(value));
}
}
encodeRequest(map, multipartHeaders, template);
}
private boolean isMultipartFile(Object object) {
return object instanceof MultipartFile;
}
private boolean isMultipartFileArray(Object o) {
return o != null && o.getClass().isArray() && MultipartFile.class.isAssignableFrom(o.getClass().getComponentType());
}
/**
* Wraps a single {@link MultipartFile} into a {@link HttpEntity} and sets the
* {@code Content-type} header to {@code application/octet-stream}
*
* @param file
* @return
*/
private HttpEntity<?> encodeMultipartFile(MultipartFile file) {
HttpHeaders filePartHeaders = new HttpHeaders();
filePartHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
try {
Resource multipartFileResource = new MultipartFileResource(file.getOriginalFilename(), file.getSize(), file.getInputStream());
return new HttpEntity<>(multipartFileResource, filePartHeaders);
} catch (IOException ex) {
throw new EncodeException("Cannot encode request.", ex);
}
}
/**
* Fills the request map with {@link HttpEntity}s containing the given {@link MultipartFile}s.
* Sets the {@code Content-type} header to {@code application/octet-stream} for each file.
*
* @param the current request map.
* @param name the name of the array field in the multipart form.
* @param files
*/
private void encodeMultipartFiles(LinkedMultiValueMap<String, Object> map, String name, List<? extends MultipartFile> files) {
HttpHeaders filePartHeaders = new HttpHeaders();
filePartHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
try {
for (MultipartFile file : files) {
Resource multipartFileResource = new MultipartFileResource(file.getOriginalFilename(), file.getSize(), file.getInputStream());
map.add(name, new HttpEntity<>(multipartFileResource, filePartHeaders));
}
} catch (IOException ex) {
throw new EncodeException("Cannot encode request.", ex);
}
}
/**
* Wraps an object into a {@link HttpEntity} and sets the {@code Content-type} header to
* {@code application/json}
*
* @param o
* @return
*/
private HttpEntity<?> encodeJsonObject(Object o) {
HttpHeaders jsonPartHeaders = new HttpHeaders();
jsonPartHeaders.setContentType(MediaType.APPLICATION_JSON);
return new HttpEntity<>(o, jsonPartHeaders);
}
/**
* Calls the conversion chain actually used by
* {@link RestTemplate}, filling the body of the request
* template.
*
* @param value
* @param requestHeaders
* @param template
* @throws EncodeException
*/
private void encodeRequest(Object value, HttpHeaders requestHeaders, RequestTemplate template) throws EncodeException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
HttpOutputMessage dummyRequest = new HttpOutputMessageImpl(outputStream, requestHeaders);
try {
Class<?> requestType = value.getClass();
MediaType requestContentType = requestHeaders.getContentType();
for (HttpMessageConverter<?> messageConverter : converters) {
if (messageConverter.canWrite(requestType, requestContentType)) {
((HttpMessageConverter<Object>) messageConverter).write(
value, requestContentType, dummyRequest);
break;
}
}
} catch (IOException ex) {
throw new EncodeException("Cannot encode request.", ex);
}
HttpHeaders headers = dummyRequest.getHeaders();
if (headers != null) {
for (Entry<String, List<String>> entry : headers.entrySet()) {
template.header(entry.getKey(), entry.getValue());
}
}
/*
we should use a template output stream... this will cause issues if files are too big,
since the whole request will be in memory.
*/
template.body(outputStream.toByteArray(), UTF_8);
}
/**
* Minimal implementation of {@link HttpOutputMessage}. It's needed to
* provide the request body output stream to
* {@link HttpMessageConverter}s
*/
private class HttpOutputMessageImpl implements HttpOutputMessage {
private final OutputStream body;
private final HttpHeaders headers;
public HttpOutputMessageImpl(OutputStream body, HttpHeaders headers) {
this.body = body;
this.headers = headers;
}
@Override
public OutputStream getBody() throws IOException {
return body;
}
@Override
public HttpHeaders getHeaders() {
return headers;
}
}
/**
* Heuristic check for multipart requests.
*
* @param type
* @return
* @see feign.Types#MAP_STRING_WILDCARD
*/
static boolean isFormRequest(Type type) {
return MAP_STRING_WILDCARD.equals(type);
}
/**
* Dummy resource class. Wraps file content and its original name.
*/
static class MultipartFileResource extends InputStreamResource {
private final String filename;
private final long size;
public MultipartFileResource(String filename, long size, InputStream inputStream) {
super(inputStream);
this.size = size;
this.filename = filename;
}
@Override
public String getFilename() {
return this.filename;
}
@Override
public InputStream getInputStream() throws IOException, IllegalStateException {
return super.getInputStream(); //To change body of generated methods, choose Tools | Templates.
}
@Override
public long contentLength() throws IOException {
return size;
}
}
}
定义Service接口,并加入MultipartSupportConfig配置,其中配置将请求按照FeignSpringFormEncoder进行编码(注意:利用@FeignClient注解时,最好配合@RequestMapping注解,貌似对@PostMapping、@RequestPart支持不算好)
package com.dhl.uploadpicture.feign;
import com.dhl.uploadpicture.config.FeignSpringFormEncoder;
import com.dhl.uploadpicture.feign.UploadPictureFeignHystrixClient.HystrixClientFallback;
import feign.codec.Encoder;
import org.springframework.beans.factory.ObjectFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.web.HttpMessageConverters;
import org.springframework.cloud.netflix.feign.FeignClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.multipart.MultipartFile;
/**
* Created by daihl on 2017/10/18.
*/
@FeignClient(name = "uploadpicture-microservice", configuration = UploadPictureFeignHystrixClient.MultipartSupportConfig.class, fallback = HystrixClientFallback.class)
public interface UploadPictureFeignHystrixClient {
@RequestMapping(method = RequestMethod.POST, value = "/uploadpicture")
public String uploadPictureCmdFeign(@PathVariable("files") MultipartFile[] files);
@Component
static class HystrixClientFallback implements UploadPictureFeignHystrixClient {
@Override
public String uploadPictureCmdFeign(MultipartFile[] files) {
return "fail";
}
}
@Configuration
class MultipartSupportConfig {
@Autowired
ObjectFactory<HttpMessageConverters> messageConverters;
@Bean
@Primary
@Scope("prototype")
public Encoder multipartFormEncoder() {
return new FeignSpringFormEncoder();
}
}
}
定义Controller
package com.dhl.uploadpicture.controller;
import com.dhl.uploadpicture.feign.UploadPictureFeignHystrixClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
/**
* Created by daihl on 2017/10/18.
*/
@RestController
public class FeignHystrixController {
@Autowired
private UploadPictureFeignHystrixClient uploadPictureFeignHystrixClient;
@RequestMapping(method = RequestMethod.POST, value = "/uploadpicture")
public String uploadPictureCmdFeign(@PathVariable("files") MultipartFile[] files) throws IOException {
return uploadPictureFeignHystrixClient.uploadPictureCmdFeign(files);
}
}
结果
启动服务注册发现服务、启动服务提供者和服务消费者
利用Postman模拟POST请求
在配置的服务器路径下发现上传图片文件