Kafka暂停消费--consumer.pause()

遇到了需要暂停消费的场景，使用pause()方法暂停消费，resume()方法恢复消费，基于springboot的demo如下:

注意，如果需要暂停消费的话，需要consumer 订阅 topic 的方式必须是 Assign。

assign 和 subscribe 的区别：assign方法由用户直接手动consumer实例消费哪些具体分区，assign的consumer不会拥有kafka的group management机制，也就是当group内消费者数量变化的时候不会有reblance行为发生。assign的方法不能和subscribe方法同时使用。

pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.yqs</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
            <version>1.5.21.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <version>1.5.21.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>1.5.21.RELEASE</version>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>1.5.21.RELEASE</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
            <version>1.5.21.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot</artifactId>
            <version>1.5.21.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-log4j2</artifactId>
            <version>1.5.21.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>1.3.9.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.73</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/log4j/log4j -->
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.14</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.4</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

kafka消费者 :

通过switchOn变量来手动的控制暂停跟恢复

@Component
public class KafkaPauseTest implements ApplicationListener<ContextRefreshedEvent> {
    static Logger log = Logger.getLogger(KafkaPauseTest.class);
    @Autowired
    private Consumer consumer;
    TopicPartition  partition0 = new TopicPartition("test_topic", 0);

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        new Thread(()->{
            try {
                consumer.assign(Arrays.asList(new TopicPartition[]{partition0}));
                while (true) {
                    log.info("获取开关变量="+ SwitchController.switchOn);
                    if (SwitchController.switchOn == 1){
                        log.info("暂停消费开始");
                        consumer.pause(Arrays.asList(new TopicPartition[]{partition0}));
                        log.info("暂停消费结束");
                    }else {
                        log.info("恢复消费开始");
                        consumer.resume(Collections.singletonList(partition0));
                        log.info("恢复消费结束");
                    }
                    ConsumerRecords<String, String> records = consumer.poll(5000);
                    log.info("poll结果结束");
                    for (ConsumerRecord<String, String> record : records) {
                        log.info("topic = " + record.topic() + ", partition = " + record.partition());
                        log.info("offset = " + record.offset());
                        log.info("value = " + record.value());
                        String msg = record.value();
                    }
                    consumer.commitSync();
                }
            } finally {
                consumer.close();
            }
        }).start();
    }
}

控制类：

@RestController
public class SwitchController {

    public static volatile int switchOn = 0;

    @RequestMapping(value = "/pause", method = RequestMethod.GET)
    public String pause() {
        switchOn = 1;
        return "暂停监听";
    }

    @RequestMapping(value = "/reuse", method = RequestMethod.GET)
    public String reuse() {
        switchOn = 0;
        return "恢复监听";
    }
}