报错如下
failed: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records
排查
1、有消费者宕机下线。消费者并不一定需要真正下线,例如遇到长时间的GC、网络延迟导致消费者长时间未向 GroupCoordinator 发送心跳等情况时, GroupCoordinator 会认为消费者已经下线。
修改参数
props.put("max.poll.interval.ms",50**60*1000);
props.put("max.poll.records",500);
//设置消费者心跳间隔
props.put("heartbeat.interval.ms",3000);
//session超时时间
props.put("session.timeout.ms", 10000);
2、kafkaConsumer.assign() 点对点消费方式和 subscribe()订阅消费方式 ,使用了相同的消费组,也就是他们 group id 相同时, group coordinator无法识别具有相同消费组group id的consumer,直接抛异常CommitFailedException
如果是这种情况,提示的解决方法max.poll.interval.ms,max.poll.record都没用,无法解决,只能修改消费组id