kafka 0.8,0.10兼容性测试

一. 不同版本的读兼容性结果

服务端版本 是否开snappy压缩 客户端0.8.2.2 客户端0.10.2.2
0.8.2.2 true 成功 失败
0.8.2.2 false 成功 失败
0.10.2.2 true 成功 成功
0.10.2.2 false 成功 成功

二. 0.8server 命令

2.1 0.8 启动

bin/kafka-server-start.sh -daemon config/server.properties

2.2 0.8 发送消息，不开压缩

bin/kafka-topics.sh --zookeeper localhost:2181/ --create --topic t1 --partitions 3 --replication-factor 1
 bin/kafka-console-producer.sh  --broker-list localhost:9092 --topic t1 --compression-codec none

2.2.1 0.10 消费不开压缩的消息 失败

afly@afly-desktop:~/local/kafka_2.11-0.10.2.2$ bin/kafka-console-consumer.sh --zookeeper localhost:2181/ --topic t1
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
[2019-07-06 22:43:47,725] WARN [ConsumerFetcherThread-console-consumer-66613_afly-desktop-1562424227400-8033055c-0-0], Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@5185dcf4 (kafka.consumer.ConsumerFetcherThread)
java.io.EOFException
    at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:83)
    at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:129)
    at kafka.network.BlockingChannel.receive(BlockingChannel.scala:120)
    at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:99)
    at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:131)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130)
    at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:111)
    at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:31)
    at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
    at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

 afly@afly-desktop:~/local/kafka_2.11-0.10.2.2$ bin/kafka-console-consumer.sh --zookeeper localhost:2181/ --topic t1
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
[2019-07-06 22:43:47,725] WARN [ConsumerFetcherThread-console-consumer-66613_afly-desktop-1562424227400-8033055c-0-0], Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@5185dcf4 (kafka.consumer.ConsumerFetcherThread)
java.io.EOFException
    at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:83)
    at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:129)
    at kafka.network.BlockingChannel.receive(BlockingChannel.scala:120)
    at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:99)
    at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:131)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
    at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130)
    at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:111)
    at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:31)
    at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
    at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

2.2.2 0.8 消费不开压缩的消息 Okay

 bin/kafka-console-consumer.sh  --zookeeper localhost:2181/ --topic t1

2.3 0.8 发送消息，开压缩

bin/kafka-topics.sh --zookeeper localhost:2181/ --create --topic t2 --partitions 3 --replication-factor  1
 bin/kafka-console-producer.sh  --broker-list localhost:9092 --topic t1 --compression-codec snappy

2.3.1 0.10消费开压缩的消息 无反应，消费失败

bin/kafka-console-consumer.sh  --bootstrap-server localhost:9092 --topic t1 

==> logs/server.log <==
[2019-07-06 22:46:08,957] ERROR Closing socket for /127.0.0.1 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
    at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:64)
    at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
    at kafka.network.Processor.read(SocketServer.scala:450)
    at kafka.network.Processor.run(SocketServer.scala:340)
    at java.lang.Thread.run(Thread.java:748)

2.3.2 0.8消费开压缩的消息 Okay

bin/kafka-console-consumer.sh  --zookeeper localhost:2181/ --topic t1

三. 0.10 启动

bin/kafka-server-start.sh -daemon config/server.properties

0.10 发送消息，不开压缩

bin/kafka-topics.sh --zookeeper localhost:2181/kafka010 --create --topic t1 --partitions 3 --replication-factor 1

bin/kafka-console-producer.sh  --broker-list localhost:9092 --compression-codec none --topic t1

0.10 消费不开压缩的消息 Okay

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic t1

0.8 消费不开压缩的消息 Okay

bin/kafka-console-consumer.sh  --zookeeper localhost:2181/kafka010 --topic t1

0.10 发送消息，开压缩

bin/kafka-topics.sh --zookeeper localhost:2181/kafka010 --create --topic t2 --partitions 3 --replication-factor 1

bin/kafka-console-producer.sh  --broker-list localhost:9092 --compression-codec snappy --topic t2

0.10消费开压缩的消息 Okay

bin/kafka-console-consumer.sh  --bootstrap-server localhost:9092 --topic t2 --from-beginning

0.8消费开压缩的消息 Okay

bin/kafka-console-consumer.sh  --zookeeper localhost:2181/kafka010 --topic t2 --from-beginning
最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
平台声明：文章内容（如有图片或视频亦包括在内）由作者上传并发布，文章内容仅代表作者本人观点，简书系信息发布平台，仅提供信息存储服务。