zookeeper删除kafka元数据

问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : Topic "hive-test-table" already exists.
[2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
    at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
    at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
    at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)
 (kafka.admin.TopicCommand$)

原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据

解决:

zookeeper shell命令

[zk: d65:2181/kafka_hive(CONNECTED) 7] ll
ZooKeeper -server host:port cmd args
    connect host:port
    get path [watch]
    ls path [watch]
    set path data [version]
    rmr path
    delquota [-n|-b] path
    quit 
    printwatches on|off
    create [-s] [-e] path data acl
    stat path [watch]
    close 
    ls2 path [watch]
    history 
    listquota path
    setAcl path acl
    getAcl path
    sync path
    redo cmdno
    addauth scheme auth
    delete path [version]
    setquota -n|-b val path

进入zookeeper目录,删除kafka topic 元数据

[root@d100 bin]# sh zkCli.sh -server  d65:2181/kafka_hive

[zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
[consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]

[zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
[seqid, topics, ids]

[zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
[略...]

[zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table

[zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
[略...]

重新创建topic成功

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
Created topic "hive-test-table".

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc  --topic hive-test-table
Topic:hive-test-table   PartitionCount:5    ReplicationFactor:2 Configs:
    Topic: hive-test-table  Partition: 0    Leader: 1285    Replicas: 1285,1281 Isr: 1285,1281
    Topic: hive-test-table  Partition: 1    Leader: 1281    Replicas: 1281,1282 Isr: 1281,1282
    Topic: hive-test-table  Partition: 2    Leader: 1282    Replicas: 1282,1283 Isr: 1282,1283
    Topic: hive-test-table  Partition: 3    Leader: 1283    Replicas: 1283,1284 Isr: 1283,1284
    Topic: hive-test-table  Partition: 4    Leader: 1284    Replicas: 1284,1285 Isr: 1284,1285
最后编辑于 :
©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

相关阅读更多精彩内容

  • kafka的定义:是一个分布式消息系统,由LinkedIn使用Scala编写,用作LinkedIn的活动流(Act...
    时待吾阅读 5,574评论 1赞 15
  • Kafka入门经典教程-Kafka-about云开发 http://www.aboutyun.com/threa...
    葡萄喃喃呓语阅读 11,013评论 4赞 54
  • Spring Cloud为开发人员提供了快速构建分布式系统中一些常见模式的工具(例如配置管理,服务发现,断路器,智...
    卡卡罗2017阅读 137,188评论 19赞 139
  • 本文转载自http://dataunion.org/?p=9307 背景介绍Kafka简介Kafka是一种分布式的...
    Bottle丶Fish阅读 5,623评论 0赞 34
  • 一、入门1、简介Kafka is a distributed,partitioned,replicated com...
    HxLiang阅读 3,730评论 0赞 9

友情链接更多精彩内容