kafka安装部署

上传kafka安装包

kafka_2.10-0.10.0.0.tgz

安装zookeeper

kafka需要使用到zookeeper，虽然kafka自带了一个zookeeper。不过我都是使用单独的zookeeper。
(如何安装zookeeper详细zookeeper安装部署)

单机部署

修改配置文件

config/server.properties
主要是listeners和zookeeper.connect

listeners=PLAINTEXT://172.16.0.154:9092
log.dirs=/opt/kafka_2.11-0.9.0.1/logs
zookeeper.connect=172.16.0.154:2181,172.16.0.154:2182,172.16.0.154:2183

启动服务

nohup ./kafka-server-start.sh ../config/server.properties &

nohup 后台启动方式，日志记录在nohup.out文件中

kafka常用命令

1. 查看topic list
   ./kafka-topics.sh --zookeeper 172.18.21.197:2181 --list
2. 修改topic分区
   ./kafka-topics.sh --zookeeper 172.18.21.197:2181 -alter --partitions 5 --topic test1
3. 查询topic当前消费情况
   ./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group gc --topic gc --zookeeper 10.10.0.83:2181
   [2017-06-21 10:01:35,543] WARN WARNING: ConsumerOffsetChecker is deprecated and will be dropped in releases following 0.9.0. Use ConsumerGroupCommand instead. (kafka.tools.ConsumerOffsetChecker$)
   Group Topic Pid Offset logSize Lag Owner
   gc gc 0 3336 3336 0 none
   gc gc 1 3099 3099 0 none
   gc gc 2 3222 3222 0 none
   gc gc 3 3211 3211 0 none
   gc gc 4 3221 3221 0 none
4. 查看topic信息
   ./kafka-topics.sh --zookeeper 172.18.21.197:2181 --topic test1 --describe