上传kafka安装包
kafka_2.10-0.10.0.0.tgz
安装zookeeper
kafka需要使用到zookeeper，虽然kafka自带了一个zookeeper。不过我都是使用单独的zookeeper。
(如何安装zookeeper详细zookeeper安装部署)
单机部署
修改配置文件
config/server.properties
主要是listeners和zookeeper.connect
listeners=PLAINTEXT://172.16.0.154:9092
log.dirs=/opt/kafka_2.11-0.9.0.1/logs
zookeeper.connect=172.16.0.154:2181,172.16.0.154:2182,172.16.0.154:2183
启动服务
nohup ./kafka-server-start.sh ../config/server.properties &
nohup 后台启动方式，日志记录在nohup.out文件中
kafka常用命令
- 查看topic list
./kafka-topics.sh --zookeeper 172.18.21.197:2181 --list - 修改topic分区
./kafka-topics.sh --zookeeper 172.18.21.197:2181 -alter --partitions 5 --topic test1 - 查询topic当前消费情况
./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group gc --topic gc --zookeeper 10.10.0.83:2181
[2017-06-21 10:01:35,543] WARN WARNING: ConsumerOffsetChecker is deprecated and will be dropped in releases following 0.9.0. Use ConsumerGroupCommand instead. (kafka.tools.ConsumerOffsetChecker$)
Group Topic Pid Offset logSize Lag Owner
gc gc 0 3336 3336 0 none
gc gc 1 3099 3099 0 none
gc gc 2 3222 3222 0 none
gc gc 3 3211 3211 0 none
gc gc 4 3221 3221 0 none - 查看topic信息
./kafka-topics.sh --zookeeper 172.18.21.197:2181 --topic test1 --describe
