基于docker镜像搭建confluent运行环境
FROM confluentinc/cp-kafka-connect:5.2.1
ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:latest
基于上面的代码,制作yml文件,运行docker-compose up -d -build下载镜像、构建并启动相关容器
由于已经生成容器了,可运行如下的命令批量重启或停止:
批量重启
docker ps -a | grep confluentinc | awk -F' ' '{ print $1}' | xargs docker start
批量停止
docker ps -a | grep confluentinc | awk -F' ' '{ print $1}' | xargs docker stop
各组件端口默认映射关系
ZK
默认端口:2181,
启动命令:service zkcluster start (默认开机自启动)
Kafka
默认端口:9092,
配置文件:server.properties
启动命令:kafka-server-start /etc/kafka/schema-registryserver.properties
connect
默认端口:8083,
配置文件:connect-file-sink.properties、connect-file-source.properties、connect-standalone.properties
启动命令:./bin/connect-standalone.sh config/connect-standalone.properties \config/connect-file-source.properties config/connect-file-sink.properties
schema-registry
默认端口:8081,
配置文件:connect-avro-standalone.properties、schema-registry.properties
启动命令:schema-registry-start /etc/schema-registry/schema-registry.properties
kafka-rest
默认端口:8082,
配置文件:kafka-rest.properties
启动命令:kafka-rest-start /etc/kafka-rest/kafka-rest.properties
confluent-control-center
默认端口:9021,
配置文件:control-center.properties
启动命令:control-center-start /etc/confluent-control-center/control-center.properties
ksql
默认端口:8088,
启动命令:ksql-server-start ksql-server.properties
最终通过http://XXXXXX:9021 访问对应的kafka监控界面及ksql使用平台