kafka + zookeeper 伪集群搭建

扫码关注公众号:Java 技术驿站

发送:vip
将链接复制到本浏览器,永久解锁本站全部文章

【公众号:Java 技术驿站】 【加作者微信交流技术,拉技术群】
免费领取10G资料包与项目实战视频资料

一 zookeeper伪集群搭建

1.创建3个zoo.cfg(zoo1.cfg,zoo2.cfg,zoo3.cfg)

zoo1.cfg 设置:

# 数据文件夹
dataDir=/usr/local/zookeeper/data/data1
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs1

clientPort=2181

server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889

zoo2.cfg 设置:

# 数据文件夹
dataDir=/usr/local/zookeeper/data/data2
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs2

clientPort=2182

server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889

zoo3.cfg 设置:

# 数据文件夹
dataDir=/usr/local/zookeeper/data/data3
# 日志文件夹
dataLogDir=/usr/local/zookeeper/logs/logs2

clientPort=2183

server.1=192.168.94.132:2887:3887
server.2=192.168.94.132:2888:3888
server.3=192.168.94.132:2889:3889

2.创建3个myid

在usr/local/zookeeper/data/datax下闯进myid文件作为标识符,对应server.x数字   3.启动伪集群服务    

bin/zkServer.sh start conf/zoo1.cfg
bin/zkServer.sh start conf/zoo2.cfg
bin/zkServer.sh start conf/zoo3.cfg

4.查看伪集群服务每个zookeeper状态

[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo1.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo1.cfg
Mode: follower
[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo2.cfg
Mode: leader
[root@bogon zookeeper-3.4.11]# bin/zkServer.sh status conf/zoo3.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo3.cfg
Mode: follower

二 kafka伪集群搭建

1. 将配置文件拷贝多份

[root@master zookeeper-3.4.11]$cp config/server.properties config/server-1.properties
[root@master zookeeper-3.4.11]$cp config/server.properties config/server-2.properties
[root@master zookeeper-3.4.11]$cp config/server.properties config/server-3.properties

2.修改每个server-x.properties

server-1.properties 设置:

broker.id=1
port=9092
host.name=192.168.94.132

log.dirs=/usr/local/kafka/kafka-logs-1

message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880

zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true

server-2.properties 设置:

broker.id=2
port=9093
host.name=192.168.94.132

log.dirs=/usr/local/kafka/kafka-logs-2

message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880

zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true

server-3.properties 设置:

broker.id=3
port=9093
host.name=192.168.94.132

log.dirs=/usr/local/kafka/kafka-logs-3

message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880

zookeeper.connect=192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183
# 可删除topic
delete.topic.enable=true

每个kafka的broker.id不一样,作为唯一标识符

三 启动集群测试

注:启动时:先启动 zookeeper,后启动 kafka;关闭时:先关闭 kafka,后关闭zookeeper 

1. 分别在每个节点上启动zookeeper

[root@master zookeeper-3.4.11]$ bin/zkServer.sh start conf/zoo1.cfg

[root@worker1 zookeeper-3.4.11]$ bin/zkServer.sh start conf/zoo2.cfg

[root@worker2 zookeeper-3.4.11]$ bin/zkServer.sh start conf/zoo3.cfg

2.验证zookeeper集群

# 在master节点上 
[root@master zookeeper-3.4.11]$ bin/zkServer.sh status conf/zoo1.cfg
# 在worker1节点上
[root@worker1 zookeeper-3.4.11]$ bin/zkServer.sh status conf/zoo2.cfg
# 在worker2节点上 
[root@worker2 zookeeper-3.4.11]$ bin/zkServer.sh status conf/zoo3.cfg

显示结果为:有一个是 leader,剩下的都是 follower 

3.启动kafka集群

后台启动方式:

[root@master kafka_2.12-1.1.0]$ bin/kafka-server-start.sh -daemon config/server-1.properties

[root@master kafka_2.12-1.1.0]$ bin/kafka-server-start.sh  config/server-1.properties

[root@worker1 kafka_2.12-1.1.0]$ bin/kafka-server-start.sh  config/server-1.properties

[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-start.sh  config/server-1.properties

4.测试kafka集群 

4.1 创建 topic 和 显示 topic 信息

# 创建topic
bin/kafka-topics.sh --create --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --replication-factor 3 --partitions 3 --topic test

# 显示topic信息
bin/kafka-topics.sh --describe --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test

# 列出topic
bin/kafka-topics.sh --list --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183

4.2 创建 producer

bin/kafka-console-producer.sh --broker-list master:9092 -topic test

4.3 创建 consumer

# 测试消费 也可分别做测试
bin/kafka-console-consumer.sh --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 -topic test --from-beginning

然后在 producer 里输入消息,consumer 中就会显示出同样的内容,表示消费成功

4.4 删除节点

bin/kafka-topics.sh --delete --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test

四 关闭集群服务

# 删除topic
bin/kafka-topics.sh --delete --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test

# 关闭kafka 
[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-1.properties

[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-2.properties

[root@worker2 kafka_2.12-1.1.0]$ bin/kafka-server-stop.sh conf/server-3.properties

# 关闭zookeeper 
[root@master zookeeper-3.4.11]$ bin/zkServer.sh stop conf/zoo1.cfg

[root@worker1 zookeeper-3.4.11]$ bin/zkServer.shstop conf/zoo2.cfg

[root@worker2 zookeeper-3.4.11]$ bin/zkServer.shstop conf/zoo3.cfg
赞(1) 打赏
版权归原创作者所有,任何形式的转载请联系博主:daming_90:Java 技术驿站 » kafka + zookeeper 伪集群搭建

  • 暂无文章

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏