摘要：
kafka集群安装部署到使用 二、Kafka集群安装配置1.安装：[root@ZooKeeper-Kafka-01src]#tar-zxvfkafka_2.12-2.2.0.tgz-C/usr/local/[root@ZooKeeper-Kafka-01src]#cd..[root@ZooKeeper-Kafka-01local]#ln-skafka_2.12-2.2.0kafka2.修
由于条件有限整了三台虚拟机进行部署
主机规划：
10.250.23.213 ZooKeeper-Kafka-01
10.250.23.214 ZooKeeper-Kafka-02
10.250.23.215 ZooKeeper-Kafka-03
一段小插曲：虚拟机网络不通：
1、设置网络为桥接模式
2、修改配置文件内容/etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet BOOTPROTO=staticNAME=ens33 DEVICE=ens33 ONBOOT=yes GATEWAY=10.250.23.254IPADDR=10.250.23.213DNS1=8.8.8.8NETMASK=255.255.255.0
3、关掉防火墙
4、更新yum源
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repoyum makecache yum -y update
言归正传，软件下载地址：
wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.5.4-beta/zookeeper-3.5.4-beta.tar.gz wget http://mirror.bit.edu.cn/apache/kafka/2.2.0/kafka_2.12-2.2.0.tgz
三台主机hosts文件一致：
vim /etc/hosts
10.250.23.213 ZooKeeper-Kafka-01 10.250.23.214 ZooKeeper-Kafka-02 10.250.23.215 ZooKeeper-Kafka-03
一、安装zookeeper
1.在master节点上操作：
[root@ZooKeeper-Kafka-01 src]# tar -zxvf zookeeper-3.5.4-beta.tar.gz -C /usr/local/[root@ZooKeeper-Kafka-01src]# cd .. [root@ZooKeeper-Kafka-01 local]# ln -s zookeeper-3.5.4-betazookeeper [root@ZooKeeper-Kafka-01 local]# cd zookeeper/conf/[root@ZooKeeper-Kafka-01conf]# cp zoo_sample.cfg zoo.cfg tickTime=2000initLimit=10syncLimit=5dataDir=/tmp/zookeeper clientPort=2181server.1=10.250.23.213-01:2888:3888server.2=10.250.23.214-02:2888:3888server.3=10.250.23.215-03:2888:3888
2.创建dataDir目录创建/tmp/zookeeper
# 在master节点上
[root@ZooKeeper-Kafka-01 conf]# mkdir /tmp/zookeeper [root@ZooKeeper-Kafka-01 conf]# touch /tmp/zookeeper/myid [root@ZooKeeper-Kafka-01 conf]# echo 1 > /tmp/zookeeper/myid
3.将zookeeper文件复制到另外两个节点：
[root@ZooKeeper-Kafka-01 local]# scp -r zookeeper-3.5.4-beta/ 10.250.23.214:/usr/local/[root@ZooKeeper-Kafka-01 local]# scp -r zookeeper-3.5.4-beta/ 10.250.23.215:/usr/local/
4.在两个slave节点创建目录和文件
#ZooKeeper-Kafka-02节点:
[root@ZooKeeper-Kafka-02 local]# ln -s zookeeper-3.5.4-beta zookeeper [root@ZooKeeper-Kafka-02 local]# mkdir /tmp/zookeeper [root@ZooKeeper-Kafka-02 local]# touch /tmp/zookeeper/myid [root@ZooKeeper-Kafka-02 local]# echo 2 > /tmp/zookeeper/myid
#ZooKeeper-Kafka-03节点
[root@ZooKeeper-Kafka-03 local]# ln -s zookeeper-3.5.4-beta zookeeper [root@ZooKeeper-Kafka-03 local]# mkdir /tmp/zookeeper [root@ZooKeeper-Kafka-03 local]# touch /tmp/zookeeper/myid [root@ZooKeeper-Kafka-03 local]# echo 3 > /tmp/zookeeper/myid
5.分别在每个节点上启动 zookeeper测试：
[root@ZooKeeper-Kafka-01 zookeeper]# ./bin/zkServer.sh start [root@ZooKeeper-Kafka-02 zookeeper]# ./bin/zkServer.sh start [root@ZooKeeper-Kafka-03 zookeeper]# ./bin/zkServer.sh start
6.查看状态：
[root@ZooKeeper-Kafka-01 zookeeper]# ./bin/zkServer.sh status ZooKeeper JMX enabled by defaultUsing config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: follower [root@ZooKeeper-Kafka-02 zookeeper]# ./bin/zkServer.sh status ZooKeeper JMX enabled by defaultUsing config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: leader [root@ZooKeeper-Kafka-03 zookeeper]# ./bin/zkServer.sh status ZooKeeper JMX enabled by defaultUsing config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: follower
至此zookeeper集群安装成功！！！
二、Kafka集群安装配置
1.安装：
[root@ZooKeeper-Kafka-01 src]# tar -zxvf kafka_2.12-2.2.0.tgz -C /usr/local/[root@ZooKeeper-Kafka-01src]# cd .. [root@ZooKeeper-Kafka-01 local]# ln -s kafka_2.12-2.2.0 kafka
2.修改server.properties文件
# master为0 broker.id=0listeners=PLAINTEXT://10.250.23.213:9092 advertised.listeners=PLAINTEXT://10.250.23.213:9092 num.network.threads=3num.io.threads=8socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=/tmp/kafka-logs num.partitions=5num.recovery.threads.per.data.dir=1offsets.topic.replication.factor=1transaction.state.log.replication.factor=1transaction.state.log.min.isr=1log.retention.hours=24log.segment.bytes=1073741824log.retention.check.interval.ms=300000# 连接 zookeeper.connect=10.250.23.213:2181,10.250.23.214:2181,10.250.23.215:2181zookeeper.connection.timeout.ms=6000group.initial.rebalance.delay.ms=0# 可删除topic delete.topic.enable=true
3.将 kafka_2.12-1.1.0 文件夹复制到另外两个节点下
[root@ZooKeeper-Kafka-01 local]# scp -r kafka_2.12-2.2.0/ 10.250.23.214:/usr/local/[root@ZooKeeper-Kafka-01 local]# scp -r kafka_2.12-2.2.0/ 10.250.23.215:/usr/local/
并修改每个节点对应的 server.properties 文件的 broker.id和listenrs：
[root@ZooKeeper-Kafka-02config]# cat server.properties broker.id=1listeners=PLAINTEXT://10.250.23.214:9092 advertised.listeners=PLAINTEXT://10.250.23.214:9092 [root@ZooKeeper-Kafka-03config]# cat server.properties broker.id=2listeners=PLAINTEXT://10.250.23.215:9092 advertised.listeners=PLAINTEXT://10.250.23.215:9092
4.启动三台服务器（三台的操作一样）
./bin/kafka-server-start.sh ../config/server.properties &
Zookeeper+Kafka集群测试
创建topic：
[root@ZooKeeper-Kafka-01 kafka]# ./bin/kafka-topics.sh --create --zookeeper 10.250.23.213:2181, 10.250.23.214:2181, 10.250.23.215:2181
--replication-factor 3 --partitions 3 --topic test
显示topic：
[root@ZooKeeper-Kafka-01 kafka]# ./bin/kafka-topics.sh --describe --zookeeper 10.250.23.213:2181, 10.250.23.214:2181, 10.250.23.215:2181 --topic test
列出topic：
[root@ZooKeeper-Kafka-01 kafka]# ./bin/kafka-topics.sh --list --zookeeper 10.250.23.213:2181, 10.250.23.214:2181, 10.250.23.215:2181 test
创建 producer(生产者);
# 在master节点上 测试生产消息
[root@ZooKeeper-Kafka-01 kafka]# ./bin/kafka-console-producer.sh --broker-list 10.250.23.213:9092 -topic test
>hello world
创建 consumer(消费者):
# 在ZooKeeper-Kafka-02节点上 测试消费
[root@ZooKeeper-Kafka-02 kafka]# ./bin/kafka-console-consumer.sh --bootstrap-server 10.250.23.213:9092, 10.250.23.214:9092, 10.250.23.215:9092
-topic test --from-beginning
#在ZooKeeper-Kafka-03节点上 测试消费
[root@ZooKeeper-Kafka-03 kafka]# ./bin/kafka-console-consumer.sh --bootstrap-server 10.250.23.213:9092, 10.250.23.214:9092, 10.250.23.215:9092 -topic test --from-beginning
然后在 producer 里输入消息，consumer 中就会显示出同样的内容，表示消费成功
删除 topic 和关闭服务
[root@ZooKeeper-Kafka-01 kafka]# bin/kafka-topics.sh --delete --zookeeper 10.250.23.213:2181, 10.250.23.214:2181, 10.250.23.215:2181 --topic test至此Zookeeper+Kafka集群配置成功.