一、环境准备

1、准备3台机器

主机名称 主机IP zookeeper版本 kafka版本

2、3台机器安装jdk1.8环境

3、下载kafka安装包

（此处下载，可忽略第二步：下载安装包）:
kafka_2.12-2.2.1.tgz

4、下载zookeeper安装包

（此处下载，可忽略第二步：下载安装包）：
zookeeper-3.4.14.tar.gz

https://download.csdn.net/download/qq_22075913/87440001

二、下载安装包（官方下载，仅供参考）

1、登录官方：https://downloads.apache.org/

三、环境配置

1、修改主机名（分别单独执行）

# 修改主机名，在worker01上执行
hostnamectl set-hostname worker01
bash # 不重启，使修改主机名生效# 修改主机名，在worker02上执行
hostnamectl set-hostname worker02
bash # 不重启，使修改主机名生效# 修改主机名，在worker03上执行
hostnamectl set-hostname worker03
bash # 不重启，使修改主机名生效

2、关闭防火墙

(worker01、worker02、worker03都执行)

systemctl stop firewalldsystemctl disable firewalldsystemctl status firewalld

3、配置/etc/hosts

(worker01、worker02、worker03都执行)

vi /etc/hosts
#添加worker01、worker02、worker03 ip域名配置
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.179.128 worker01
192.168.179.129 worker02
192.168.179.130 worker03

四、安装 jdk

1、查询仓库jdk软件包

(worker01、worker02、worker03都执行)

# 查询jdk软件包
yum list |grep jdk# 安装系统对应版本
yum install -y java-1.8.0-openjdk-devel.x86_64# 查看jdk是否配置生效
java -version# 后续可以用jps查看，主机启动的所有java进程
jps

五、安装zookeeper集群

1、解压目录

（worker01、worker02、worker03，都要执行）

mkdir -p /home/kafka-zookeeper
cd /home/kafka-zookeeper# 上传安装包# 解压
tar -xf zookeeper-3.4.14.tar.gz[root@worker01 kafka-zookeeper]# ls
zookeeper-3.4.14  zookeeper-3.4.14.tar.gz

2、修改配置文件

（worker01、worker02、worker03，都执行）

# 创建目录
mkdir -p /home/kafka-zookeeper/zookeeper-3.4.14/data
mkdir -p /home/kafka-zookeeper/zookeeper-3.4.14/logs
# 修改配置文件名
cd /home/kafka-zookeeper/zookeeper-3.4.14/conf
mv zoo_sample.cfg zoo.cfg

3、修改配置文件

（worker01、worker02、worker03，都要执行）

vi zoo.cfgtickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/kafka-zookeeper/zookeeper-3.4.14/data
dataLogDir=/home/kafka-zookeeper/zookeeper-3.4.14/logs
clientPort=2181
server.1=192.168.179.128:2888:3888
server.2=192.168.179.129:2888:3888
server.3=192.168.179.130:2888:3888

4、创建myid

 # 在 worker01、worker02、worker03 上都执行
cd /home/kafka-zookeeper/zookeeper-3.4.14/data# 在 worker01、worker02、worker03 上都执行
touch myid# 只在 192.168.179.128 worker01 上执行
echo 1 > /home/kafka-zookeeper/zookeeper-3.4.14/data/myid# 只在192.168.179.129 worker02 上执行
echo 2 > /home/kafka-zookeeper/zookeeper-3.4.14/data/myid# 只在192.168.179.130 worker03 上执行
echo 3 > /home/kafka-zookeeper/zookeeper-3.4.14/data/myid

5、zookeeper启动、停止

# zookeeper 启动
/home/kafka-zookeeper/zookeeper-3.4.14/bin/zkServer.sh start# zookeeper 停止
/home/kafka-zookeeper/zookeeper-3.4.14/bin/zkServer.sh stop# zookeeper 查看状态
/home/kafka-zookeeper/zookeeper-3.4.14/bin/zkServer.sh status# 3台机器的状态
mode：follower    # 从
mode：leader      # 主
mode：follower    # 从# 运行客户端（后面介绍kafka注册）
/home/kafaka-zookeeper/apache-zookeeper-3.6.3/bin/zkCli.sh -server worker0x的IP:2181

3. 查看zookeeper启动状态

3台主机，需要全部执行启动命令，查看zookeeper启动状态，如果正常选主，就启动成功了；反之，失败，需要检查日志 dataLogDir=/home/kafka-zookeeper/zookeeper-3.4.14/logs。

六、安装kafka集群

1、解压目录（worker01、worker02、worker03，都要执行）

# 拷贝安装包
cp kafka_2.12-2.2.1.tgz /home/kafka-zookeeper/# 进入目录
cd /home/kafka-zookeeper/#解压安装包kafka_2.12-2.2.1.tgz
tar -xf kafka_2.12-2.2.1.tgz

2、修改配置

（worker01、worker02、worker03，都要执行）

# 创建目录
mkdir -p /home/kafka-zookeeper/kafka_2.12-2.2.1/data
mkdir -p /home/kafka-zookeeper/kafka_2.12-2.2.1/logs

# 进入kafka_2.12-2.2.1的配置文件目录
cd /home/kafka-zookeeper/kafka_2.12-2.2.1/config# 备份配置文件
cp server.properties server.properties.bak# 删除注释行
sed -i "/#/d" server.properties

vi server.properties# worker01 192.168.179.128 配置文件修改
broker.id=1
listeners=PLAINTEXT://192.168.179.128:9092
log.dirs=/home/kafka-zookeeper/kafka_2.12-2.2.1/logs
num.partitions=3
zookeeper.connect=192.168.179.128:2181,192.168.179.129:2181,192.168.179.130:2181
zookeeper.connection.timeout.ms=18000# 其它2台配置# worker02 192.168.179.129 配置文件修改
broker.id=2
listeners=PLAINTEXT://192.168.179.129:9092
log.dirs=/home/kafka-zookeeper/kafka_2.12-2.2.1/logs
num.partitions=3
zookeeper.connect=192.168.179.128:2181,192.168.179.129:2181,192.168.179.130:2181
zookeeper.connection.timeout.ms=18000# worker03 192.168.179.130 配置文件修改
broker.id=3
listeners=PLAINTEXT://192.168.179.130:9092
log.dirs=/home/kafka-zookeeper/kafka_2.12-2.2.1/logs
num.partitions=3
zookeeper.connect=192.168.179.128:2181,192.168.179.129:2181,192.168.179.130:2181
zookeeper.connection.timeout.ms=18000

3、启动kafka集群

（worker01、worker02、worker03，都要执行）

systemctl stop firewalld
/home/kafka-zookeeper/kafka_2.12-2.2.1/bin/kafka-server-start.sh /home/kafka-zookeeper/kafka_2.12-2.2.1/config/server.properties

4、验证kafka集群

cd /home/kafka-zookeeper/kafka_2.12-2.2.1/bin# 创建 kafka topic
./kafka-topics.sh --create --bootstrap-server 192.168.179.128:9092,192.168.179.129:9092,192.168.179.130:9092 --replication-factor 3 --partitions 3 --topic haha# 列出所有的topic
./kafka-topics.sh --list --bootstrap-server 192.168.179.128:9092

# 在worker01 192.168.179.128 上发布消息
cd /home/kafka-zookeeper/kafka_2.12-2.2.1/bin/
./kafka-console-producer.sh --broker-list 192.168.179.128:9092 --topic haha
>我过的挺好
>zoo#  在worker02 192.168.179.129 上消费
cd /home/kafka-zookeeper/kafka_2.12-2.2.1/bin/
./kafka-console-consumer.sh --bootstrap-server 192.168.179.129:9092 --topic haha --from-beginning
我过的挺好
zoo#  在worker02 192.168.179.130 上消费
cd /home/kafka-zookeeper/kafka_2.12-2.2.1/bin/
./kafka-console-consumer.sh --bootstrap-server 192.168.179.130:9092 --topic haha --from-beginning
我过的挺好
zoo

5、验证zookeeper集群

cd /home/kafka-zookeeper/kafka_2.12-2.2.1/bin/# 进入zookeeper客户端，如果是自定义端口一定要 -server 指定IP:port，否则默认进入2181端口./zkCli.sh -server 192.168.179.128:2181# 查看服务
WatchedEvent state:SyncConnected type:None path:null
[zk: 192.168.179.128:2181(CONNECTED) 0] ls /brokers/ids
[1, 2, 3]
[zk: 192.168.179.128:2181(CONNECTED) 1] ls /brokers/topics
[haha, __consumer_offsets]# 进入zookeeper客户端，不是自定义
WATCHER::WatchedEvent state:SyncConnected type:None path:null
[zk: 192.168.179.128:2181(CONNECTED) 3] ls /
[cluster, controller_epoch, controller, brokers, zookeeper, admin, isr_change_notification, consumers, log_dir_event_notification, latest_producer_id_block, config]
[zk: 192.168.179.128:2181(CONNECTED) 4] ls /config/brokers
[]
[zk: 192.168.179.128:2181(CONNECTED) 5] ls /config
[changes, clients, brokers, topics, users]
[zk: 192.168.179.128:2181(CONNECTED) 6] ls /config/topics
[haha, __consumer_offsets]