淘先锋技术网

首页 1 2 3 4 5 6 7

脚本一键启动zk，kafka集群

一 zk的一键启动脚本

注意： 停止 Kafka 集群时，一定要等 Kafka 所有节点进程全部停止后再停止 Zookeeper
集群。因为 Zookeeper 集群当中记录着 Kafka 集群相关信息， Zookeeper 集群一旦先停止，
Kafka 集群就没有办法再获取停止进程的信息，只能手动杀死 Kafka 进程了。

1.1 脚本内容

#!/bin/bash

case $1 in
"start"){
	for i in 192.168.152.136  192.168.152.138 192.168.152.140
	do
		echo  ------------- zookeeper $i 启动 ------------
		ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
art"
	done
}
;;
"stop"){
	for i in 192.168.152.136  192.168.152.138 192.168.152.140
	do
		echo  ------------- zookeeper $i 停止 ------------
		ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
op"
	done
}
;;
"status"){
	for i in 192.168.152.136  192.168.152.138 192.168.152.140
	do
		echo  ------------- zookeeper $i 状态 ------------
		ssh $i "/root/export/servers/zookeeper-3.4.10/bin/zkServer.sh st
atus"
	done
}
;;
esac

1.2 赋予执行权限

[root@localhost servers]# ls -l
total 8
drwxr-xr-x.  7 root root  118 Feb 13 22:25 kafka_2.12-2.1.0
-rw-r--r--.  1 root root  659 Mar  1 22:18 zk.sh
drwxr-xr-x. 12 root root 4096 Feb  6 17:43 zookeeper-3.4.10
[root@localhost servers]# chmod +x zk.sh 
[root@localhost servers]# ls -l
total 8
drwxr-xr-x.  7 root root  118 Feb 13 22:25 kafka_2.12-2.1.0
-rwxr-xr-x.  1 root root  659 Mar  1 22:18 zk.sh
drwxr-xr-x. 12 root root 4096 Feb  6 17:43 zookeeper-3.4.10

 1.3 启动脚本

[root@localhost servers]# zk.sh start
 

1.4 启动效果

[root@localhost servers]# sh zk.sh start
------------- zookeeper 192.168.152.136 启动 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
------------- zookeeper 192.168.152.138 启动 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
------------- zookeeper 192.168.152.140 启动 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost servers]# jps
3347 QuorumPeerMain
3381 Jps

在其他节点138查看：

[root@localhost bin]# jps
3347 QuorumPeerMain
3671 Jps

在其他节点140查看：

 [root@localhost bin]# jps
3347 QuorumPeerMain
3671 Jps

注意：因为没有配置节点之间ssh的免登陆，所以在启动一个个节点时候，还需要手动输入密码。

1.5 关闭zk

1.在140上执行关闭脚本

[root@localhost servers]# ./zk.sh stop
------------- zookeeper 192.168.152.136 停止 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
------------- zookeeper 192.168.152.138 停止 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
------------- zookeeper 192.168.152.140 停止 ------------
[email protected]'s password: 
ZooKeeper JMX enabled by default
Using config: /root/export/servers/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@localhost servers]# jps
5847 Jps
[root@localhost servers]# 
2.在138上查看zk进程

[root@localhost servers]# jps
5847 Jps
[root@localhost servers]# 

3.在140上查看zk进程

[root@localhost servers]# jps
5847 Jps
[root@localhost servers]# 

二 kafka的一键启动脚本

2.1 脚本内容

创建kf.sh脚本： touch  kf.sh

#!/bin/bash
case $1 in
"start"){
 for i in 192.168.152.136  192.168.152.138  192.168.152.140
 do
 echo " --------启动 $i Kafka-------"
 ssh $i "nohup /root/export/servers/kafka_2.12-2.1.0/bin/kafka-server-start.sh  
 /root/export/servers/kafka_2.12-2.1.0/config/server.properties 2>&1 &"
 done
};;
"stop"){
 for i in 192.168.152.136  192.168.152.138  192.168.152.140
 do
 echo " --------停止 $i Kafka-------"
 ssh $i "/root/export/servers/kafka_2.12-2.1.0/bin/kafka-server-stop.sh "
 done
};;
esac

2.2 赋予执行权限

[root@localhost servers]# ls -l
total 12
drwxr-xr-x.  7 root root  118 Feb 13 22:25 kafka_2.12-2.1.0
-rw-r--r--.  1 root root  461 Mar  1 22:38 kf.sh
-rwxr-xr-x.  1 root root  659 Mar  1 22:29 zk.sh
drwxr-xr-x. 12 root root 4096 Feb  6 17:43 zookeeper-3.4.10
[root@localhost servers]# chmod +x kf.sh 
[root@localhost servers]# ls -l
total 12
drwxr-xr-x.  7 root root  118 Feb 13 22:25 kafka_2.12-2.1.0
-rwxr-xr-x.  1 root root  461 Mar  1 22:38 kf.sh
-rwxr-xr-x.  1 root root  659 Mar  1 22:29 zk.sh
drwxr-xr-x. 12 root root 4096 Feb  6 17:43 zookeeper-3.4.10


2.3 启动脚本

[root@localhost servers]# sh kf.sh start
 --------启动 192.168.152.136 Kafka-------
[email protected]'s password: 
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
[2022-03-01 22:39:00,424] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2022-03-01 22:39:00,540] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.nio.file.NoSuchFileException: config/server.properties
。。。。。。。

注意：因为没有配置节点之间ssh的免登陆，所以在启动一个个节点时候，还需要手动输入密码。

2.4 查看启动效果

1.在136结果：

[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps
2.在138上查看

[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps
3.在140上查看

[root@localhost servers]# jps
3969 Kafka
3506 QuorumPeerMain
4389 Jps

2.5 查看主题

1.136上查看

[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh  --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]# 
2.在138上查看

[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh  --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]# 
3.在140上查看

[root@localhost kafka_2.12-2.1.0]# bin/kafka-topics.sh  --list --zookeeper 192.168.152.138
__consumer_offsets
kafka_test
[root@localhost kafka_2.12-2.1.0]# 

2.6 关闭kafka

1.在140上执行脚本，关闭kafka

[root@localhost servers]# ./kf.sh stop
 --------停止 192.168.152.136 Kafka-------
[email protected]'s password: 
 --------停止 192.168.152.138 Kafka-------
[email protected]'s password: 
 --------停止 192.168.152.140 Kafka-------
[email protected]'s password: 

查看进程号：

[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]# 

注意执行脚本后，需要稍等一下，再查看

2.在136上关闭kafka

[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]# 

3.在140上关闭kafka

[root@localhost servers]# jps
5762 Jps
3347 QuorumPeerMain
[root@localhost servers]#