Centos6.5下kafka如何部署安装-创新互联
小编给大家分享一下Centos6.5下kafka如何部署安装,希望大家阅读完这篇文章之后都有所收获,下面让我们一起去探讨吧!
成都创新互联公司坚持“要么做到,要么别承诺”的工作理念,服务领域包括:成都做网站、网站建设、外贸营销网站建设、企业官网、英文网站、手机端网站、网站推广等服务,满足客户于互联网时代的南浔网站设计、移动媒体设计的需求,帮助企业找到有效的互联网解决方案。努力成为您成熟可靠的网络建设合作伙伴!一)安装JDK
步骤1:新建路径/usr/Java, 并在其下解压 jdk-7u71-linux-x64.tar.gz
# mkdir -p /usr/java
# tar -zxvf jdk-7u71-linux-x64.tar.gz
步骤2:添加JDK到系统环境变量
# vi /etc/profile
新增以下内容:
export JAVA_HOME=/usr/java/jdk1.7.0_71
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
步骤3:使配置生效
source /etc/profile #使配置文件立即生效
# java -version //JDK版本java version "1.7.0_79" 也可以,目前JDK1.8版本不支持;
Java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)
二)安装zookeeper
步骤1:解压zookeeper安装包(路径无关,安装路径可以自定义)
# tar -zxvf zookeeper-3.3.6.tar.gz
步骤2:配置环境变量
# vim /etc/profile
export zookeeper_home=/home/laoyang/zookeeper
#source /etc/profile
# sh $zookeeper_home/bin/zkServer.sh start
JMX enabled by default
Using config: /home/laoyang/zookeeper/bin/../conf/zoo.cfg
grep: /home/laoyang/zookeeper/bin/../conf/zoo.cfg: No such file or directory
Starting zookeeper ... STARTED
三)安装kafka
步骤1:解压&移动到/usr/local
# tar -xzvf kafka_2.11-0.9.0.1.tgz
# mv kafka_2.11-0.9.0.1 /usr/local/kafka
步骤2:配置kafka,修改server.properties
# mkdir /usr/local/kafka/log/kafka #创建kafka日志目录
# cd /usr/local/kafka/config #进入配置目录
# vim server.properties #编辑修改相应的参数
broker.id=0
port=9092 #端口号
host.name=10.8.5.101 #服务器IP地址,修改为自己的服务器IP
log.dirs=/usr/local/kafka/log/kafka #日志存放路径,上面创建的目录
zookeeper.connect=localhost:2181 #zookeeper地址和端口,单机配置部署,localhost:2181
步骤3、配置kafka下的zookeeper
# mkdir /usr/local/kafka/zookeeper #创建zookeeper目录
# mkdir /usr/local/kafka/log/zookeeper #创建zookeeper日志目录
# cd /usr/local/kafka/config #进入配置目录
# vim zookeeper.properties #编辑修改相应的参数
dataDir=/usr/local/kafka/zookeeper #zookeeper数据目录
dataLogDir=/usr/local/kafka/log/zookeeper #zookeeper日志目录
clientPort=2181
maxClientCnxns=100
tickTime=2000
initLimit=10
syncLimit=5
步骤4:创建启动、关闭kafka脚本
1)创建启动kafka脚本
#cd /usr/local/kafka/bin/
# vim kafkastart.sh
#!/bin/bash
#启动zookeeper
/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties &
sleep 3
#Start kafka
/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &
2)创建关闭kafka脚本
# vim kafkastop.sh
#!/bin/bash
#stop zookeeper
/usr/local/kafka/bin/zookeeper-server-stop.sh /usr/local/kafka/config/zookeeper.properties &
sleep 3
#Stop kafka
/usr/local/kafka/bin/kafka-server-stop.sh /usr/local/kafka/config/server.properties &
3)#添加脚本执行权限
chmod +x kafkastart.sh
chmod +x kafkastop.sh
4)设置脚本开机自动执行
# vim /etc/rc.d/rc.local #编辑,在最后添加一行
sh /usr/local/kafka/kafkastart.sh & #设置开机自动在后台运行脚本
sh /usr/local/kafka/kafkastart.sh #启动kafka
sh /usr/local/kafka/kafkastop.sh #关闭kafka
到此为止CentOs6.5下Kafka安装配置完成。
成功标志:
四)测试kafka
步骤1:启动zookeeper
# sh $zookeeper_home/bin/zkServer.sh start
JMX enabled by default
Using config: /home/laoyang/zookeeper/bin/../conf/zoo.cfg
grep: /home/laoyang/zookeeper/bin/../conf/zoo.cfg: No such file or directory
Starting zookeeper ... STARTED
步骤2:启动kafka
# ./kafkastart.sh
步骤3:测试创建topic
# cd /usr/local/kafka/bin
# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
步骤4:通过list命令查看创建的topic
# cd /usr/local/kafka/bin
# ./kafka-topics.sh –list –zookeeper localhost:2181
步骤5:生产消息测试
# ./kafka-console-producer.sh --broker-list localhost:9092 --topic test
Kafka test! //这里的“Kafka test!”为手动输入的,输入后“ctrl +c”退出;
步骤6:消费消息测试
# ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
I'm laoyang #之前测试输入的内容
kafka test!
经过上面几步,代表kafka安装成功。
五)停止kafka
步骤1:停止Kafka
cd /usr/local/kafka
# ./kafkastop.sh //这里的kafka进程并没有关掉,需要手动kill
步骤2:停止Zookeeper server
# sh $zookeeper_home/bin/zkServer.sh stop
至此,启动,测试,停止全过程ok。
kafka启动过程中的报错:
WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
EndOfStreamException: Unable to read additional data from client sessionid 0x157ac94cd940002, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
报错原因:
zookeeper服务端和客户端的jar版本不一致导致的 ;
解决办法:
从zookeeper服务端安装目录的根目录复制zookeeper-xxx.jar到kafka安装目录下的libs目录下即可,将kafka下libs目录下的jar包删除;
报错:
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
at kafka.utils.ZkUtils$.registerBrokerInZk(ZkUtils.scala:205)
at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:57)
at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:44)
at kafka.server.KafkaServer.startup(KafkaServer.scala:103)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
报错原因:
该错误是由于server.properties 中的broker.id 重复
解决办法:
# vim server.properties
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
看完了这篇文章,相信你对“Centos6.5下kafka如何部署安装”有了一定的了解,如果想了解更多相关知识,欢迎关注创新互联行业资讯频道,感谢各位的阅读!
另外有需要云服务器可以了解下创新互联scvps.cn,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。
名称栏目:Centos6.5下kafka如何部署安装-创新互联
文章位置:http://myzitong.com/article/ddpshd.html