葫芦的运维日志_teg

葫芦的运维日志

浏览量 4791

2019/11/14 03:01

部署kafka依赖：

1. java环境

2. zookeeper

1.部署java

1.1 获取部署包

jdk-8u152-linux-x64.tar.gz

1.2 解压

tar –zxvf jdk-8u152-linux-x64.tar.gz –C /usr/local/

1.3 设置软连接

ln –s jdk1.8.0_152 jdk

1.4 配置环境变量

##Vim /etc/profile

export JAVA_HOME=/usr/local/jdk

export PATH=$PATH:$JAVA_HOME/bin

source /etc/profile

1.5测试

Java –version

2.部署zookeeper(单机集群版)

2.1 获取部署包

zookeeper-3.4.6.tar.gz

2.2 解压

tar –zxvf zookeeper-3.4.6.tar.gz –C /usr/local/

mkdir -p /data/zk

mkdir -p /data/zk1

mkdir -p /data/zk2

mkdir –p /data/log/zookeeper

mkdir –p /data/log/zookeeper1

mkdir –p /data/log/zookeeper2

2.4 拷贝文件

cp -r /usr/local/zookeeper-3.4.6/* /data/zk/

cp -r /usr/local/zookeeper-3.4.6/* /data/zk1/

cp -r /usr/local/zookeeper-3.4.6/* /data/zk2/

2.5 修改配置文件

#vim /data/zk/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/data/zk

dataLogDir=/data/log/zookeeper

maxClientCnxns=3000

clientPort=2181

autopurge.snapRetainCount=20

autopurge.purgeInterval=12

server.1=10.1.1.1:2888:3888

server.2=10.1.1.1:2889:3889

server.3=10.1.1.1:2890:3890

#vim /data/zk1/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/data/zk1

dataLogDir=/data/log/zookeeper1

maxClientCnxns=3000

clientPort=2182

autopurge.snapRetainCount=20

autopurge.purgeInterval=12

server.1=10.1.1.1:2888:3888

server.2=10.1.1.1:2889:3889

server.3=10.1.1.1:2890:3890

#vim /data/zk2/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/data/zk2

dataLogDir=/data/log/zookeeper2

maxClientCnxns=3000

clientPort=2183

autopurge.snapRetainCount=20

autopurge.purgeInterval=12

server.1=10.1.1.1:2888:3888

server.2=10.1.1.1:2889:3889

server.3=10.1.1.1:2890:3890

2.6添加myid文件

echo “1” > /data/zk/myid

echo “2” > /data/zk1/myid

echo “3” > /data/zk2/myid

2.7启动zookeeper

/data/zk/bin/zk.Server.sh start

/data/zk1/bin/zk.Server.sh start

/data/zk2/bin/zk.Server.sh start

2.8查看zookeeper状态

/data/zk/bin/zk.Server.sh status

/data/zk1/bin/zk.Server.sh status

/data/zk2/bin/zk.Server.sh status

检查到一个leader 两个follower即为正常

3.部署kafka

3.1获取部署包

kafka_2.11-0.9.0.0.tgz

3.2解压

tar –zxvf kafka_2.11-0.9.0.0.tgz –C /usr/local/

3.3 配置软连接及创建日志目录

ln –s kafka_2.11-0.9.0.0 kafka

mkdir -p /data/log/kafka

mkdir -p /data/log/kafka1

mkdir -p /data/log/kafka2

3.4配置多broker

cp /usr/local/kafka/config/server.properties /usr/local/kafka/config/server1.properties

cp /usr/local/kafka/config/server.properties /usr/local/kafka/config/server2.properties

3.5修改配置文件

vim /usr/local/kafka/config/server.properties

auto.create.topics.enable=true

broker.id=1

listeners=PLAINTEXT://10.1.1.1:9092

port=9092

host.name=10.1.1.1

num.network.threads=20

num.io.threads=4

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=/data/log/kafka

num.partitions=10

num.recovery.threads.per.data.dir=1

log.retention.hours=168

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

log.cleaner.enable=false

zookeeper.connect=10.1.1.1:2181,10.1.1.1:2182,10.1.1.1:2183

zookeeper.connection.timeout.ms=6000

vim /usr/local/kafka/config/server1.properties

auto.create.topics.enable=true

broker.id=2

listeners=PLAINTEXT://10.1.1.1:9093

port=9093

host.name=10.1.1.1

num.network.threads=20

num.io.threads=4

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=/data/log/kafka1

num.partitions=10

num.recovery.threads.per.data.dir=1

log.retention.hours=168

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

log.cleaner.enable=false

zookeeper.connect=10.1.1.1:2181,10.1.1.1:2182,10.1.1.1:2183

zookeeper.connection.timeout.ms=6000

vim /usr/local/kafka/config/server2.properties

auto.create.topics.enable=true

broker.id=3

listeners=PLAINTEXT://10.1.1.1:9094

port=9094

host.name=10.1.1.1

num.network.threads=20

num.io.threads=4

socket.send.buffer.bytes=102400

socket.receive.buffer.bytes=102400

socket.request.max.bytes=104857600

log.dirs=/data/log/kafka2

num.partitions=10

num.recovery.threads.per.data.dir=1

log.retention.hours=168

log.segment.bytes=1073741824

log.retention.check.interval.ms=300000

log.cleaner.enable=false

zookeeper.connect=10.1.1.1:2181,10.1.1.1:2182,10.1.1.1:2183

zookeeper.connection.timeout.ms=6000

3.6启动kafka

/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/ server.properties

/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/ server1.properties

/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/ server2.properties

3.7检查

ps –ef | grep kafka

查到3个进程即可

葫芦的运维日志

打赏