kafka生产者启动报错及kafka常用命令

运行环境 | centos 6.6 | kafka_2.11-1.1.1

kafka启动生产者后报错
WARN [Producer clientId=console-producer] Got error produce response with correlation id 3 on topic-partition test-2, retrying (2 attempts left). Error: NETWORK_EXCEPTION (org.apache.kafka.clients.producer.internals.Sender)

运行生产者命令后报错

1
2
3
./kafka-console-producer.sh --broker-list 192.168.83.128:9092 --topic test

>[2019-04-29 10:37:39,906] WARN [Producer clientId=console-producer] Got error produce response with correlation id 3 on topic-partition test-2, retrying (2 attempts left). Error: NETWORK_EXCEPTION (org.apache.kafka.clients.producer.internals.Sender)

运行消费者报错

1
2
3
./kafka-console-consumer.sh -zookeeper 192.168.83.129:2181 --from-beginning --topic test

Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].

查看日志

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[2019-05-02 16:10:25,182] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentBrokerIdException: Configured broker.id 2 doesn't match stored broker.id 0 in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).
at kafka.server.KafkaServer.getBrokerIdAndOfflineDirs(KafkaServer.scala:673)
at kafka.server.KafkaServer.startup(KafkaServer.scala:209)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:75)
at kafka.Kafka.main(Kafka.scala)
[2019-05-02 16:10:25,185] INFO shutting down (kafka.server.KafkaServer)
[2019-05-02 16:10:25,209] WARN (kafka.utils.CoreUtils$)
java.lang.NullPointerException
at kafka.server.KafkaServer$$anonfun$shutdown$5.apply$mcV$sp(KafkaServer.scala:572)
at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:85)
at kafka.server.KafkaServer.shutdown(KafkaServer.scala:572)
at kafka.server.KafkaServer.startup(KafkaServer.scala:329)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:75)
at kafka.Kafka.main(Kafka.scala)
[2019-05-02 16:10:25,211] INFO [ZooKeeperClient] Closing. (kafka.zookeeper.ZooKeeperClient)
[2019-05-02 16:10:25,244] INFO Session: 0x26a778905c40001 closed (org.apache.zookeeper.ZooKeeper)
[2019-05-02 16:10:25,246] INFO [ZooKeeperClient] Closed. (kafka.zookeeper.ZooKeeperClient)
[2019-05-02 16:10:25,250] INFO shut down completed (kafka.server.KafkaServer)
[2019-05-02 16:10:25,250] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)

原因: 所有配置 broker.id=0

修改 kafka-logs-data/meta.properties
修改后重启正常生产和消费

常用命令

生产者操作:
./kafka-console-producer.sh --broker-list master:9092 --topic test

消费者操作:
./kafka-console-consumer.sh --bootstrap-server master:9092 --topic test --from-beginning

创建topic
./kafka-topics.sh --create --zookeeper master:2181 --replication-factor 2 --partitions 3 --topic test

查看topic列表
./kafka-topics.sh --list --zookeeper master:2181

如果需要查看topic的详细信息,需要使用describe命令
./kafka-topics.sh --describe --zookeeper node1:2181 --topic test-topic

若不指定topic,则查看所有topic的信息
./kafka-topics.sh --describe --zookeeper node1:2181

删除topic
./kafka-topics.sh --delete --zookeeper master:2181 --topic test

本文结束感谢您的阅读,本文原创–支持原创
有问题请联系我--strivedeer@163.com