I am trying to use Kafka. All configurations are done properly but when I try to produce message from console I keep getting the following error WARN Error while fetching metadata with correlation id 39 : {4-3-16-topic1=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) Kafka version: 2.11-0.9.0.0 26 Answers 26
I am relatively new to Kafka. I have done a bit of experimenting with it, but a few things are unclear to me regarding consumer offset. From what I have understood so far, when a consumer starts, the offset it will start reading from is determined by the configuration setting auto.offset.reset (correct me if I … Read more
Is there a way to purge the topic in kafka? I pushed a message that was too big into a kafka message topic on my local machine, now I’m getting an error: kafka.common.InvalidMessageSizeException: invalid message size Increasing the fetch.size is not ideal here, because I don’t actually want to accept messages that big. 24 Answers … Read more
Although I’ve come across Kafka before, I just recently realized Kafka may perhaps be used as (the basis of) a CQRS, eventstore. One of the main points that Kafka supports: Event capturing / storing, all HA of course. Pub / sub architecture Ability to replay the eventlog which allows the ability for new subscribers to … Read more
I am starting to learn Kafka, during my readings, some questions came to my mind: When a producer is producing a message – it will specify the topic it wants to send the message to, is that right? Does it care about partitions? When a subscriber is running – does it specify its group id … Read more
Closed. This question is opinion-based. It is not currently accepting answers. Want to improve this question? Update the question so it can be answered with facts and citations by editing this post. Closed 1 year ago. Improve this question I’ve been asked to evaluate RabbitMQ instead of Kafka but found it hard to find a … Read more
Deletion of a topic has been supported since 0.8.2.x version. You have to enable topic deletion (setting delete.topic.enable to true) on all brokers first. Note: Ever since 1.0.x, the functionality being stable, delete.topic.enable is by default true. Follow this step by step process for manual deletion of topics Stop Kafka server Delete the topic directory, … Read more
