3 articles Kafka

Manual offsets in Kafka Consumers Example

The consumer code in Kafka Producer And Consumer Example so far auto-commits records every 5 seconds. Now let’s update the consumer to take a third argument that manually sets your offset consumption. If you use the value of the last argument equal to 0, the consumer will assume that you want to start from the beginning,…

Partitioning in Kafka Example

DefaultPartitioner is good enough for most cases for sending messages to each partition on a round robin basis to balance out the load. But if you want to control which partition your messages are sent to you need to implement a custom partitioner instead. For this example, let’s assume that we have a retail site that consumers can…

Kafka Producer And Consumer Example

A simple producer/consumer application The Kafka producer will retrieve user input from the console and send each new line as a message to a Kafka server. The consumer will retrieve messages for a given topic and print them to the console. The producer and consumer components in this case are your own implementations of kafka-console-producer.sh…