START/STOP
Start ZOOKEEPER:
./zookeeper-server-start.sh ../config/zookeeper.properties
Start KAFKA:
./kafka-server-start.sh ../config/server.properties
BASIC COMMANDS IN KAFKA
Create TOPIC:
./kafka-topics.sh --zookeeper localhost:2181 --create --topic my-first-topic --partitions 3 --replication-factor 1 --if-not-exists
In new versions use:
./kafka-topics.sh --create --topic my-first-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1 --if-not-exists
List the TOPICS:
./kafka-topics.sh --zookeeper localhost:2181 --list /* just the list of the names */
./kafka-topics.sh --bootstrap-server localhost:9092 --list
./kafka-topics.sh --zookeeper localhost:2181 --describe /* more detailed output */
./kafka-topics.sh --bootstrap-server localhost:9092 --describe
./kafka-topics.sh --zookeeper localhost:2181 --list --topic my-first-topic /* list only specific topics */
Alter the TOPICS:
./kafka-topics.sh --zookeeper localhost:2181 --alter --topic my-first-topic --partitions 5 /* changes the topic partitions to 5 */
—> Cannot decrease the partition counts on topics and it is recommended to increase partitions
Delete the TOPICS:
./kafka-topics.sh --zookeeper localhost:2181 --delete --topic my-first-topic
Create Kafka Console Producer
./kafka-console-producer.sh --bootstrap-server localhost:9092 --topic my-first-topic
Create Kafka Console Consumer
/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-first-topic
Additional Parameters:
--from-beginning - will show all previous consumed messages
--partition 0 - will show the messages from the 0 partition
--max-messages 10 - will show first 10 message
--offset [earliest] [latest] - when you use offset you cannot use --from-beginning
Kafka Consumer Groups
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group console-consumer-66813
