3 KAFKA TRY: Practice Kafka on LINUX [ Producer Console -> Single Broker -> Consumer Console ]

3 KAFKA TRY: Practice Kafka on LINUX [ Producer Console -> Single Broker -> Consumer Console ]

Step1: Download Kafka

https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.0/kafka_2.11-0.10.1.0.tgz

tar -xzf kafka_2.11-0.10.1.0.tgz

Step2: Start Zookeeper (Terminal1)

è                           Zookeeper coordinates between brokers

è                          We have to start zookeeper before broker/kafka server starts.

cd kafka_2.11-0.10.1.0

//For Linux bin/zookeeper-server-start.sh

//For Windows bin\windows\zookeeper-server-start

bin/zookeeper-server-start.sh config/zookeeper.properties

    In Windows : while starting zookeeper if we get error like : "Classpath is empty. Please build the project first e.g. by running 'gradlew jarAll'"

    Then fix is - https://medium.com/@praveenkumarsingh/confluent-kafka-on-windows-how-to-fix-classpath-is-empty-cf7c31d9c787

Step3: Start Broker/Kafka server (Terminal2)

For Windows:

bin\windows\kafka-server-start.bat config\server.properties

For Linux:

bin/kafka-server-start.sh config/server.properties

Step4: Create topic (Terminal3)

For windows: 

bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myfirstTopic1

For Linux:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myfirstTopic1

 

//We can check list of topics created

bin/kafka-topics.sh --list --zookeeper localhost:2181

//Start Producer to send message to myfirstTopic1 (Terminal 3)

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic myfirstTopic1

 

//Start Consumer to read message to myfirstTopic1 (Terminal 4)

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic myfirstTopic1