kafka-workshop

Exercise: Configuring Multi-Broker Kafka Cluster

In this exercise you will configure a Kafka cluster with 3 brokers.

Duration: 30 mins

TIP Use bit.ly/kafka-exercise-admin-03 to access this exercise.

Review config/zookeeper.properties default configuration file of Zookeeper
Start a Zookeeper server
- Use zookeeper-server-start.sh shell script
Use zookeeper-shell.sh :2181 ls /brokers/ids to make sure no Kafka brokers are available
Review config/server.properties default configuration file of a Kafka broker
Start a Kafka broker with the default configuration
- Use kafka-server-start.sh shell script
Use zookeeper-shell.sh :2181 ls /brokers/ids to make sure that 1 broker is registered
Start another Kafka broker with broker.id=10
- Create a separate server.properties configuration file, e.g. server-10.properties
- Set other configuration properties to have the server up and running
Use zookeeper-shell.sh :2181 ls /brokers/ids to make sure that 2 brokers are registered
Start another Kafka broker with broker.id=20
- Use –override command-line option (of kafka-server-start.sh) to override configuration properties
Use zookeeper-shell.sh :2181 ls /brokers/ids to make sure that 3 brokers are registered

After the above steps you should have broker 0 as the controller broker. Let’s see controller election in action.

Remove /controller znode and observe the logs
After this znode gets recreated, it will contain the ID of the active controller broker