Monitoring
Kafka Cluster
Apache Kafka™ 2.7.0
@jaceklaskowski
/
StackOverflow
/
GitHub
/
LinkedIn
The "Internals" Books:
Apache Kafka
•
Kafka Streams
## Agenda 1. [Introduction](#/intro) 1. [Broker Metrics](#/Broker-Metrics) 1. [Producer Metrics](#/Producer-Metrics) 1. [Consumer Metrics](#/Consumer-Metrics) 1. [JmxTool](#/JmxTool) 1. [Demo: Monitoring Kafka Cluster Using JConsole](#/demo)
## Credits The slides are based on the following materials: * [Monitoring Kafka](https://docs.confluent.io/current/kafka/monitoring.html#monitoring-kafka)
## Introduction * Kafka brokers and clients report many internal metrics * JMX is the default reporter * You can add any pluggable reporter * Confluent Control Center
## Broker Metrics * kafka.server:type=ReplicaManager * UnderReplicatedPartitions * PartitionCount * LeaderCount * kafka.controller:type=KafkaController * OfflinePartitionsCount * ActiveControllerCount * kafka.server:type=BrokerTopicMetrics * BytesInPerSec * BytesOutPerSec * kafka.controller:type=ControllerStats * kafka.server:type=ReplicaFetcherManager * kafka.network:type=RequestMetrics * kafka.server:type=SessionExpireListener
## Producer Metrics * kafka.producer:type=producer-metrics * kafka.producer:type=producer-node-metrics * kafka.producer:type=producer-topic-metrics
## Consumer Metrics * kafka.consumer:type=consumer-fetch-manager-metrics * client-id * topic * kafka.consumer:type=consumer-coordinator-metrics * kafka.consumer:type=consumer-metrics * kafka.consumer:type=consumer-node-metrics
## JmxTool * **JmxTool** dumps JMX values to standard output ```shell ./bin/kafka-run-class.sh kafka.tools.JmxTool ```
# Demo ## Monitoring Kafka Cluster Using JConsole
## Recap 1. [Introduction](#/intro) 1. [Broker Metrics](#/Broker-Metrics) 1. [Producer Metrics](#/Producer-Metrics) 1. [Consumer Metrics](#/Consumer-Metrics) 1. [JmxTool](#/JmxTool) 1. [Demo: Monitoring Kafka Cluster Using JConsole](#/demo)
# Questions? * Read [The Internals of Apache Kafka](https://bit.ly/apache-kafka-internals) * Read [The Internals of Kafka Streams](https://bit.ly/kafka-streams-internals) * Follow [@jaceklaskowski](https://twitter.com/jaceklaskowski) on twitter (DMs open) * Upvote [my questions and answers on StackOverflow](http://stackoverflow.com/users/1305344/jacek-laskowski) * Contact me at **jacek@japila.pl**
