A beginner-friendly Q&A guide that breaks down how Kafka works behind the scenes — from brokers and partitions to group coordinators and offsets.
Does the group coordinator assign a broker to a partition?
No. The group coordinator assigns partitions to consumers, not brokers. Brokers are already assigned to partitions (as leaders) by the Kafka Controller.
Who assigns a broker to a partition?
Kafka Controller. It selects a leader broker for each partition from the list of replicas.
What is the role of the group coordinator in Kafka?
The group coordinator (a broker) is responsible for managing consumer group membership, assigning partitions to consumers, and handling offset commits.
What is the role of the Kafka Controller?
The Kafka Controller is one of the brokers elected to handle partition leader elections, broker failure recovery, and managing replica state (ISR).
What is a partition leader?
A partition leader is the broker responsible for handling all read/write operations for a partition and syncing data to replicas.
If a broker fails, who handles it?
The Kafka Controller detects the failure and reassigns partition leadership to another in-sync replica broker.
Is the controller a separate service?
No. The controller is just one of the brokers in the cluster given extra coordination responsibilities.
Where are consumer offsets stored?
Offsets are stored in an internal Kafka topic called __consumer_offsets. These are managed by the group coordinator and replicated across brokers.
Can one consumer group read the same partition with multiple consumers?
No. Within a group, only one consumer reads from a given partition at a time to ensure message ordering and processing consistency.
Can multiple consumer groups read from the same topic?
Yes. Each group has its own offset tracking and partition assignment, so they consume independently from the same topic.
What happens when a new consumer joins a group?
A rebalance is triggered. The group coordinator reassigns partitions among all active consumers, temporarily pausing consumption.