Kafka Troubleshooting¶

Overview¶

• Ops quick-reference skeleton; broker settings: Kafka Configuration

1. Metadata / connection failures¶

Symptoms¶

Connection to node -1 could not be established or unreachable brokers from Metadata.

Keywords¶

advertised.listeners / NAT / container networking

Steps¶

1. Verify bootstrap addresses are reachable.
2. Check advertised.listeners on each broker.
3. Test every host:port returned in Metadata.
4. Align security protocol with listener.security.protocol.map.

Reference¶

• Kafka Configuration — advertised.listeners

2. Consumer lag keeps growing¶

Steps¶

1. Check consumer process health and rebalance storms.
2. Match partition count to consumer parallelism.
3. Review max.poll.interval.ms and processing time.
4. Inspect broker disk/network and ISR shrink.

3. Under-Replicated Partitions (URP)¶

Steps¶

1. kafka-topics.sh --describe for URP partitions.
2. Check offline replica brokers (disk, GC, network).
3. Tune replica.lag.time.max.ms for cross-DC RTT.
4. Reassign partitions after replicas recover.

4. Disk full / retention issues¶

Steps¶

1. kafka-log-dirs.sh --describe for large topics.
2. Compare broker defaults vs topic-level retention overrides.
3. Verify cleanup.policy for compact topics.
4. Emergency: expand disk or adjust retention per change process.