ReviseAlgo Logo

Producers & Message Delivery

Producer Acknowledgments — acks=0, acks=1, acks=all

Trade-offs between durability and latency.

Introduction

Producer acknowledgments (acks) define the durability guarantees for messages sent to the Kafka cluster. By tweaking this parameter, engineers configure how many replica nodes must write a message before the broker acknowledges the write back to the producer.

Why It Matters

Configuring acks is a fundamental trade-off between data durability and write latency. Choosing the wrong setting can lead to silent message loss on broker crashes, or create unnecessary write bottleneck delays.

Real-World Analogy

- acks=0: Sending a postcard. You drop it in a mailbox and assume it arrives. No confirmation.
- acks=1: Handing a registered package to a clerk. They scan it and print a receipt (leader acknowledges). If the post office burns down that night, the package is lost.
- acks=all: Handing a package to a bank vault. The clerk and two witnesses sign off, securing copies of the receipt in separate branches. High security, slower processing.

How It Works

Sender client configurations map as follows:

  • acks=0: Senders consider writes successful the moment the network packet is sent. No broker response is expected.
  • acks=1: Senders consider writes successful once the partition leader broker writes the record to its local log. Does not wait for follower replicas.
  • acks=all (or acks=-1): Senders consider writes successful only after the partition leader and all active in-sync replicas (ISR) write the record to disk.

Internal Architecture

The acks=all guarantee works in tandem with the broker parameter min.insync.replicas. If a topic has a replication factor of 3 and min.insync.replicas=2, a write with acks=all will succeed as long as at least 2 replicas (leader + 1 follower) are active. If the active ISR list drops below 2, the write fails with NotEnoughReplicasException.

Visual Explanation

Below is a sequence flow diagram showing client write coordination under different acknowledgments settings: 

[acks=0]   Client ──(Send)──> [Leader] (No response expected)

[acks=1]   Client ──(Send)──> [Leader (Writes to Log)] ──(Ack)──> Client

[acks=all] Client ──(Send)──> [Leader (Writes)] ──> [Followers (Sync)]
                                                         │
                               Client <──────(Ack)───────┘

Practical Example

Below is a configurations setup demonstrating how to enable strong durability guarantees using acks=all along with a retry mechanism:

Common Mistakes

  • Setting acks=all without min.insync.replicas: If min.insync.replicas is left at the default 1, then acks=all behaves exactly like acks=1 if follower replicas crash. A single broker failure can lead to data loss.
  • Using acks=0 for critical data: Doing so maximizes throughput but makes debugging impossible, as the producer cannot detect network dropped packets or broker disk failures.

Quick Quiz

Q1: What exception is thrown if you send a write with acks=all but the number of active ISRs is below min.insync.replicas?

NotEnoughReplicasException.

Q2: Which configuration provides the lowest write latency?

acks=0.

Scenario-Based Challenge

The Challenge: Balancing financial logs

You are building an audit stream. Financial records must never be lost. Senders require write confirmations, but also need to handle temporary broker offline events without dropping data. How do you configure the producer and topic?

Solution Tip: Configure acks=all, set min.insync.replicas=2 on a 3-replica topic, and set retries=Integer.MAX_VALUE on the producer client.

Debugging Exercise

A broker crashes, and a follower node is elected leader. Senders report that several orders processed successfully prior to the crash are missing from the log.

The Fix: The producer used acks=1. The leader wrote the order and sent an ACK, but crashed before the followers synced. Senders did not retry because they received an ACK. The fix requires setting acks=all to ensure followers sync before acknowledgments are sent.

Interview Questions

1. Explain the relationship between acks=all and min.insync.replicas.

acks=all instructs the leader to wait for all nodes currently in the In-Sync Replica (ISR) list. min.insync.replicas sets the minimum number of ISR nodes required for the leader to accept the write in the first place.

2. Does acks=all guarantee that all replicas in the cluster have written the record?

No. It only guarantees that the active nodes in the ISR list have written the record. If some followers are offline and out of the ISR, the write still succeeds.

Production Considerations

Run with a replication factor of 3, min.insync.replicas=2, and acks=all as the standard baseline for production clusters.