Kafka Partition Calculator
A Kafka partition calculator. Enter your target throughput and the per-partition limits your producers and consumers can sustain, plus how many consumers you want to run in parallel, and get a recommended partition count with the reasoning — because partitions cap both throughput and consumer parallelism. Estimates topic storage from retention and replication too.
In Kafka, the partition count is the one number that quietly decides both your throughput and your parallelism. A partition is the unit of ordering, of parallel consumption, and of bandwidth — and within a consumer group, only one consumer can read a partition at a time. So the partition count is a hard ceiling: you can never run more working consumers than you have partitions, no matter how much hardware you throw at it.
That makes sizing a two-constraint problem. First, throughput: divide your target rate by what a single partition can sustain, on whichever of the producer or consumer side is tighter. Second, parallelism: you need at least as many partitions as consumers you want reading in parallel. The recommendation is the larger of the two, usually rounded up with headroom — because both requirements have to be met at once. This calculator does that max for you and shows why the number came out where it did.
The reason to get it right up front is that growing partitions later is painful. Adding partitions rehashes where keyed messages land, breaking the per-key ordering guarantee across the change, and old data isn’t redistributed. Combine that with a storage estimate — throughput × retention × replication, where the replication factor multiplies your disk footprint — and you can see why partitions and retention are decisions worth making deliberately before launch, not tuning knobs to twist under production load.
How it works
- partitions = max(throughput requirement, desired consumers).
- Considers both producer- and consumer-side per-partition limits.
- Partition count caps consumer-group parallelism — a hard ceiling.
- Estimates storage = throughput × retention × replication.
Frequently asked questions
Why does partition count matter so much in Kafka?
Partitions are Kafka’s unit of parallelism and its throughput ceiling. Within a consumer group, at most one consumer reads a given partition, so you can never have more actively-consuming instances than partitions — the partition count is a hard cap on parallelism. It also bounds throughput, since each partition sustains only so much read/write bandwidth. Too few partitions throttles you; too many adds overhead and rebalancing cost. Picking the number deliberately is the point.
How is the recommendation computed?
It takes the larger of two requirements: the partitions needed to hit your target throughput (target ÷ per-partition throughput, for both the producer and consumer side, whichever is tighter), and the number of consumers you want running in parallel (since parallelism can’t exceed partitions). Then it typically rounds up with headroom. The formula is roughly partitions = max(throughput requirement, desired consumer count), because both constraints must be satisfied.
Can I just add partitions later?
You can increase a topic’s partition count, but it is not free: it changes how keyed messages hash to partitions, so the ordering guarantee for a given key (all messages with the same key go to the same partition) breaks across the change, and existing data is not repartitioned. For keyed topics especially, it is much better to provision enough partitions up front — with some growth headroom — than to add them under load later.
How do I estimate topic storage?
Storage ≈ throughput × retention period × replication factor. If a topic ingests 10 MB/s, retains data for 7 days, and replicates 3×, that is roughly 10 MB/s × 604,800 s × 3 ≈ 18 TB. Replication multiplies the on-disk footprint because every partition’s data is copied to that many brokers. This tool estimates it so you can size disks and understand the cost of long retention and high replication.