Vibe Engines Learn
Systems · Networking

Spread the Load.

A load balancer sits in front of a fleet of identical servers and decides, for each incoming request, which server handles it. The policy it uses matters more than it looks. Round-robin deals requests out in strict rotation — simple, but blind to how busy each server actually is. Least-connections sends each new request to the server with the fewest requests in flight — it notices load. When every request takes the same time these behave alike, but real requests vary wildly in duration, and that’s where they diverge: round-robin can hand a burst of slow requests to one unlucky server while least-connections routes around the busy ones. Send traffic, switch the policy, and watch the load pile up or stay even.

round-robin (blind rotation) vs least-connections (load-aware) · uneven durations expose the gap
round-robin

Assign requests to servers in fixed rotation, ignoring current load.

least-connections

Assign each request to the server with the fewest active requests.

random

Pick a server uniformly at random — surprisingly decent, and stateless.

in-flight

How many requests a server is currently processing — its live load.

lb.js — pick the server for each request
Ready
Four backend servers. Each tick a new request (random duration) arrives and is routed by the current policy, and busy servers finish work. Watch the load balance — or not. Tick to run; switch the policy to compare.
round-robin
policy
0
busiest server
0
imbalance

How it works

The gap between the policies is entirely about information. Round-robin uses none: it advances a counter and hands the next request to the next server, so if request durations are equal, load stays perfectly even — but if a run of long requests happens to land on server 2, round-robin keeps dealing to server 3, 4, 1, 2… and dumps the next long one right back on the already-overloaded server 2. Least-connections uses the one piece of state that matters — each server’s current in-flight count — and always steers the new request to the least-loaded server, so slow requests naturally get routed around. The “imbalance” meter (busiest minus least-busy) makes this visible: it stays low under least-connections and spikes under round-robin when durations vary. Random sits in between: with no coordination it still avoids the pathological round-robin lock-step, which is why “power of two random choices” (pick two at random, send to the less-loaded) is a popular, nearly-stateless middle ground.

1

Requests arrive with varying durations

Real traffic is a mix of quick and slow requests. This variance is what separates good load balancing from bad — with equal durations, any policy looks fine.

2

Round-robin rotates blindly

The balancer hands out requests in fixed order, ignoring how loaded each server is. A cluster of slow requests on one server doesn’t change where the next request goes.

3

Least-connections routes to the least busy

The balancer tracks each server’s in-flight count and sends each new request to the least-loaded server, so slow requests get routed around rather than piled on.

✓

Watch the imbalance

The spread between the busiest and least-busy server stays low under least-connections and spikes under round-robin when durations vary. Least-connections adapts to real, uneven load.

Round-robin
blind, simple
Least-conn
load-aware
Power of 2
random ×2, pick less
Sticky
hash for affinity

The code

# choosing a backend server per policy def pick_server(servers, policy): if policy == "round_robin": s = servers[rr_index % len(servers)]; rr_index += 1 return s # blind rotation if policy == "least_connections": return min(servers, key=lambda s: s.in_flight) # load-aware if policy == "random": return random.choice(servers) if policy == "power_of_two": a, b = random.sample(servers, 2) return a if a.in_flight <= b.in_flight else b # near-optimal, cheap

Quick check

1. When do round-robin and least-connections behave differently?

2. What information does least-connections use that round-robin ignores?

3. What is the "power of two choices" load-balancing trick?

FAQ

What is a load balancer?

A component that distributes incoming requests across a pool of backend servers so none is overwhelmed, improving throughput, latency, and availability, and routing around failures via health checks. It operates at L4 (connections) or L7 (HTTP-aware) and uses a policy like round-robin or least-connections.

What are the common load-balancing algorithms?

Round-robin, weighted round-robin, least-connections, least-response-time, random, power-of-two-choices, and hash-based (IP-hash / consistent hashing for affinity or cache locality). The choice depends on request uniformity, whether you need stickiness, and how much state the balancer keeps.

When would you use hash-based (sticky) load balancing?

When the same client or key should consistently hit the same server — for session affinity (server-held session state) or cache locality (same key to same cache node for hit rate). Consistent hashing is preferred because it keeps most keys mapped to the same server when the pool changes.

What is the difference between L4 and L7 load balancing?

L4 works at TCP/UDP, forwarding by IP and port without inspecting content — fast and protocol-agnostic. L7 understands HTTP, routing by path, host, headers, or cookies, terminating TLS, and doing content-based routing — more capable, more costly per request. Systems often use both.

Keep going

Lab

Queue Backpressure →
Handbook

Networking →
System Design

Design a Load Balancer →
Algorithm

Consistent Hashing →
▶  Watch it explained

Load Balancing Algorithms: How the Load Balancer Picks a Server

Finished this one? 0 / 61 Labs done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

Distributed Systems

More Labs

Explore more from Vibe Engines

Handbooks →
Roadmaps →
System Designs →
AI System Designs →
Algorithms →
Paper Breakdowns →
Challenges →