The big idea
What is service discovery?
In a microservices fleet, "payments" isn't one server — it's a shifting set of instances that autoscale up and down, get redeployed, move hosts, and die. When "orders" needs to call "payments", which IP does it use? Hardcode one and it's wrong within minutes. The addresses are never stable.
Service discovery lets services find each other by logical name instead of hardcoded address. Instances register their location in a registry; consumers look up the current healthy instances of a service and call one. Health checks keep the directory honest, and the registry itself is made highly available.
How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the design grow, and at the end kill the registry and kill health checks to see stale-but-available vs routing-into-the-void. Hit Begin.
Step 1 · The hardcoded trap
Static IPs break
The simplest approach: put the target service's IP in a config file. It works on day one. Why does it fall apart in a real, dynamic fleet?
Design decision: Hardcoded target IPs in config. Why does it break in a microservices fleet?
The call: Config files are hard to edit. — Editing isn't the problem; the churn is. Even a perfectly-edited IP is stale minutes later when that instance is replaced or rescheduled. You need dynamic lookup, not better editing.
Instances are ephemeral: autoscaling, deploys, crashes and rescheduling add and remove them constantly, so any hardcoded address is stale within minutes and can't spread load across the live set. In a dynamic fleet, the mapping from service name → live instances changes continuously. You must look it up, not bake it in.
Names, not addresses: The core shift: consumers should depend on a stable logical name ("payments") and resolve it to a current address at call time — exactly like DNS turns a hostname into an IP, but updated in seconds as instances change.
Step 2 · A live directory
The service registry
If addresses must be looked up, something has to hold the current map of name → live instances. What is it, and how do entries get in?
Introduce a service registry: a live directory mapping each service name to its currently-registered instance addresses. Instances put themselves in it (next step), and consumers query it to get the current set. The registry is the single source of truth for "who's out there right now" — the heart of discovery.
Registry = the phone book: The registry is a fast, frequently-updated key-value directory: payments → [9001, 9002, 9003]. Everything else — registration, health, lookup, HA — is about keeping this directory accurate and available as the fleet churns.
Step 3 · Getting in the book
Self-registration
How do the registry's entries appear and stay current as instances start and stop? Who's responsible for registering an instance?
Design decision: How do instances get into (and out of) the registry as they start/stop?
The call: Each instance self-registers on startup (and deregisters on shutdown), or a platform agent does it for them. — On boot, an instance registers its address with the registry; on graceful shutdown it deregisters. In self-registration the instance does this directly; in third-party registration a platform (like Kubernetes) registers pods for you. Either way it's automatic and tied to the instance lifecycle.
Instances self-register: on startup an instance tells the registry "payments is available at 10.0.0.7:9001", and on graceful shutdown it deregisters. (In the third-party variant, the platform — e.g. Kubernetes — registers/deregisters instances for you as pods come and go.) Registration is automatic and tied to the instance lifecycle, so the directory tracks the fleet without humans.
Self- vs third-party registration: Self-registration: the instance manages its own registry entry (simple, but couples app code to the registry). Third-party: a platform/agent watches instances and registers them (cleaner app, more infra). Kubernetes does the latter — pods are auto-registered and resolvable by service name.
Step 4 · Only the living
Health checks & heartbeats
Graceful deregistration is the happy path — but instances also crash without a chance to deregister. If the registry keeps handing out a dead instance's address, consumers fail their calls. How does the registry know an instance is actually alive?
Design decision: An instance crashes without deregistering. How does the registry stop handing out its address?
The call: Wait for a consumer to report the failure. — Relying on consumers to notice failures is slow and pushes many failed calls before removal. The registry must proactively track liveness via checks/heartbeats.
Track liveness continuously. Either the registry actively probes each instance's health endpoint, or instances send periodic heartbeats with a TTL — miss a few and you're presumed dead. An instance that fails checks (or stops heartbeating) is deregistered, so consumers only ever receive addresses of instances that are healthy right now. Crash cleanup is automatic.
Registered ≠ alive: Because crashes skip deregistration, the registry can't trust the last registration — it must verify. Health checks (active probes) and heartbeats (self-reported liveness with TTL) are how the directory stays honest. This is the same discipline as a load balancer evicting dead backends.
Step 5 · Who does the lookup?
Client-side vs server-side discovery
A consumer needs one healthy instance to call. Should the consumer query the registry and pick an instance itself, or should a router in the middle do it? This choice shapes where the discovery logic lives.
Design decision: Where should the "query registry + pick an instance" logic live?
The call: Either: client-side (consumer queries registry + load-balances) or server-side (a router/proxy does it) — a real tradeoff. — In client-side discovery the consumer asks the registry and load-balances across instances itself — fewer hops, but logic in every client. In server-side discovery a router/proxy (or platform LB) queries the registry and forwards — dumb clients, but an extra hop and component. Pick per ecosystem; service meshes push this into a sidecar.
Two patterns. Client-side discovery: the consumer queries the registry, gets the instance list, and load-balances itself — fewer hops, but discovery logic lives in every client/language. Server-side discovery: a router/proxy (or platform load balancer) queries the registry and forwards the call — clients stay dumb, at the cost of an extra hop and a component to run. Both are valid; modern service meshes push this logic into a per-instance sidecar.
Client-side vs server-side: Client-side: fast, no extra hop, but couples every service to the registry and re-implements balancing everywhere. Server-side: clean clients, centralized policy, extra hop. The sidecar (mesh) model gets the best of both — client-side speed with the logic factored out of app code.
Step 6 · The registry can't lie or die
Consensus & availability
The registry is now critical infrastructure that everything depends on. If it goes down, no one can discover anyone; if two copies of it disagree during a partition, services get contradictory routing. How do you make it both available and consistent?
Back the registry with a consensus system — etcd, Consul, or ZooKeeper (Raft/ZAB) — so a majority quorum agrees on the registry state and it survives node failure without losing data or splitting. This is a deliberate CP choice (consistent, may reject writes during a partition). The alternative, Eureka, chooses AP: stay available and serve possibly-stale data during partitions, betting that slightly-stale routing beats no routing. Pick per your tolerance — and either way, let consumers cache so a registry blip doesn't stop calls.
CP vs AP registry: CP (etcd/Consul/ZK): the registry is always consistent, but may be unavailable for writes during a partition. AP (Eureka): always available, may serve stale entries. Discovery data is often fine slightly stale (a client just retries a dead instance), so many production systems lean AP + client caching for resilience.
Step 7 · Fast and fresh
Caching, watches & DNS
Consumers can't query the registry on every single call — that's a lookup per request and huge load on the registry. But cached instance lists go stale as instances change. How do you get both speed and freshness?
Consumers cache the instance list locally (fast lookups, resilience if the registry blips) and stay fresh via watches/subscriptions — the registry pushes changes so caches update within seconds instead of polling. Expose discovery through familiar interfaces: many systems offer DNS-based discovery (resolve payments.svc to current instances) alongside a rich API. Kubernetes bakes this in — a Service name resolves via cluster DNS to healthy pod endpoints, with kube-proxy/the mesh load-balancing.
Cache + watch: Polling per call would hammer the registry; caching plus a watch (push on change) gives near-real-time freshness with almost no lookup load. DNS-based discovery makes it transparent to apps, at the cost of DNS caching lag — so change-sensitive systems prefer watches.
Step 8 · The sharp edges
Stale entries, herds & split brain
Discovery has classic failure modes: an instance dies but lingers in caches (stale entry), a registry restart triggers a re-registration stampede, and a network partition can split the registry's view.
Tolerate stale entries at the caller with retries, timeouts and circuit breakers — assume any address might be dead and fail over to another instance. Smooth heartbeat/registration storms with jitter and sensible TTLs so a registry restart doesn't get hammered. Handle partitions per your CP/AP choice (quorum rejects a minority side; AP serves stale). And keep discovery decoupled from the call path via caching so the registry is never a synchronous dependency of every request.
Design for the unhappy path: Stale address → retry + circuit-break (never trust a lookup blindly). Restart → jittered heartbeats/TTL. Partition → CP quorum or AP stale. The registry must never be a hard, synchronous dependency of every call — cache and degrade, so a discovery hiccup slows freshness, not the whole fleet.
You did it
You just designed service discovery.
- Hardcoded IPs break because instances constantly autoscale, deploy, crash and move.
- A service registry maps logical names to current live instance addresses — the phone book.
- Instances self-register on start and deregister on stop (or a platform registers them).
- Health checks and heartbeats deregister crashed instances so only live addresses are handed out.
- Client-side vs server-side discovery decides where lookup + load-balancing lives (sidecars split the difference).
- Back the registry with consensus (CP: etcd/Consul/ZK) or accept AP staleness (Eureka) — plus client caching.
- Cache + watch for fast, fresh lookups; DNS-based discovery (as in Kubernetes) makes it transparent.