The big idea

What is an API gateway?

You broke a monolith into a dozen microservices — great for the backend. But now the mobile app has to know a dozen hostnames, implement auth against each, handle a dozen retry policies, and re-deploy every time a service moves. Every client re-implements the same plumbing, and every service re-implements auth, rate limiting and logging.

An API gateway is a single front door in front of all your services. Clients call one endpoint; the gateway authenticates, rate-limits, routes each request to the right service, and can even compose several services into one response. Cross-cutting concerns live in one place instead of scattered across every client and every service.

How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the diagram grow, and at the end break a service and kill auth to see how the gateway isolates failure. Hit Begin.

Step 1 · The mess

Clients talking to every service

Without a gateway, each client calls each microservice directly. The mobile app hardcodes a dozen endpoints, re-implements token handling for each, and breaks whenever a service is split or moved. Every service, meanwhile, re-writes auth, rate limiting and logging. What's the underlying problem?

Design decision: Clients call a dozen microservices directly. What is the core problem?

The call: Clients are coupled to internal topology, and cross-cutting concerns are duplicated everywhere. — Clients must know every service and re-implement auth/retries; each service re-implements auth/rate-limit/logging. Both problems vanish if one component sits in front and owns the edge concerns.

Two coupled problems: clients are welded to your internal topology (they must know every service and where it lives), and every service re-implements the same cross-cutting concerns (auth, rate limiting, TLS, logging). Both dissolve if one component owns the edge.

Cross-cutting concerns: Auth, rate limiting, TLS termination, request logging, CORS — these aren't any one service's job, they apply to all traffic. Implemented per-service, they drift and duplicate. Implemented once at the edge, they're consistent and cheap to change.

Step 2 · One front door

A single entry point

You want clients to call one stable endpoint, regardless of how many services exist behind it or how they're split. Where does that consolidation happen?

Design decision: How do you give clients one stable API over many changing services?

The call: Put an API gateway in front: clients hit it, it routes to services. — The gateway is the single public endpoint. Clients know only the gateway; it holds the routing table and forwards each request to the right internal service, which can move or split freely.

Introduce the gateway as the single public endpoint. Clients know only api.example.com. The gateway holds the routing rules and forwards each call to the right service — which can be moved, split, renamed or rescaled without any client ever noticing.

A façade for your backend: The gateway decouples your public API from your internal architecture. The two can evolve independently: refactor services all you want, the client-facing contract stays stable behind the gateway.

Step 3 · Routing

Which service handles this call?

A request for GET /orders/42 arrives at the gateway. It has to reach the orders service — and only the orders service — even as instances come and go. How does the gateway decide?

Design decision: A call to /orders/42 arrives. How does the gateway send it to the right service?

The call: Match the request (path/host/method) against routing rules to a service. — The gateway maps /orders/* → orders service, /users/* → users service, and so on. Rules can key off path, host, method or headers, and forward to a healthy instance of the target.

The gateway routes by matching each request against rules — usually the path (/orders/* → orders service), sometimes host, method or headers — and forwards to a healthy instance of the target service. Add, split or rescale services by editing routes, not clients.

Path-based routing: The routing table is the heart of a gateway: /users/* → users-svc, /orders/* → orders-svc. It can also rewrite paths, strip prefixes, and pick an instance via the service registry (step 8) so it always targets something alive.

Step 4 · One place to check identity

Authentication at the edge

Every service needs to know who is calling and whether they're allowed. Re-implementing token validation in all twelve services means twelve chances to get security wrong. Where should auth happen?

Design decision: Every service needs to authenticate callers. Where should that logic live?

The call: At the gateway: verify the token once, pass identity to services. — The gateway authenticates every request at the edge — validating a JWT (locally, via a cached public key) or an opaque token against the identity service — then forwards the verified user identity to the service, which can trust it.

Do authentication at the gateway. It validates the caller's token on every request — a JWT checked locally against the identity service's public key (no round trip), or an opaque token introspected against the identity service — then passes the verified identity to the downstream service, which trusts the edge. One consistent gate instead of twelve.

Authn vs authz: The gateway is ideal for authentication ("who are you?") and coarse authorization ("can you call this route?"). Fine-grained, data-level authorization ("can you edit this order?") usually stays in the owning service, which alone knows the data. Split the responsibility accordingly.

Step 5 · Protect the services

Rate limiting & quotas

A buggy client in a retry loop, or an abusive third party, can hammer your API and take down the very services behind it. You want to cut them off before they reach a service. Where and how?

Design decision: An abusive caller is hammering your API. Where do you stop them?

The call: At the gateway, with per-client counters (token bucket) in a shared store. — The gateway tracks each client's request rate in Redis (a token bucket) and returns 429 once they exceed their quota — rejecting abuse before it touches any service, consistently across all routes.

Rate-limit at the gateway. Keep per-client counters — a token bucket in Redis so the limit holds across all gateway instances — and return 429 Too Many Requests (with Retry-After) once a caller exceeds its quota. Abuse is shed at the edge, before it ever reaches a service, uniformly across every route.

Token bucket in Redis: Each client gets a bucket that refills at their allowed rate; each request spends a token. Storing buckets in Redis means every stateless gateway instance sees the same count, so the limit is global, not per-box. Tiered quotas (free vs paid) are just different bucket sizes.

Step 6 · Fewer round trips

Aggregation & caching

A mobile home screen needs data from three services. Making the app fire three separate calls over a slow mobile network is wasteful, and hammering the catalog service for the same rarely-changing data on every request is pointless. Can the edge help?

Two edge optimizations. Aggregation (a Backend-for-Frontend): the gateway fans out to several services and composes one response, so a mobile screen makes a single call instead of three. Response caching: cacheable GETs (like catalog reads) are served from a cache at the gateway, so repeat calls never touch the service. Fewer round trips for the client, less load for the services.

BFF & the fan-out: A Backend-for-Frontend is a gateway tailored to one client type (mobile vs web) that aggregates and shapes responses for it. Careful, though: aggregation couples the gateway to service response shapes, so keep it thin. Cache only what's safely cacheable, and honor Cache-Control.

Step 7 · Don't become the bottleneck

Keep the gateway stateless & pooled

Every request now flows through the gateway. If it holds state or runs as one box, you've built a single point of failure and a scaling ceiling in front of everything. How do you keep it from being the weak link?

Keep the gateway stateless — counters and sessions live in Redis, config is pushed in — so you can run many identical instances behind a load balancer and scale them horizontally. Any instance handles any request; losing one costs a retry. The gateway becomes a pool, not a chokepoint.

Stateless at the edge: The same rule as every shared layer: if the box holds nothing precious, it's disposable and scalable. Rate-limit buckets in Redis, JWT keys cached locally, routing config pushed from a control plane — the gateway instance itself carries no per-user state.

Step 8 · The sharp edges

Discovery, timeouts & versioning

Real systems shift under you: service instances come and go, a slow service threatens to stall the gateway, and your public API has to change without breaking existing clients.

Use a service registry so the gateway always routes to healthy, current instances (service discovery). Enforce per-route timeouts + circuit breakers so one slow service fails fast and in isolation instead of tying up gateway connections. And version the public API (/v1, /v2) so you can evolve it while old clients keep working. Watch the gateway from becoming a monolith of business logic — keep it thin.

Design for the unhappy path: Instance moved → discovery, not a hardcoded IP. Service slow → timeout + circuit breaker, isolate it. API must change → version it. Gateway growing brains → push logic back into services. The edge coordinates; it shouldn't become the app.

You did it

You just designed an API gateway.

• D — i
• O — n
• P — a
• A — u
• R — a
• A — g
• K — e