Integration Debugging Playbook

When the API call, webhook, or auth handshake fails at the customer’s edge — the systematic checklist that finds it fast.

Integration bugs live in the gap between systems you do not fully control: your code, the customer’s network and a third-party API. The fastest way through is to stop guessing and move layer by layer (DNS, then TCP and TLS, then authentication, then the payload), confirming each before blaming the next, and to reproduce from the machine that actually fails rather than from your laptop.

Every step should leave evidence: the exact request, the status code, the response body and the request ID. That evidence is also what gets a vendor’s support team to take you seriously.

Read the status code

401 / 403: Auth. Wrong/expired token, missing scope, or IP not allow-listed. Check the exact header.
404: Wrong URL/base path or the resource isn’t there yet. Sandbox vs prod host?
400 / 422: their validation rejected the payload — read the error body and diff it against the schema
409: conflict: a reused idempotency key, a version mismatch, or the resource already exists
3xx: a redirect your client didn’t follow (curl -L), often HTTP → HTTPS; POST may turn into GET
429: Rate limited. Back off (jitter), check Retry-After, batch requests.
5xx: Their side (or a proxy). Retry idempotent calls; capture request-id; escalate with it.
Timeout / no response: Firewall/egress, DNS, or wrong port. Try curl -v from the deploy host.

Auth handshake

Token in the right place: Header vs query vs body. Authorization: Bearer … — exact casing/prefix.
Clock skew: Signed requests / JWT fail if the server clock is off. Sync NTP.
Scopes / audience: Token valid but lacks the scope, or wrong aud for this API.
mTLS / cert: Customer requires a client cert? Verify chain + that it’s actually being sent.
Token refresh: an access token expired mid-run; refresh before expiry and retry once on 401
Wrong environment key: a sandbox key against production, or the reverse — the most common 401 of all

DNS, TLS & network

dig +short api.host: does the name resolve from the deploy host, and to the IP you expect?
nc -zv api.host 443: TCP reachability in one line — separates network blocks from app errors
openssl s_client: -connect host:443 -servername host shows the certificate chain and handshake errors
Expired certificate: yours or theirs; the validity dates are in the s_client output
TLS-inspecting proxy: a corporate proxy swaps certificates — trust its CA or exclude the host via NO_PROXY
IP allow-lists: their firewall must allow your egress IPs, which NAT gateways and serverless platforms can change

Payloads & encoding

Content-Type mismatch: JSON sent as form-encoded, or the reverse — the server sees an empty payload
Character encoding: non-UTF-8 input mangles names; normalize to UTF-8 at the edge
Dates & time zones: send ISO 8601 with an offset, e.g. 2026-10-22T09:00:00Z; naive local times drift by hours
Large integer IDs: IDs above 2⁵³ lose precision in JavaScript — carry them as strings
Pagination: follow the cursor until it is empty, and count distinct IDs to prove nothing was skipped
Null vs missing: many APIs treat an explicit null differently from an omitted field

Retries, limits & timeouts

Exponential backoff: retry with growing, jittered delays; stop at a cap and surface the error
Retry-After: when present on a 429 or 503, wait exactly that long before retrying
Idempotency keys: one per logical operation, so a retried POST cannot charge or create twice
Timeout budget: set connect and read timeouts below your caller’s own timeout, or retries pile up
Batch over loop: one bulk call instead of a thousand single calls keeps you under rate limits

Webhooks (inbound)

Signature mismatch: Verify over the raw body (not re-serialized JSON); use the right secret + algorithm.
Reachability: Is your endpoint public + TLS? Use their delivery log / a request-bin to confirm arrival.
Duplicates / retries: They retry on non-2xx. Be idempotent; ack fast (2xx) then process async.
Replay window: Reject events older than N minutes to stop replay attacks.
Out-of-order events: deliveries arrive in any order — use event timestamps or versions, not arrival order
Test locally: expose localhost through a tunnel, or resend events from the provider’s dashboard

Isolate the layer

curl from the box: Reproduce from the actual deploy host — not your laptop. Egress rules differ.
Log the request-id: Most APIs return one. It’s your ticket when you escalate to their team.
Binary-search the stack: DNS → TCP/TLS → auth → payload. Confirm each layer before blaming the next.
Diff working vs broken: A Postman call that works vs your code that doesn’t → compare headers byte-for-byte.
Change one thing: swap only the token, or only the payload, between attempts — otherwise you learn nothing

Escalate with evidence

Request ID + timestamp: the exact UTC time and request ID are what their support needs to find your call
Minimal curl repro: one failing command with secrets redacted beats a paragraph of description
Expected vs actual: what you sent, what you expected and what came back, headers included
Impact: how many calls or customers are affected, and since when

Recipes

Walk the layers from the deploy host

dig +short api.vendor.com
nc -zv api.vendor.com 443
openssl s_client -connect api.vendor.com:443 -servername api.vendor.com < /dev/null | head -20
curl -sv -H "Authorization: Bearer $TOKEN" https://api.vendor.com/v1/ping

Each line tests one layer: name resolution, TCP reachability, the TLS handshake and certificate, then authentication and the HTTP response. The first line that fails is where to look.

Verify a webhook signature over the raw body (Python)

import hashlib, hmac

def verify(raw_body: bytes, header_sig: str, secret: str) -> bool:
    expected = hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, header_sig)

Sign the bytes exactly as received, since re-serialized JSON changes whitespace and key order; match the provider’s encoding too (hex or base64, any sha256= prefix).

Retry with backoff and an idempotency key (Python)

import random, time, uuid
import requests

def post_with_retry(url, payload, attempts=5):
    key = str(uuid.uuid4())
    for i in range(attempts):
        r = requests.post(url, json=payload, timeout=(3, 10),
                          headers={"Idempotency-Key": key})
        if r.status_code not in (429, 500, 502, 503, 504):
            return r
        delay = float(r.headers.get("Retry-After", 2 ** i))
        time.sleep(delay + random.uniform(0, 1))
    r.raise_for_status()
    return r

The same key on every attempt lets the server drop duplicates, a Retry-After in seconds overrides the computed delay, and jitter stops many clients retrying in lockstep.

An escalation note a vendor can act on

Subject: 401 on POST /v1/orders since 2026-10-22 08:10 UTC (production)
Request ID:  req_8f3a2c (41 more attached)
Time:        2026-10-22T08:14:03Z
Repro:       curl -sv -X POST https://api.vendor.com/v1/orders -H "Authorization: Bearer [redacted]" --json @order.json
Expected:    201 Created, as on 2026-10-21
Actual:      401 {"error":"invalid_token"}; the same key works on GET /v1/ping
Impact:      all order creation for 3 customers since 08:10 UTC

Precise times, request IDs, a one-line repro and the impact let their engineers find your traffic in their logs without a single follow-up question.

Questions about Integration Debugging Playbook

How do I debug a 401 Unauthorized error from an API?

Check the basics in order. Confirm the key or token belongs to the right environment, since a sandbox key used against production is the most common cause. Look at the exact header: its name, the Bearer prefix, and no stray whitespace or quotes. Decode a JWT to check its expiry, audience and scopes, and make sure the server clock is correct, because skew invalidates signed requests. Then reproduce with curl from the same host; if curl succeeds and your code fails, compare the two requests header by header.

Why does webhook signature verification fail?

Almost always because the signature was computed over different bytes from the ones the provider signed. Frameworks often parse the JSON body before your handler runs, and serializing it again changes whitespace, key order or Unicode escaping, so verify against the raw request body exactly as it arrived. Other causes are the wrong secret, often one from a different environment or endpoint, the wrong algorithm or encoding such as hex versus base64, a timestamp missing from the signed string, and reading the wrong header.

What is the difference between a 401 and a 403 error?

A 401 Unauthorized means the request was not authenticated: credentials were missing, expired or could not be verified, and the response should include a WWW-Authenticate header describing how to authenticate. A 403 Forbidden means the server knows who you are but you are not allowed to do this, because of missing scopes or roles, an IP allow-list or an account restriction. So for a 401, fix the credential itself, and for a 403, fix the permissions attached to it. Some APIs blur the two, so read the error body as well.

How should I retry failed API calls?

Retry only failures that are likely to be transient, such as timeouts, connection resets, 429 and 502 to 504, and never client errors such as 400 or 401. Use exponential backoff with random jitter, honor Retry-After when the server sends it, and cap both the number of attempts and the total time. Make retried writes safe with an idempotency key, so a request that succeeded but timed out is not applied twice. Finally, log every retry: a steadily rising retry rate is often the first sign of an outage.

Go deeper

HTTP status codes →curl cheatsheet →