Design an Audit Trail

The answer, out loud

What a strong answer to “Design an Audit Logging & Compliance Trail” sounds like, first question to last trade-off. It is about 7 minutes of talking; the whiteboard and the interviewer fill the rest of the 45. Read it aloud once, then close the page and give it yourself.

1. 0–4 min

   What makes this different from logging

   Before I draw anything I want to say what this system is for, because it decides every choice after it. We already have logs. What we don’t have is evidence — something a customer’s security reviewer, or a regulator, can use to answer who accessed or changed a thing, and then believe the answer even though we run the system. That second half is the hard part. The trail has to hold up against the people who operate it, including our own admins. Everything I add is there to make a specific doubt unreasonable.

2. 4–9 min

   Start with a dedicated trail

   So the first thing I’d do is separate it. The application emits audit events into its own store, not into the same place as debug output. People often want to skip this and mine the existing logs instead, and it fails for boring reasons rather than clever ones: those logs only contain what somebody happened to print, they get rotated away on their own schedule, and they can be edited. So this is a separate system with its own guarantees, and the rest of the session is me adding each guarantee in turn.

   Built in step 1: Record what happens
3. 9–16 min

   What an event has to contain

   Next, what goes in one entry. I want a fixed structure on every consequential action: who — the resolved identity, not just a session token — what, meaning the action and the specific target, when, and where the request came from, plus the before and after when something changed. The reason I insist on one schema across all actions is that an auditor should be able to query the trail, not read it like prose. And I want this written at the moment the action happens. If it can be reconstructed afterwards, someone will reconstruct it differently later, and now there are two versions of the truth and no way to tell which one the system meant.

   Built in step 2: Structured who / what / when
4. 16–23 min

   Make it unrewritable, off the hot path

   Now the part I’d push hardest on. This store has to be append-only and write-once. The usual counter-proposal is to restrict edits to admins, and that concedes the whole argument, because admins are exactly who the trail is supposed to hold accountable. If anyone can edit it, an auditor has no reason to trust any entry, including the ones nobody touched. So immutability is a property of the storage, not a permission somebody can be granted. The practical worry is that audit now sits in the request path, so I’d carry events over a durable, ordered pipeline into the store. The app doesn’t wait on it and a spike doesn’t lose events.

   Built in step 3: Append-only + immutable
5. 23–30 min

   Being able to prove it, not assert it

   Append-only stops the ordinary case, but someone with access to the storage layer underneath could still splice a record in or take one out. At this point I can only assert the trail is intact, and I’d rather be able to prove it. So each entry carries a hash of the one before it. Any edit, deletion or reorder changes a hash and the chain stops validating, so tampering becomes something you detect rather than something you forbid and hope. People sometimes suggest encrypting the log instead. That protects confidentiality, which is a different problem — an attacker who can write can still remove a ciphertext record and nobody is any wiser. And I’d sign the head of the chain periodically so the history is anchored to something outside this system.

   Built in step 4: Tamper-evidence
6. 30–36 min

   Keeping it exactly as long as we should

   Retention I’d frame as a control that cuts both ways, because people usually only see one side of it. We have to keep records long enough to answer questions later, so there’s a required window per record type. But holding personal data longer than we need is itself a risk and, under data-protection rules, its own violation — keeping everything forever isn’t the safe default it sounds like. So the lifecycle is explicit: hot while it’s queried often, archived to cold storage as it ages, deleted once the window passes, with legal hold able to pause that deletion while an investigation is open.

   Built in step 5: Retention + archival
7. 36–42 min

   Letting auditors in without opening a hole

   Last piece is reading it. Auditors need to search by actor, by target, by time range, and that wants an index over the store rather than scanning it. But I’d flag that this trail is now one of the most sensitive datasets we own — it’s a record of who touched whose data — so unrestricted access to it is a breach with extra steps. So reads are access-controlled, sensitive fields are redacted on the way out, and every read is itself an audited action. That last one closes the loop: the people watching the system are watched too, and nobody browses the accountability record quietly.

   Built in step 6: Query + access control
8. 42–45 min

   The trade-off, and what I’d do next

   To close: the trade I’ve made throughout is trustworthiness over convenience. An immutable, chained, retained store is harder to operate than a log table — you can’t fix a bad entry, anything you might ever have to erase has to be kept out of it from the start, and you pay to keep and to verify. I think that’s the right side to be on, because a trail that’s easy to change is worth nothing at the moment it matters. With more time I’d work through two things: how we evidence that an erasure actually happened, given the way to honour one here is to destroy the key that made a record readable, and the record that would show it is the one we just made unreadable — and how an auditor verifies the chain themselves rather than taking our verification on trust.