Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path →
Security is the
critical path.
An FDE can build the perfect solution and still watch it die in a security review. At most enterprises, the security and compliance team holds a veto — and the fastest way to lose a quarter is to discover their requirements at the end instead of the start. You don’t need to be a security expert; you need to earn their yes.
Security is the critical path
Engineers tend to file security under “things to clean up before launch.” In an FDE deployment, that framing loses deals. At most enterprises the security and compliance function holds a genuine veto: no matter how much the users love your solution, if security says no, it doesn’t ship. And security teams say no reflexively to anything they don’t understand or weren’t consulted on.
So the move is to treat security as a stakeholder from discovery onward, not a gate at the end. Ask early: “What does it take to get a new tool approved here?” Loop the right person in before you’ve committed to an architecture. A security requirement discovered in week one is a design input; the same requirement discovered in week six is a rebuild — or a dead deal.
Security isn’t the thing standing between you and shipping. Security is the path to shipping. Earning their yes is as much a deliverable as the working demo.
The frameworks, decoded
You don’t need to be an auditor. You need to know what each framework actually cares about, so you can answer questions without bluffing.
| Framework | Covers | What it actually asks for |
|---|---|---|
| SOC 2 | General trust controls | Access control, encryption, change management, monitoring, vendor risk. A report, not a law. |
| HIPAA | US health data (PHI) | A signed BAA, encryption, access logs, minimum-necessary access. |
| GDPR | EU personal data | A lawful basis, data-subject rights (access/delete), a safeguard for any transfer outside the EEA (e.g. Standard Contractual Clauses — not data residency as such), a DPA. |
| PCI-DSS | Payment card data | Strict controls — the winning move is usually to never touch card data (tokenize / use a processor). |
The practical takeaway is a hierarchy of avoidance: the easiest compliance problem is the one you design out of existence. Don’t store card data. Don’t move health data off-network. Don’t collect personal data you don’t need. Every category of sensitive data you can avoid touching is an entire class of requirements you skip. The quick-reference keeps these straight in the moment.
Data handling
Underneath every framework sit the same few data disciplines. Get these right and you’re most of the way to any of them.
The non-negotiables
- Encrypt everywhere — TLS in transit, encryption at rest. Table stakes.
- Data residency — know which region/network data must stay in; it drives the deployment target.
- Minimize + retain — collect only what you need; delete on a schedule.
PII discipline
- Redact or mask personal data in logs — a leaked log is a real incident.
- Never paste customer PII into a third-party API without approval.
- Treat the model provider as a subprocessor: what leaves matters.
The phrase to remember is less data, less risk. Every sensitive field you don’t store is a field that can’t leak, can’t be subpoenaed, and can’t sink a review. Data minimization is the cheapest security control there is.
Secrets and access
Secrets are where good intentions meet a leaked GitHub repo. In a customer deployment the rules tighten, and the reviewer will ask about every one of them.
There’s a strategic angle here: least privilege is a speed tactic. A tightly scoped access request is easy for security to reason about and approve; a broad one demands a deep review. Asking for exactly what you need — and no more — is often how you deploy this quarter instead of next.
The security review
Eventually you’ll sit across from (or send a questionnaire to) the security team. Walk in with the answers and it’s a conversation; walk in empty and it’s an interrogation that stalls for weeks.
A data-flow diagram (what data, from where, to where). Where data lives and whether it leaves the network. Encryption in transit and at rest. Your access model and least-privilege scoping. Your subprocessors (including any model provider). Answer these before they’re asked and you’ll clear reviews others get stuck in for a month.
Two principles make reviews go fast. A smaller ask gets a faster yes — minimize scope and footprint so there’s less to scrutinize. And involve them early — a team that helped shape the design defends it; a team ambushed at the end blocks it. Security is a relationship, not a checkpoint.
The AI-specific risks
Modern FDE work adds a new risk surface that traditional security reviews are only starting to catch up with. Know these before the customer asks.
Because so many deployments now put an LLM to work on the customer’s data, three AI-specific risks land squarely on the FDE.
| Risk | What goes wrong | The mitigation |
|---|---|---|
| Prompt injection | Untrusted content (a document, a webhook, a tool result) hides instructions that hijack the model into unintended actions. | Treat tool/document outputs as untrusted input, not commands; keep a human in the loop for consequential actions. |
| Data leakage | Sensitive customer data is sent to a third-party model provider without approval, or logged where it shouldn’t be. | Confirm what may leave the network; self-host the model if it can’t; treat the provider as a subprocessor. |
| PII in prompts / logs | Personal data ends up in prompts, traces, and logs that weren’t designed to hold it. | Redact PII before it reaches a model or a log — a practiced reflex. |
These deserve their own study — the AI security handbook goes deep — but the FDE-level instinct is simple: assume any content the model reads could be adversarial, and assume anything you send to a model could leave. Design around both, and you’ll answer the AI security questions before the customer knows to ask them.
Prompt injection is an architecture problem, not a filtering problem
Their framework predates all of this, so you will be asked to explain the question as well as answer it. The answer that survives a technical review assumes the injection succeeds.
Two properties combine badly in every AI deployment. A model treats retrieved content as information, and content retrieved from a customer’s systems is written by people — including, sometimes, people outside the company. A claim document, a support ticket, a supplier email and a PDF filename are all untrusted text that will end up in a context window next to your instructions.
The instinct is to filter for suspicious phrases. It is a reasonable defence-in-depth layer and a weak primary control, because paraphrase defeats it and because leading with it tells a reviewer that the architectural boundary is not there. The framing that holds is: assume the injection succeeds, and make the consequence small.
Identity
Whose permissions does a tool call run with? It must be the end user’s. A service account holding the union of everyone’s access turns a read tool into an exfiltration path.
Blast radius
Which tools mutate state, and which of those are reversible? Write tools separated, confirmation required, destructive operations have no tool at all.
Content
Is retrieved text ever treated as instruction? It must not be — delimited and labelled as data at the boundary, every time.
assert tool_call.identity == request.user, \
"tools run as the asking user — never a shared service account"
assert retrieval.scope == permissions_of(request.user), \
"enforced at RETRIEVAL time, not filtered after generation"
# filtering citations does not unsee the text the model already read
assert all(t.confirmed for t in tool_calls if t.mutates), \
"writes require explicit confirmation and are reversible"
assert audit.has(user, inputs, retrieved_source_ids, output, model_version), \
"reconstructable six months later, to their retention policy"
# and the demo you will be asked for:
# put hostile text in a record the agent will read. show what happens.
# run it against yourself first — a recorded answer turns a hard
# meeting into a short one.
The second assertion is the one most often got wrong in a way that looks fine. Filtering retrieved chunks against the user’s permissions after retrieval leaves the model having read documents that user cannot see, and the answer it generates is derived from them. The boundary has to be at retrieval or it is not a boundary.
A specific no with a compensating control, an owner and a date outscores a vague yes every time — and outscores a false yes by a margin nothing else recovers. Volunteering a gap raises your score, because finding what you did not mention is the reviewer’s entire job. Practise the full review in the security review gauntlet.
Worked example: designing a review down, step by step
Say you are deploying an assistant that drafts replies for a US health plan’s member-support team. Health plans are HIPAA covered entities, so the claim notes it reads are PHI. The first draft is the quickest one to build. Each step below is a design change that removes something the security review would otherwise have to examine. The thing to watch is the column of places PHI goes outside the customer’s network.
| Step | Design decision | Places PHI is stored or sent | Of those, outside their network | In scope |
|---|---|---|---|---|
| 0 | First draft: your hosted app and vector index, app logs, a third-party tracing tool, a public model API. Ingests the whole member record, including the saved payment card. | 5 | 5 | HIPAA, PCI-DSS |
| 1 | Drop the card field at ingestion. Drafting a reply never needs it. | 5 | 5 | HIPAA |
| 2 | Redact names and member IDs before anything reaches logs or traces. | 3 (app DB, index, model) | 3 | HIPAA |
| 3 | Run the app and the index inside the customer’s own cloud account. | 3 | 1 (the model) | HIPAA |
| 4a | Use the model only under a signed BAA that covers the endpoint you call. | 3 | 1, under contract | HIPAA |
| 4b | Or self-host an open-weights model in their account. | 3 | 0 | HIPAA |
| 5 | Access via their SSO, read-only, scoped to the support queue, short-lived tokens. | 3 | 0 or 1 | HIPAA |
Five places outside their network became one or none, and two frameworks became one. The questionnaire did not get shorter. What changed is that a whole section of it can now be answered “not applicable: no card data is collected”, and the hardest question becomes “PHI never leaves your account”. Step 1 cost nothing and removed PCI-DSS entirely. Step 4 is the real decision. A BAA is a contract, so it costs legal time. A self-hosted model is infrastructure, so it costs engineering time. And the customer’s risk appetite decides which one they will pay for. Expect to sign a BAA yourself too if your engineers can reach the data.
Cost and trade-offs: where the model runs
This is the decision that shapes every review. Here Time means how the security review tends to go, and Space means what you have to build and run.
| Case | Time | Space | Why |
|---|---|---|---|
| Public model API, default terms | Quickest to build; for regulated data, often a flat no | Nothing to run | Data leaves to a party with no BAA or DPA in place. Fine for public or synthetic data only. |
| Model API under a signed BAA or DPA | Legal work rather than engineering work | Nothing to run; one more subprocessor to list | The common middle path. Check which endpoints, regions and features the agreement actually covers. |
| Managed model inside the customer’s cloud account | Reviewed like any other service they already run | Their bill, their region | Data stays under a cloud agreement they already signed. You are limited to the models that cloud offers. |
| Self-hosted open-weights model in their VPC | Easiest yes from security; slowest to build | GPUs, the serving stack and upgrades are yours | Nothing leaves. You own capacity, patching and any quality gap. |
| Air-gapped, on premises | Slowest at every stage | Hardware on site, no remote access | Required for some government and defence work. Every update ships by hand. |
Choose the row the data forces, not the row the demo was built on. Teams that demo on row one and are told in week six that only row four is acceptable rebuild the system. The data-residency question from discovery is how you find out in week one.
Common mistakes
- Waving your SOC 2 report at a custom deployment
A SOC 2 report covers the systems and controls that were in its audit scope, usually your own hosted product. A bespoke deployment inside the customer’s cloud was not audited. Say what the report covers and answer the deployment’s questions directly.
- Treating “EU region” as “GDPR done”
Storing data in Frankfurt does not give you a lawful basis, honour a deletion request or cover a US model provider. A transfer outside the EEA needs a mechanism such as Standard Contractual Clauses, and the provider goes on the DPA’s subprocessor list. Residency is often a customer requirement; it is not the whole of GDPR.
- Debug logging left on after an incident
Someone turns on full prompt and response logging to chase a bug, and it stays on. The logs now hold PHI, the log store becomes in scope, and your data-flow diagram is wrong. Make verbose logging expire automatically, and redact at the logger, not at each call site.
- Deleting the record but not its copies
A deletion request has to reach the vector index (which usually stores the chunk text next to the vector), caches, evaluation sets and fine-tuning data. Keep a map from each source record to everything derived from it, or you cannot honestly say it is deleted.
- Testing on a production extract
Copying “just a sample” of real records to a laptop or a dev environment moves regulated data somewhere that was never approved. Use synthetic data built to the real schema, or test inside the customer’s environment under their controls.
Where this shows up in production and interviews
- The vendor questionnaire
Large customers send a standard one, often based on the SIG (Shared Assessments) or the CAIQ (Cloud Security Alliance), plus their own AI questions. Keep a living answer bank per deployment so the second review takes days, not the whole quarter.
- Healthcare
No BAA, no PHI. Until the agreement is signed, the system runs on synthetic records. That is a useful forcing function: build the redaction and the data-flow diagram before the real data arrives.
- Banks and insurers
Expect a third-party risk review of you and of every subprocessor, and often a separate model-risk review that asks how outputs are validated and monitored. Your eval results and audit log are the evidence they want.
- The FDE interview
A common prompt: “Security blocks your launch two weeks before go-live. What do you do?” A strong answer asks for the specific objection, offers a smaller scope or a compensating control, names an owner and a date, and then explains how you would have brought security in during discovery so this never happened.
Check yourself
You are building search over support tickets, and some customers pasted card numbers into them. What is the cheapest way to keep PCI-DSS out of scope?
Detect and redact card numbers at ingestion, before anything is stored, embedded or sent to a model. A pattern match plus the Luhn checksum catches most of them. Redacting in the UI is too late, because the index and the logs already hold the numbers. Then say exactly that in the review: card data is removed before storage.
Your EU customer’s data is stored in an EU region, but the model provider processes requests in the US. Is the region choice enough?
No. The prompt is a transfer of personal data outside the EEA. It needs a transfer mechanism such as Standard Contractual Clauses, the provider has to be listed as a subprocessor in the DPA, and you still need a lawful basis for the processing. The region choice settles storage, not processing.
Two access requests: read on the whole data warehouse “to move fast”, or read on two views that exclude the PII columns. Which one ships first?
The two views. The reviewer can understand the whole request in a single meeting, and no PII reaches you, which removes a block of questions. The broad grant has to be reviewed against everything in the warehouse. A narrower request is faster to approve even when it takes more work to write.
A model provider offers zero data retention. Does that mean it is no longer a subprocessor?
No. It still receives and processes the data, so it goes on the subprocessor list and needs the contract to match. Zero retention shrinks the risk after the request finishes. It does not change the fact that the data left the network during the request.
Quick answers
What compliance frameworks should an FDE understand?
At a working level: SOC 2 (a report on trust controls like access, encryption, and monitoring), HIPAA (US health data — needs a BAA and strict access controls), GDPR (EU personal data — lawful basis, data-subject rights, safeguards for transfers outside the EEA), and PCI-DSS (card data — best avoided by never touching it). You don’t need to be an auditor; you need to know what each cares about so you can answer a security team’s questions credibly.
How do you pass a customer’s security review?
Come prepared with the answers they’ll ask for: a data-flow diagram, where data lives and whether it leaves the network, how it’s encrypted, your access model, and any third-party subprocessors. Request the minimum access you need — a small, well-scoped ask gets a fast yes while a broad one triggers a long review. And involve security early in the engagement, not at the end, so a veto doesn’t land after weeks of work.
How should you handle secrets in a customer deployment?
Never in code or logs. Store them in the customer’s vault or KMS and inject them via environment at runtime, prefer short-lived credentials over long-lived keys, and scope every credential to exactly what the job needs. Rotating regularly and keeping an audit trail of access are table stakes — security reviewers will ask about all of it.
What are the AI-specific security risks in a deployment?
Three matter most. Prompt injection — untrusted content (a document, a webhook) carries instructions that hijack the model into unintended actions. Data leakage — sending the customer’s sensitive data to a third-party model provider without approval. And PII in prompts and logs — personal data landing in places it shouldn’t. FDEs manage these by treating tool outputs as untrusted, keeping a human in the loop for consequential actions, and redacting PII before it reaches a model or a log.
Explore the topic
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.
More Handbooks
- The Agent Evaluations HandbookA self-contained handbook on evaluating AI agents — theory, interactive widgets, and practical guidance. Trajectory evals, tool-use scoring, LLM-as-judge, observability, and reliability for PMs, engineers, and founders.Read →
- The Agent Patterns HandbookThe design patterns behind every LLM agent — the ReAct Thought–Action–Observation loop, tool/function calling, plan-and-execute, reflection, memory, multi-agent orchestration, and the failure modes (loops, hallucinated tools, recovery, human-in-the-loop) that break agents in production.Read →
- The Kubernetes HandbookThe one idea under all the YAML — declare desired state, and a control loop makes reality match. Covers the orchestration problem, pods, deployments and replicasets, services and networking, the reconciliation loop and self-healing, the scheduler, config/secrets and health probes, autoscaling, and when you actually need Kubernetes.Read →
- The Observability HandbookSeeing inside production — monitoring vs observability, the three pillars (metrics, logs, traces) and what each answers, structured logging, metric types and the cardinality trap, distributed tracing, the golden signals and SLIs/SLOs/error budgets, alerting on symptoms not causes, and correlating all three during an incident.Read →
- The AI Security HandbookYour app now reads the internet and believes it. Why prompt injection is a design problem filters cannot solve, the lethal trifecta (private data + untrusted content + exfiltration), defense in depth from least privilege to output validation, where PII actually leaks (logs, embeddings, weights), MCP/model supply-chain trust, and the red-team eval suite that gates CI.Read →
- The Docker HandbookA container is a lie told by the kernel — namespaces (what a process sees) plus cgroups (what it uses), not a VM. Image layers and the cache-ordering rule that halves build times, multi-stage Dockerfiles that ship small, volumes vs bind mounts, name-based networking, Compose, and the handoff to Kubernetes.Read →