Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path →
No internet, no telemetry,
no second chance
Some of the highest-value forward deployed work happens in environments where none of your normal tools reach: no package registry, no model API, no logs leaving the building, and a deploy window that opens once a fortnight. Anthropic’s federal FDE role names air-gapped and impact-level deployment directly.29 This is what the work actually involves, and why engineers who can do it are scarce.
The spectrum, because “air-gapped” is used loosely
Customers say “air-gapped” to mean five different things, and the difference decides your entire architecture. Establish which one you are in during discovery, with a concrete question: can a machine in this environment reach a public IP address, and if so, through what?
| Mode | What is actually true | What it costs you |
|---|---|---|
| Their cloud tenancy | Full internet, their account, their controls | Almost nothing architecturally. This is normal work. |
| Private VPC, no public ingress | Outbound egress usually allowed, inbound blocked | Bastion access, PrivateLink endpoints, a VPN conversation. |
| No-egress VPC | Nothing reaches the internet, including your package installs | Mirrored registries, vendored dependencies, no external model API. A real re-architecture. |
| On-prem, disconnected | Their datacentre, updates arrive by an approved transfer process | Everything above, plus their hardware, their OS baseline, their change board. |
| True air gap | Physically separated network; data crosses on removable media through a formal review | Weeks of latency on every change. Your debugging loop is the constraint, not your code. |
“When you deploy an update to an existing system in this environment today, walk me through what happens and how long it takes.” Ask an operator, not an architect. The answer — a fortnightly window, a signed media transfer, a two-week security scan — is your actual iteration speed, and every estimate you give should be built on it rather than on how fast you can write the code.
The compliance vocabulary, for engineers
You do not need to be a compliance specialist. You do need to know what the words constrain.
An authorisation programme for cloud services sold to US federal agencies, with baselines at Low, Moderate and High reflecting the impact of a breach. Practical consequence for an engineer: the service must run inside an authorised boundary, every component in that boundary is inventoried, and changes go through a documented process. A dependency you add is a boundary change, not a pull request.
A parallel scale for US Department of Defense data. IL2 is public/non-critical. IL4 is controlled unclassified information. IL5 raises the bar again, typically requiring dedicated infrastructure and US-person access controls. IL6 covers classified up to SECRET and lives on separate networks entirely. Federal FDE roles reference these levels directly.29 The engineering consequence is a ladder: as the level rises, shared infrastructure, remote access and outbound telemetry each disappear in turn.
SOC 2 Type II — an audited report on controls over a period; what most enterprise buyers ask for first. ISO 27001 — the international management-system standard, common outside the US. HIPAA — US health data, with a business associate agreement that assigns you real liability. GDPR and data residency — where data may physically live, which quietly rules out a lot of architectures. CMMC — the defence supply chain’s certification of the same ideas.
All of these frameworks ask the same three questions in different vocabularies: who can reach the data, can you prove what happened, and can you show the control worked over time. Identity, audit logging and evidence — the subjects of enterprise identity. Build those three well and most of a questionnaire answers itself; build them late and every framework costs you separately.
Getting model weights into a disconnected network
The step people underestimate, because in every other context it is a download.
An open-weights model of useful size is tens to hundreds of gigabytes. Moving that across an approved boundary is a logistics exercise with a security review attached, and it is normally the long pole in the first delivery.
1. licence confirm the weights may be used in this environment, at this level
2. acquire download once, in a clean staging environment you control
3. verify checksums against the publisher — and record them; you will be asked
4. scan malware + supply-chain scan of weights, container images and deps
5. package one artefact: weights + runtime + config + manifest + hashes
6. transfer approved media or a one-way diode, with a signed manifest
7. re-verify checksums again inside the boundary, before anything runs
8. register the artefact enters their inventory as a controlled component
# elapsed time for step 6 alone, in a real programme: days to weeks
Package once, correctly
- Everything in one signed artefact — weights, runtime, model config, tokeniser, dependencies.
- A manifest listing every file with its hash and its licence.
- A version string that is meaningful to their change board, not a git SHA.
- Offline install instructions that assume no registry, no DNS, and an operator who has not met you.
- A rollback artefact, transferred at the same time. There will be no second transfer this week.
Discovered the hard way
- A container image that pulls a base layer at run time. Nothing pulls anything in there.
- A Python package that compiles from source and needs a toolchain nobody installed.
- A model runtime that phones home for a licence check on start-up.
- Hardware that does not match: different GPU generation, different driver, different quantisation support.
- Forgetting the tokeniser, or shipping a config that references a HuggingFace path.
Before any transfer, install your package on a machine with networking physically disabled. It takes an afternoon and it catches nearly every failure above. The alternative is discovering the missing base layer inside a facility where you cannot download anything, cannot take your laptop in, and cannot try again until the next window.
Updates without an update channel
Every convenience of continuous delivery is gone. What replaces it is a slower loop that must be made reliable, because you get few attempts and each one is expensive.
Batch on their cadence, not yours
If the window is fortnightly, your release train is fortnightly. Bundle changes, test the bundle as a unit, and stop thinking in individual commits. A change that misses the window waits, so scope each bundle to what you are confident about rather than what is finished.
Make configuration a separate, faster path
Prompts, thresholds, routing rules and feature flags should be data an authorised operator can change in the environment — not code requiring a full transfer. This single design decision is the difference between tuning a system in a day and tuning it in a month, and it is much harder to retrofit than to build.
Ship diagnostics with the product
You will not be able to attach a debugger, and often cannot see a log. Build a self-test command that exercises every dependency and prints a readable report, and a redaction-safe bundle an operator can produce and pass to you through the approved channel. Design what goes in that bundle deliberately — it is your only window.
Make rollback a first-class artefact
Versioned, transferred alongside the update, tested. In a connected environment rollback is a deploy command; here it is a physical process, and if the rollback package is not already inside the boundary it does not exist.
Write for an operator who cannot ask you a question
Runbooks in these environments are read by someone with no ability to phone a friend. Every step deterministic, every expected output shown, every failure branch named. This is enablement under the hardest possible constraint.
Evals. The loop everyone relies on — observe production, collect failures, improve — assumes you can see production. You cannot. The substitute is a golden set that lives inside the boundary, run by their operators against every release, with only the aggregate scores coming back out through the approved channel. Design that at the start; retrofitting an eval harness into a disconnected environment is far harder than building one. See evals for client work for the harness itself.
Working inside the constraints
The environment shapes the day, not just the architecture.
What is genuinely different
- You may not bring a phone or a laptop into the facility. Your notes are paper.
- You cannot search the internet mid-problem. Bring documentation in.
- Access clearance can take months and gates the work entirely.
- You may be unable to discuss specifics of the problem with your own colleagues.
- Pairing with a cleared operator often replaces pairing with an engineer.
Habits that make it survivable
- Prepare an offline kit: docs, dependency mirrors, reference implementations, a working local model.
- Write hypotheses down before each session — you get one experiment run, not twenty.
- Make the system explain itself, because you cannot inspect it from outside.
- Over-communicate outside the boundary, so the people who cannot see the work still trust it.
- Keep a decision log. Six months later nobody will remember why a threshold is 0.7.
This work pays well and is genuinely scarce, and it has real costs the recruiting conversation tends to skip: clearance processes that constrain travel and personal life, work you cannot put in a portfolio or describe in an interview, on-site requirements that are not negotiable, and skills that are deep but narrow. It suits some people enormously. Decide with the trade-offs visible — is forward deployed engineering worth it covers the wider version of this question.
Regulated deployments are won and lost in the review
The picture: one artefact across the boundary
Worked example: sizing the artefact and the tuning loop
Two numbers decide a first delivery: how many bytes have to cross, and how many crossings the tuning takes. Weights only, at 1 GB = 109 bytes. The tuning plan is illustrative.
| Step | Question | Arithmetic | Answer |
|---|---|---|---|
| 1 | 70B-parameter model at 16-bit | 70 billion × 2 bytes | 140 GB |
| 2 | Same model at 4-bit | 70 billion × 0.5 bytes | 35 GB, if their runtime and GPUs support it |
| 3 | An 8B model at 16-bit | 8 billion × 2 bytes | 16 GB |
| 4 | Update plus rollback in one crossing (70B, 16-bit) | 2 × 140 GB | 280 GB before runtime and images |
| 5 | Six tuning rounds, each a code transfer, fortnightly window | 6 × 14 days | 84 days, about a quarter |
| 6 | Same six rounds as operator config changes, one a day | 14 days to ship the config path + 6 | 20 days |
Rows 5 and 6 are the page’s “make configuration a separate path” advice in numbers: the same tuning takes 84 days or 20. Rows 1 to 4 explain why a smaller model is often the right call in here. Scan time, media size and GPU memory all grow with bytes, and the GPU also needs memory for the KV cache on top of the weights.
Cost and trade-offs inside the boundary
Here Time is what a choice costs you per change, and Space is what it adds to ship, store or review.
| Case | Time | Space | Why |
|---|---|---|---|
| A smaller model (8B, not 70B) | Faster to scan, transfer and load | 16 GB, not 140 GB, at 16-bit | Often enough for a narrow task. Prove it on the golden set inside the boundary, not on a public benchmark. |
| 4-bit quantisation | One more build and eval pass per release | A quarter of the 16-bit bytes | Only works if their runtime and GPU generation support the format. Test on matching hardware. |
| Config as data | Tuning in days, not windows | A schema, validation and an audit log of every change | Each setting an operator can change is a control the reviewer will ask about. Log who changed what. |
| Vendored dependencies | Each release installs on its own | Every artefact repeats the dependencies | The alternative, a mirrored registry inside, needs its own approval and its own update process. |
| Aggregate-only eval results | Slower diagnosis of any regression | A few numbers cross out per release | Split the golden set into named categories, so a score per category still points at the cause. |
The common thread: every crossing is expensive, so you spend bytes and design effort up front to make crossings rarer and safer.
Where this shows up in production and interviews
- Defence and federal work
The far end of the spectrum: IL5 and IL6 workloads, on-site work, cleared operators and paper notes. It is the case the page’s federal FDE reference describes.
- Regulated commercial work
Banks, insurers and health systems more often sit in the middle of the spectrum: a no-egress VPC or an on-prem cluster with a change board. The model API you planned to call is usually the first thing ruled out.
- Disconnected by circumstance
Factories, ships and remote sites lack connectivity for physical reasons rather than policy. The same habits apply: one artefact, a self-test, config as data and a rollback already in place.
- The interview version
“Design an assistant for a customer whose network cannot reach any external API.” Ask which mode on the spectrum they are in first. Then cover model size against their GPUs, the artefact and its hashes, the config path, a golden set inside the boundary, and the diagnostics bundle.
Check yourself
The customer says the network is air-gapped, but a server there can run pip install. Which mode are you really in?
Not a true air gap. Something can reach a package index, so there is outbound egress, perhaps through a proxy. Ask the page’s question: can a machine reach a public IP, and through what? The answer moves you from a weeks-long media process to a VPN and endpoints conversation.
Inside the boundary, the service fails at start-up while fetching its tokeniser. What would have caught it?
Installing the artefact on a machine with networking physically disabled before the transfer. A config that points at a HuggingFace path, or a missing tokeniser, fails there in minutes instead of costing a whole window.
An operator wants to raise a confidence threshold from 0.7 to 0.8. Code release or config change?
A config change, which is the point of building that path. But it still changes behaviour, so it should run the golden set inside the boundary and be written to the audit log with who changed it and why.
Only aggregate scores leave the boundary, and the score fell after a release. How do you find the cause?
Design for this in advance. Report scores per category of the golden set, so the drop points at a class of case. Ask operators for the redaction-safe diagnostics bundle. If the cause is not clear quickly, roll back, since the rollback artefact is already inside.
Quick answers
What does air-gapped actually mean in an AI deployment?
It is used loosely for five different modes: their cloud tenancy with full internet; a private VPC with no public ingress but outbound egress; a no-egress VPC where nothing reaches the internet including package installs; on-premise and disconnected, where updates arrive through an approved transfer process; and a true air gap where networks are physically separated and data crosses on removable media. Establish which one during discovery by asking whether any machine can reach a public IP address.
What are FedRAMP and DoD impact levels?
FedRAMP is the US federal cloud authorisation programme with Low, Moderate and High baselines reflecting breach impact. DoD Impact Levels are a parallel scale for defence data: IL2 for public and non-critical, IL4 for controlled unclassified information, IL5 typically requiring dedicated infrastructure and US-person access controls, and IL6 for classified up to SECRET on separate networks. As the level rises, shared infrastructure, remote access and outbound telemetry each disappear in turn.
How do model weights get into an air-gapped environment?
Confirm licensing for that environment, download once in a clean staging area, verify checksums against the publisher and record them, scan the weights and images, package everything into one signed artefact with a manifest, transfer via approved media or a one-way diode, re-verify checksums inside the boundary before anything runs, and register the artefact in their inventory. Transfer alone commonly takes days to weeks. Always transfer a tested rollback artefact at the same time.
How do you ship updates when there is no update channel?
Batch releases onto the customer’s change window rather than your own cadence, and make configuration — prompts, thresholds, routing rules, feature flags — a separate faster path that an authorised operator can change in place without a full transfer. Ship a self-test command and a redaction-safe diagnostic bundle, treat rollback as a first-class artefact transferred alongside the update, and write runbooks for an operator who cannot ask you a question.
What breaks first in a disconnected AI deployment?
Evaluation. The normal loop of observing production, collecting failures and improving assumes you can see production, and you cannot. The substitute is a golden set that lives inside the boundary, run by the customer’s operators against every release, with only aggregate scores returning through the approved channel. Design this at the start — retrofitting an eval harness into a disconnected environment is substantially harder than building one.
Sources
Every number on this page traces to one of these. Where a figure is self-reported or crowd-sourced rather than first-party, it is labelled inline.
Cited on this page
- Anthropic — Forward Deployed Engineer, Applied AI (JD) — 40/30/30 split; MCP servers, sub-agents and agent skills as deliverables. jobs.menlovc.com/companies/anthropic/jobs/69674588-forward-deployed-engineer-applied-ai
- Anthropic — Forward Deployed Engineer, Federal Civilian (JD) — air-gapped and IL-level deployment requirements. jobs.menlovc.com/companies/anthropic/jobs/76278352-forward-deployed-engineer-federal-civilian
Explore the topic
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.
More Handbooks
- The Agent Evaluations HandbookA self-contained handbook on evaluating AI agents — theory, interactive widgets, and practical guidance. Trajectory evals, tool-use scoring, LLM-as-judge, observability, and reliability for PMs, engineers, and founders.Read →
- The Agent Patterns HandbookThe design patterns behind every LLM agent — the ReAct Thought–Action–Observation loop, tool/function calling, plan-and-execute, reflection, memory, multi-agent orchestration, and the failure modes (loops, hallucinated tools, recovery, human-in-the-loop) that break agents in production.Read →
- The Kubernetes HandbookThe one idea under all the YAML — declare desired state, and a control loop makes reality match. Covers the orchestration problem, pods, deployments and replicasets, services and networking, the reconciliation loop and self-healing, the scheduler, config/secrets and health probes, autoscaling, and when you actually need Kubernetes.Read →
- The Observability HandbookSeeing inside production — monitoring vs observability, the three pillars (metrics, logs, traces) and what each answers, structured logging, metric types and the cardinality trap, distributed tracing, the golden signals and SLIs/SLOs/error budgets, alerting on symptoms not causes, and correlating all three during an incident.Read →
- The AI Security HandbookYour app now reads the internet and believes it. Why prompt injection is a design problem filters cannot solve, the lethal trifecta (private data + untrusted content + exfiltration), defense in depth from least privilege to output validation, where PII actually leaks (logs, embeddings, weights), MCP/model supply-chain trust, and the red-team eval suite that gates CI.Read →
- The Docker HandbookA container is a lie told by the kernel — namespaces (what a process sees) plus cgroups (what it uses), not a VM. Image layers and the cache-ordering rule that halves build times, multi-stage Dockerfiles that ship small, volumes vs bind mounts, name-based networking, Compose, and the handoff to Kubernetes.Read →