Forward Deployed Engineer / Deploying in Customer Environments

FDE Track 05~13 min readIntermediate

Where It Actually Runs

Ship it where they
can actually run it.

Q: What are the deployment options for delivering software to a customer?

Four targets on a spectrum: your SaaS (multi-tenant, you host), the customer’s cloud/VPC (you deploy into their environment), on-prem (their datacenter), and air-gapped (no internet). Left is fastest to ship/update, right is most locked-down; the customer’s security and data-residency rules usually dictate which.

Q: What is an air-gapped deployment?

A deployment with no public-internet connection (common in defense, finance, healthcare). Everything — images, dependencies, model weights — is brought in deliberately, updates are scheduled and manual, and you often can’t reach the system to debug. The hardest target; it pushes every decision toward self-containment.

Q: How do you host an LLM when data can’t leave the customer’s network?

You can’t call a hosted API, so self-host an open-weight model (e.g. with vLLM) inside the customer’s environment. It trades API convenience for full data control — the model runs where the data lives, nothing leaves. It adds GPU/serving/update work a SaaS deployment avoids, so confirm the constraint is real first.

Q: Why does the deployment target affect the whole architecture?

Because everything downstream depends on it: data egress, how you ship updates, where the model runs, secrets, where logs go, and whether you can access the running system. A design assuming a cloud API and your own dashboards falls apart on-prem. That’s why FDEs ask "where can this run?" in discovery.

The single question that reshapes an FDE architecture more than any other isn’t about the model or the UI — it’s “where can this run?”. The answer, on a spectrum from your SaaS to a fully air-gapped datacenter, decides your data flow, your updates, your model hosting, and even your own ability to touch the thing you built.

The question that reshapes everything

Most engineers design a system and then think about where to host it. FDEs invert that, because the hosting environment isn’t a detail you bolt on — it’s the constraint that determines the design. A solution that assumes a hosted model API, a managed database, and your own observability dashboards is elegant and fast to build. It also becomes impossible the moment the customer says “our data can’t leave our network.”

So the highest-value question in discovery is deceptively simple: “Where can this run, and who approves that?” The answer lands on a spectrum, and every step along it tightens what you’re allowed to do — while the customer’s security posture, not your preference, decides where you land.

The four targets

From the place you control completely to the place you barely control at all.

Target	Who hosts	Best when	The cost
Your SaaS	You, multi-tenant	Data may leave; speed matters most	Blocked outright when data can’t leave their network
Customer cloud / VPC	You deploy into their account	Data stays in their cloud; they’ll grant a scoped role	Their IAM, their network rules, their approval
On-prem	Their datacenter / servers	Regulatory or policy reasons keep everything in-house	You manage installs + updates; no managed services
Air-gapped	Their isolated network, no internet	Defense, finance, healthcare; maximum control	Everything ships in by hand; hardest to build + update

Think of it as a dial from convenience to control. Your SaaS is maximum convenience for you and minimum control for them; air-gapped is the reverse. Most enterprise AI deployments land in the middle two — the customer’s cloud/VPC or on-prem — where you get some tooling but live inside their rules. The deployment-targets cheatsheet is the quick decision aid.

What changes as you move right

Each step toward air-gapped changes the same set of concerns. Internalize this table and you can predict what a given target will demand before you start.

Concern	SaaS / VPC	On-prem / air-gapped
Data egress	Data can leave (SaaS) or stays in their cloud (VPC)	Nothing leaves the network — ever
Updates	You push anytime	Scheduled, manual, versioned bundles brought in deliberately
Model hosting	Hosted LLM API is fine	Self-host an open model (vLLM etc.) — the model runs where the data lives
Secrets	Your vault / their KMS	Their vault, their rotation policy — not your .env
Observability	Your dashboards	Ship logs/metrics to their stack, or run a local one
Your access	Full	Maybe a screen-share and someone else’s hands on the keyboard

→ The one that surprises people

Losing your own access. On a locked-down deployment you may never touch the running system — you debug through a customer engineer on a call. That makes logging, health checks, and a good runbook not niceties but survival tools.

Package portably

The defense against target uncertainty is portability. If discovery hasn’t nailed down the target yet — or if it might change — build so that moving right is a repackaging job, not a rewrite. In practice that means a small set of habits.

Do

Containerize everything (Docker) so it runs the same anywhere.
Depend on open, self-hostable components where the target might be on-prem.
Externalize config — one image, per-environment settings.
Have a plan for self-hosting the model from day one, even if you use an API now.

Avoid

Cloud-only managed services with no on-prem equivalent (if on-prem is possible).
Hardcoding endpoints, regions, or a single cloud’s identity model.
Assuming internet access at runtime.
Baking secrets or per-customer values into the image.

Portability isn’t free — it can cost you some of the convenience of a cloud’s managed magic. But paid up front, it turns “we need this on-prem now” from a project-threatening surprise into an afternoon of repackaging.

Secrets, identity, least privilege

In the customer’s environment, security stops being your convention and becomes their policy. Secrets live in their vault or KMS with their rotation rules; identity flows through their SSO; and every credential you hold should be scoped to the absolute minimum the job needs. This isn’t bureaucracy for its own sake — it’s exactly what gets a security team to approve you, and approval is often the critical path of the whole deployment.

→ Least privilege is a speed tactic

A small, well-scoped access request gets a fast yes; a broad one triggers a long review. Asking for exactly what you need — and no more — is how you deploy this quarter instead of next. Security and compliance are deep enough to warrant their own quick-reference.

The update path

The question new FDEs forget until it bites them: how do you ship v2? On your SaaS it’s a deploy. On-prem it’s a coordinated release. Air-gapped, it might mean handing someone a signed, versioned bundle they carry in on approved media and install during a maintenance window. Decide this before v1, because an architecture that can’t be updated is a liability disguised as a delivery.

Pair the update path with real observability in their world. You may not be able to watch your own dashboards, so wire structured logs, a genuine health check, and alerts into the stack the customer actually uses — and into a channel their team monitors. On a locked-down deployment, knowing the system broke before the customer tells you is often the difference between a quick fix and a lost account. Once it’s deployed and observable, the work turns to hardening and handoff — making the thing durable and then making it theirs.

Frequently asked

Quick answers

What are the deployment options for delivering software to a customer?

Four main targets on a spectrum. Your own SaaS (multi-tenant, you host everything). The customer’s cloud account or VPC (you deploy into their environment). On-prem (their own datacenter or servers). And air-gapped (no internet connection at all). They move from fastest-to-ship-and-update on the left to most-locked-down on the right, and the customer’s security and data-residency rules usually dictate which one you get.

What is an air-gapped deployment?

A deployment in an environment with no connection to the public internet — common in defense, finance, and healthcare. Everything the software needs (container images, dependencies, model weights) has to be brought in deliberately, updates are scheduled and manual, and you often can’t reach the system to debug it directly. It’s the hardest target and reshapes almost every architectural decision toward self-containment.

How do you host an LLM when data can’t leave the customer’s network?

You can’t call a hosted API, so you self-host an open-weight model (e.g. with a serving stack like vLLM) inside the customer’s environment. That trades the convenience of a cloud API for full data control — the model runs where the data lives, and nothing leaves. It adds infrastructure work (GPUs, serving, updates) that a SaaS deployment avoids, so it’s worth confirming the constraint is real before taking it on.

Why does the deployment target affect the whole architecture?

Because everything downstream depends on it: whether data can leave, how you ship updates, where the model runs, how secrets are managed, where logs go, and whether you can even access the running system. A design that assumes a cloud API and your own dashboards falls apart on-prem. That’s why FDEs ask “where can this run?” in discovery — the answer, not the feature list, is the biggest architectural fork.

Finished this one? 0 / 139 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

Forward Deployed Engineering

More Handbooks

Cite this page

Reference it in your work, paper or an AI's context window.

APA

Singh, S. (2026). Deploying in Customer Environments. Vibe Engines. https://vibeengines.com/handbook/deploying-in-customer-environments

MLA

Singh, Saurabh. “Deploying in Customer Environments.” Vibe Engines, 2026, vibeengines.com/handbook/deploying-in-customer-environments.

BibTeX

@online{vibeengines-deploying-in-customer-environments,
  author       = {Singh, Saurabh},
  title        = {Deploying in Customer Environments},
  year         = {2026},
  organization = {Vibe Engines},
  url          = {https://vibeengines.com/handbook/deploying-in-customer-environments}
}