Reference
Glossary.
The vocabulary of AI engineering and system design, in plain English — every term defined, and most linked onward to a hands-on lab, challenge or handbook so you can actually learn it.
A
A2A ProtocolThe open standard for agents talking to other agents — discovery, task delegation, and result exchange.AI & LLMsACIDThe four guarantees — Atomicity, Consistency, Isolation, Durability — that make database transactions reliable.Systems & BackendActivation FunctionThe nonlinear function at each neuron that lets a network learn more than straight lines.AI & LLMsActive LearningLetting the model choose which unlabeled examples to have labeled next — labeling the most informative ones first.AI & LLMsACVThe annualised revenue of a customer contract — the number that determines how much deployment effort a vendor can afford to spend.Systems & BackendAgentA model wrapped in a loop that lets it call tools, read the results, and act until a task is done.AI & LLMsAgent EngineerAn engineer who builds, tunes and operates customer-specific agents, and owns their behaviour in production.AI & LLMsAgent HarnessThe scaffolding around a model that turns it into a working system — and, increasingly, the thing engineers build instead of the feature code itself.AI & LLMsAgent SkillsPackaged, on-demand instruction bundles an agent loads only when the task calls for them.AI & LLMsAgentic BrowserA browser driven by an AI agent that navigates and acts on web pages on the user’s behalf.Systems & BackendAir-Gapped DeploymentA deployment in an environment with no network path to the internet — updates and model weights arrive physically or through a controlled transfer.Systems & BackendAnycastOne IP address announced from many locations, so the network routes each user to the nearest instance automatically.Systems & BackendAP2 (Agent Payments Protocol)An open protocol for letting AI agents make payments on a user’s behalf, backed by verifiable mandates.Systems & BackendAPI GatewayThe single entry point that routes, authenticates, and rate-limits every request before it reaches a backend service.Systems & BackendAPI KeyA secret token identifying and authorizing an application (not a user) when it calls an API.Systems & BackendAPI VersioningEvolving an API without breaking existing clients — via version in the path, header, or backward-compatible changes.Systems & BackendApplied AI EngineerA frontier-lab framing of forward-deployed work: an AI engineer pointed at a specific customer’s deployment.AI & LLMsAt-Least-OnceA delivery guarantee that a message is never lost but may be delivered more than once — so consumers must be idempotent.Systems & BackendAttentionThe operation that lets each token look at every other token and weight how much to listen to each.AI & LLMsAttribution GraphA graph tracing which internal features caused a model’s output, used to reverse-engineer its reasoning.AI & LLMsAutoencoderA network trained to reconstruct its input through a narrow bottleneck, learning a compressed representation.AI & LLMsAutoregressiveGenerating a sequence one token at a time, each conditioned on all the tokens produced so far.AI & LLMsAutoscalingAutomatically adding or removing server instances based on load, so capacity follows demand.Systems & Backend
B
B-TreeThe wide, shallow sorted tree behind nearly every relational index — few disk reads to any row.Systems & BackendBackfillRunning a pipeline over past data to populate or correct history — after adding a metric or fixing a bug.Data & RetrievalBackpressureSignaling upstream to slow down when a system can’t keep up, instead of collapsing.Systems & BackendBackpropagationThe algorithm that computes how much each weight contributed to the error, so the network knows which way to adjust.AI & LLMsBatch ProcessingProcessing data in large scheduled chunks rather than continuously as it arrives.Data & RetrievalBatch SizeHow many training examples the model processes before each weight update.AI & LLMsBeam SearchDecoding that keeps the k best partial sequences instead of one — search, not sampling.AI & LLMsBi-EncoderAn encoder that embeds queries and documents separately, so documents can be precomputed and searched fast.Data & RetrievalBias-Variance TradeoffThe core tradeoff in ML: too simple a model underfits (high bias), too complex overfits (high variance).AI & LLMsBlast RadiusHow much of a system a single failure or change can damage — you design to keep it small.Systems & BackendBlock DiffusionA hybrid that runs autoregressively across blocks of tokens but uses diffusion to fill in each block.AI & LLMsBlocker (stakeholder)Someone whose interests your project threatens, and whose objections are therefore an incentive problem rather than a comprehension gap.Systems & BackendBloom FilterA tiny probabilistic set that answers “maybe present” or “definitely absent” — no false negatives.Data & RetrievalBlue-Green DeploymentRunning two identical production environments and switching traffic between them atomically, for a release with zero downtime.Systems & BackendBM25The classic keyword-ranking function — still the baseline every fancy retriever must beat.Data & RetrievalBounded ContextA boundary within which a domain model and its terms have one consistent meaning — the seam for splitting systems and teams.Systems & BackendBulkheadIsolating resources into pools so a failure in one part cannot sink the whole ship — named after a ship’s compartments.Systems & BackendByte-Pair EncodingThe algorithm most tokenizers use: repeatedly merge the most frequent adjacent symbol pair to build a subword vocabulary.AI & LLMs
C
Cache InvalidationKeeping cached data correct as the underlying source changes — famously one of the hard problems.Systems & BackendCache-AsideThe default caching pattern: check the cache, and on a miss load from the database and populate the cache yourself.Systems & BackendCache-ControlThe HTTP header that tells browsers and CDNs how (and how long) they may cache a response.Systems & BackendCanary ReleaseShipping a change to a small slice of production traffic first, watching for regressions, before rolling it out to everyone.Systems & BackendCAP TheoremDuring a network partition a distributed store must choose consistency or availability, not both.Systems & BackendCardinalityHow many distinct values a column or dataset has — high cardinality (many unique) versus low (few).Data & RetrievalCatastrophic ForgettingFine-tuning on a new task quietly erodes capabilities the model already had, because every weight is free to move.AI & LLMsCDNA network of edge servers that cache content near users, cutting latency and origin load.Systems & BackendChain of ThoughtPrompting a model to reason step by step before answering, improving hard multi-step tasks.AI & LLMsChampionThe person inside the customer who wants your project to succeed and spends their own credibility defending it.Systems & BackendChange Data CaptureStreaming a database’s change log to other systems — sync caches, indexes and warehouses without dual writes.Systems & BackendChaos EngineeringDeliberately injecting failures into a production-like system to verify it actually survives them, instead of hoping it would.Systems & BackendChunkingSplitting documents into retrievable pieces — the decision that quietly makes or breaks RAG quality.Data & RetrievalCircuit BreakerStop calling a failing dependency; fail fast, probe for recovery — the pattern that prevents cascade failures.Systems & BackendCNNConvolutional Neural Network: slides small filters over an image to detect local patterns, the workhorse of computer vision.AI & LLMsCold StartThe extra latency when a serverless function or scaled-to-zero service handles its first request and must spin up.Systems & BackendColumnar StorageStoring table data by column instead of by row, so analytics that touch a few columns scan far less data.Data & RetrievalCompactionMerging and rewriting storage files in the background to reclaim space and keep reads fast — the housekeeping of LSM stores.Systems & BackendConfusion MatrixA table of a classifier’s predictions vs. reality — true/false positives and negatives — the basis of most metrics.AI & LLMsConnection PoolA reusable set of open database connections, so requests skip the expensive handshake each time.Systems & BackendConsistent HashingMapping keys and nodes onto a ring so adding a node moves only ~K/N keys, not all of them.Systems & BackendContainerA lightweight, isolated package of an app plus its dependencies that runs the same everywhere — sharing the host OS kernel.Systems & BackendContext EngineeringCurating everything the model sees each step — the discipline that replaced prompt-tweaking.AI & LLMsContext WindowThe maximum number of tokens a model can consider in one request — a hard, shared budget.AI & LLMsContinuous BatchingServing many users per GPU by advancing all in-flight generations together, admitting new ones mid-flight.AI & LLMsContinuous BatchingAdding and removing requests from a running batch on the fly, instead of waiting for a fixed batch to fill before starting.AI & LLMsConvolutionThe core operation of a CNN: slide a small filter across the input, computing a weighted sum at each position.AI & LLMsCookieA small piece of data the server stores in the browser, sent back on every request — how the web remembers you.Systems & BackendCORSThe browser rule that a page cannot call a different origin’s API unless that API opts in with the right headers.Systems & BackendCosine SimilarityHow aligned two vectors are, ignoring their length — 1 identical, 0 unrelated, −1 opposite.Data & RetrievalCQRSSeparate the write model from the read model, optimizing each independently.Systems & BackendCross-AttentionAttention where queries come from one sequence and keys/values from another — how a decoder looks at an encoder.AI & LLMsCross-EncoderA reranker that reads the query and a document TOGETHER for an accurate relevance score — slow but precise.Data & RetrievalCross-Entropy LossThe training loss that measures how surprised the model was by the correct answer.AI & LLMsCross-ValidationEstimating a model’s true performance by training and testing on multiple data splits, not just one.AI & LLMsCSRFCross-Site Request Forgery: tricking a logged-in user’s browser into making an unwanted state-changing request.Systems & Backend
D
DAGA Directed Acyclic Graph: tasks with dependencies but no cycles — the model behind data pipelines and build systems.Data & RetrievalData ContractAn explicit, enforced agreement about the schema and semantics of data passed between a producer and its consumers.Data & RetrievalData FoundryAn operation that manufactures high-quality training and evaluation data at scale, blending human expertise and synthetic generation.Data & RetrievalData LakeA store for raw data in any format at scale, schema applied on read — flexible but easy to turn into a swamp.Data & RetrievalData LeakageWhen information the model will not have at prediction time sneaks into training — great offline scores, collapse in production.AI & LLMsData MeshAn org approach treating data as a product owned by domain teams, not one central data team.Data & RetrievalData ParallelismCopying the whole model onto every GPU and splitting the batch across them, syncing gradients each step.AI & LLMsData PoisoningDeliberately corrupting a model’s training data so it learns a hidden backdoor or a biased, attacker-chosen behavior.AI & LLMsData WarehouseA database optimized for analytics — big aggregate queries over historical data — separate from the transactional systems.Data & RetrievalDatabase IndexA lookup structure that lets the database find rows without scanning the whole table.Systems & BackendDatabase NormalizationStructuring a database to eliminate redundancy — each fact stored once — trading some read joins for write integrity.Systems & BackendDead-Letter QueueA side queue for messages that repeatedly fail processing, so one poison message cannot block the pipeline.Systems & BackendDeadlockTwo transactions each hold a lock the other needs, so both wait forever — the database must detect and kill one.Systems & BackendDecision TreeA model that predicts by asking a series of yes/no questions about the features — interpretable but prone to overfitting.AI & LLMsDecomposition RoundThe signature FDE interview round: an open, non-coding problem where you are scored on the clarifying questions you ask before proposing anything.Systems & BackendDeflection RateThe share of cases resolved without a human touching them — the metric that proves capacity was actually created.Systems & BackendDenormalizationDeliberately duplicating data across tables to make reads faster, trading write complexity for read speed.Systems & BackendDense RetrievalRetrieval by embedding meaning into vectors and finding nearest neighbors — matches on semantics, not keywords.Data & RetrievalDeployment StrategistThe non-engineering half of forward-deployed work: owns the problem framing, the stakeholders and the direction of a deployment.Systems & BackendDifferential PrivacyA formal, mathematical guarantee that no individual record in a dataset can be detected from a system’s outputs.Systems & BackendDiffusion LLMA language model that generates text by iteratively denoising all tokens in parallel, instead of one at a time left to right.AI & LLMsDisaster RecoveryThe plan and infrastructure to restore service after a major outage — a region failure, data loss, or catastrophe.Systems & BackendDistillationTraining a small “student” model to imitate a large “teacher” — capability at a fraction of the cost.AI & LLMsDistributed TracingFollowing one request’s path across every service it touches, via a shared trace ID, to see exactly where time and errors happen.Systems & BackendDNSThe internet’s phone book — it translates human domain names into machine IP addresses.Systems & BackendDomain-Driven DesignDesigning software around the business domain and its language, keeping the model and the code aligned.Systems & BackendDot ProductA similarity measure summing the products of two vectors’ components — bigger when they point the same way and are large.Data & RetrievalDPOPreference alignment as a simple classification loss — RLHF’s results without training a reward model.AI & LLMs
E
Early StoppingHalting training when validation performance stops improving — a simple, effective guard against overfitting.AI & LLMsEconomic BuyerThe person who controls the budget for your project — often not the person you talk to most.Systems & BackendEdge ComputingRunning compute close to users (at CDN edge locations) instead of a central region — for low latency and less origin load.Systems & BackendEffort LevelA control that sets how much reasoning or compute a model spends before answering — usually low, medium or high.AI & LLMsEmbeddingA vector that places a piece of text in space so that nearby vectors mean similar things.Data & RetrievalEmbedding ModelA model whose job is to turn text (or images) into embedding vectors for search, clustering, and RAG.Data & RetrievalEncoder-DecoderAn architecture that reads an input sequence into a representation (encoder) then generates an output from it (decoder).AI & LLMsEngramA stored memory trace an agent writes during a task and later retrieves — a durable record of past experience.AI & LLMsEnsembleCombining several models’ predictions to beat any single one — averaging out their independent errors.AI & LLMsEnvironment HubA shared catalog of RL training environments — tasks bundled with their verifiers — that agents train against.AI & LLMsEpochOne full pass of the training algorithm over the entire training dataset.AI & LLMsETagA fingerprint of a resource’s content that lets the browser ask “changed since this version?” and skip re-downloading.Systems & BackendETLExtract, Transform, Load — the pipeline that moves data from source systems into a warehouse, cleaning it on the way.Data & RetrievalEuclidean DistanceStraight-line distance between two points/vectors — the intuitive “ruler” metric, sensitive to magnitude.Data & RetrievalEval GateA CI check that blocks a change when evaluation results fall below agreed thresholds — the point at which an eval stops being a document.AI & LLMsEvent SourcingStoring state as an append-only log of events, and deriving current state by replaying them — full history, perfect audit.Systems & BackendEvent-Driven ArchitectureAn architecture where components communicate by emitting and reacting to events, rather than calling each other directly.Systems & BackendEventual ConsistencyReplicas may briefly disagree but converge to the same value once writes stop.Systems & BackendExactly-OnceThe hard delivery guarantee that a message is processed once and only once — usually approximated with idempotency.Systems & BackendExponential BackoffRetry failed operations with exponentially increasing waits (plus jitter), so a struggling service is not hammered.Systems & Backend
F
FaaSFunction as a Service: deploy individual functions that the platform runs and scales per-event — the core of serverless.Systems & BackendFailoverAutomatically switching to a standby replica when the primary fails, so service continues with minimal interruption.Systems & BackendFDSEPalantir’s title for the engineering half of forward-deployed work — the person who writes the production code at the customer.Systems & BackendFeature (SAE)An interpretable direction in a model’s activations, pulled out by a sparse autoencoder as a single human-meaningful concept.AI & LLMsFeature EngineeringCrafting better input features from raw data — often the highest-leverage work in classical ML.AI & LLMsFeature FlagA runtime switch that turns features on or off without a deploy — for gradual rollout, testing, and instant rollback.Systems & BackendFeature StoreA central system for computing, storing, and serving ML features consistently between training and production.Data & RetrievalFedRAMPThe US government programme that authorises cloud services for federal use; impact levels (IL4/IL5) grade how sensitive the data may be.Systems & BackendFeed-Forward NetworkThe per-token MLP inside each transformer block — where much of the model’s knowledge is stored.AI & LLMsFencing TokenA monotonically increasing number handed out with a lock, so a stale lock-holder can be rejected.Systems & BackendFew-Shot PromptingTeaching by example inside the prompt — the model imitates demonstrations without any training.AI & LLMsFine-TuningContinuing to train a pretrained model on your data to specialize its behavior.AI & LLMsFlashAttentionAn exact attention algorithm that avoids ever materializing the full attention matrix in slow GPU memory.AI & LLMsForeign KeyA column that references another table’s primary key, letting the database enforce that relationships stay valid.Systems & BackendForward Deployed EngineerAn engineer embedded with a specific customer who owns the full arc of a deployment — discovery to production, inside the customer’s environment.Systems & BackendForward ProxyA proxy in front of clients that forwards their outbound requests — for filtering, caching, or hiding the client.Systems & Backend
G
GANGenerative Adversarial Network: a generator and a discriminator train against each other until the fakes fool the critic.AI & LLMsGELUA smooth version of ReLU used inside most transformers.AI & LLMsGeneralisation MemoThe write-up of what an engagement produced that should become a product capability — the artefact that separates an FDE from a contractor.Systems & BackendGeo-ReplicationKeeping copies of data in multiple geographic regions — for low-latency local reads and survival of a regional outage.Systems & BackendGitOpsOperating infrastructure by making Git the single source of truth: the cluster continuously syncs itself to what is in the repo.Systems & BackendGolden DatasetA curated set of inputs with known-good answers, used as the ground truth for evaluating a model or pipeline.AI & LLMsGossip ProtocolNodes trade state with random peers; information spreads like rumor — no coordinator, log-time propagation.Systems & BackendGraceful DegradationKeeping the core working when a dependency fails by dropping non-essential features, instead of failing entirely.Systems & BackendGraceful ShutdownDraining in-flight work and closing connections cleanly before a process exits, so a deploy drops no requests.Systems & BackendGradient BoostingAn ensemble that builds trees sequentially, each correcting the last one’s errors — the top performer on tabular data.AI & LLMsGradient CheckpointingTrading extra compute for less memory by recomputing activations during the backward pass instead of storing them.AI & LLMsGradient ClippingCapping the size of the gradient before applying it, so one unusually large batch can’t blow up training.AI & LLMsGradient DescentNudge each weight a little in the direction that most reduces the loss; repeat until it stops improving.AI & LLMsGraphQLA query language for APIs where the client asks for exactly the fields it needs in one request.Systems & BackendGreedy DecodingAlways pick the single highest-probability next token — deterministic, but can be repetitive.AI & LLMsGround TruthThe verified correct answer used to train and evaluate a model — the reference reality it is measured against.AI & LLMsGroundingTying model claims to supplied sources — answer from the documents, not from vibes.AI & LLMsgRPCA fast, contract-first RPC framework using Protocol Buffers over HTTP/2 — common for service-to-service calls.Systems & BackendGRPOThe RL method behind DeepSeek-R1: score groups of sampled answers relative to each other, no value network.AI & LLMsGuardrailsThe checks around a model that catch bad inputs and outputs — policy enforced outside the weights.AI & LLMs
H
HallucinationWhen a model states something fluent and confident that is simply not true.AI & LLMsHandoff RunbookSymptom-first operating documentation that lets the customer’s team run the system without the engineer who built it.Systems & BackendHarnessThe orchestration code around a model — the loop, tools, retries, budgets — that turns it into a product.AI & LLMsHealth CheckA lightweight endpoint or probe a system polls to decide whether an instance is ready to receive traffic.Systems & BackendHeartbeatA periodic “I am alive” signal between nodes, used to detect failures — miss enough and you are presumed dead.Systems & BackendHedged RequestSend a duplicate request to a second replica if the first is slow, and take whichever returns first — to cut tail latency.Systems & BackendHNSWThe graph index behind most vector databases — approximate nearest-neighbor search in milliseconds.Data & RetrievalHorizontal ScalingAdding more machines to share load (scale out), versus making one machine bigger (scale up).Systems & BackendHTTP Status CodesThe three-digit codes an HTTP response carries — 2xx success, 3xx redirect, 4xx client error, 5xx server error.Systems & BackendHTTP/2A major HTTP upgrade that multiplexes many requests over one connection to cut latency.Systems & BackendHTTP/3The newest HTTP version, built on UDP/QUIC to eliminate TCP’s head-of-line blocking.Systems & BackendHybrid SearchKeyword search and vector search fused — lexical precision plus semantic recall.Data & RetrievalHyperparameterA setting you choose before training (learning rate, batch size, layers) — as opposed to the weights the model learns.AI & LLMsHypervisorThe software layer that creates and runs virtual machines, sharing one physical machine’s hardware among them.Systems & Backend
I
IaaS / PaaS / SaaSThe cloud service spectrum: rent raw machines (IaaS), a managed platform (PaaS), or finished software (SaaS) — each hands more to the provider.Systems & BackendIdempotencyA property where doing an operation twice has the same effect as doing it once.Systems & BackendImmutable InfrastructureNever patch a running server — replace it with a freshly built one. No config drift, easy rollback.Systems & BackendIn-Context LearningA model adapting to a task from examples in the prompt, without any weight updates.AI & LLMsInferenceRunning a trained model forward to produce an output — as opposed to training, which updates its weights.AI & LLMsInfrastructure as CodeDefining servers, networks, and services in version-controlled code instead of clicking through a console — reproducible infra.Systems & BackendInstruction TuningFine-tuning a base model on many (instruction, response) pairs so it learns to follow instructions instead of just completing text.AI & LLMsInter-Annotator AgreementHow often two humans doing the same labelling task give the same answer — the practical ceiling on any model’s measured accuracy.AI & LLMs
J
K
K-MeansA clustering algorithm that partitions data into k groups by iteratively assigning points to the nearest center.AI & LLMsk-Nearest NeighborsClassify or predict a point by looking at the k closest labeled examples — no training, just distance.AI & LLMsKappa ArchitectureA simplification of lambda that handles both real-time and reprocessing with one streaming path.Data & RetrievalKubernetesThe standard orchestrator that schedules, scales, heals, and networks containers across a cluster of machines.Systems & BackendKV CacheThe stored attention state that lets a model generate each new token without re-reading the whole prompt.AI & LLMs
L
Lambda ArchitectureA data design running a slow accurate batch layer alongside a fast approximate streaming layer.Data & RetrievalLast-Mile GapThe distance between a model that works in a demo and a system that changes a specific organisation’s workflow — the gap FDEs exist to close.AI & LLMsLatent SpaceThe compressed, abstract representation space a model maps data into — where similar things sit close together.AI & LLMsLayer NormalizationRescaling a layer’s activations to a stable range on every forward pass, so deep networks train without exploding or vanishing.AI & LLMsLeader ElectionPicking exactly one node to be in charge — and handling the moment the old leader doesn’t know it lost.Systems & BackendLearning Rate ScheduleChanging the step size during training — usually a short warmup up, then a long decay down — instead of holding it fixed.AI & LLMsLearning RoundAn interview round where you absorb unfamiliar documented material live and apply it — testing learning speed, not recall.Systems & BackendLeaseA lock with an expiry: a node holds a resource only for a bounded time, so a crashed holder cannot block others forever.Systems & BackendLinear RegressionThe simplest predictive model: fit a straight line (weighted sum of features) to predict a continuous value.AI & LLMsLLM-as-a-JudgeUsing a strong model with a rubric to score other models’ outputs, at scale.AI & LLMsLoad BalancerDistributing incoming requests across multiple backend instances, so no single one is overwhelmed and a failed one can be routed around.Systems & BackendLoad SheddingDeliberately dropping or rejecting some requests under overload, to protect a system’s ability to serve the rest.Systems & BackendLogistic RegressionA simple, interpretable model for binary classification — outputs a probability via the logistic (sigmoid) function.AI & LLMsLogit BiasA per-token nudge added to the model’s logits to make specific tokens more or less likely.AI & LLMsLogitsThe raw, unnormalized scores a model outputs before softmax turns them into probabilities.AI & LLMsLoop EngineeringDesigning the system that prompts your agent for you — trigger, framing, verification, state, and a stopping condition.AI & LLMsLoRAFine-tuning by training tiny low-rank matrices alongside frozen weights — 1000× fewer trainable parameters.AI & LLMsLoss FunctionThe single number training tries to minimize — it measures how wrong the model’s predictions are.AI & LLMsLost in the MiddleThe tendency of LLMs to use information best at the START and END of a long context, and miss what is buried in the middle.AI & LLMsLSM TreeWrite fast by appending; read by merging sorted layers — the engine inside Cassandra, RocksDB, LevelDB.Systems & BackendLSTMLong Short-Term Memory: an RNN with gates that let it remember information over long sequences, mitigating vanishing gradients.AI & LLMs
M
MandateIn agent payments, a signed authorization proving a user let an agent transact within specific limits.Systems & BackendMaterialized ViewA precomputed, stored query result you read like a table — fast reads, at the cost of freshness.Systems & BackendMemtableThe in-memory, sorted write buffer of an LSM store — flushed to an immutable SSTable when it fills.Systems & BackendMessage BrokerMiddleware that receives, buffers, and routes messages between producers and consumers.Systems & BackendMessage QueueA buffer that lets services hand off work asynchronously — the producer drops a message, the consumer picks it up later.Systems & BackendMicroservicesSplitting an app into many small, independently-deployable services — team autonomy and independent scaling, at the cost of distributed complexity.Systems & BackendMixed Precision TrainingRunning most of training in 16-bit numbers for speed and memory, while keeping a 32-bit master copy of the weights for stability.AI & LLMsMixture of Experts (MoE)A model that routes each token to a few specialist sub-networks instead of the whole network.AI & LLMsModel CardA short document describing a model’s intended use, training data, performance, and limitations — its nutrition label.AI & LLMsModel Context Protocol (MCP)An open standard for connecting models to tools and data through a common server interface.AI & LLMsModel DriftA deployed model getting worse over time as the real-world data diverges from what it was trained on.AI & LLMsModel ExtractionQuerying a model enough times to train a cheap substitute that mimics its behavior — effectively stealing it without the weights.AI & LLMsModel ParallelismThe umbrella term for splitting one model’s compute across multiple devices — tensor parallelism and pipeline parallelism are its two main forms.AI & LLMsModel RegistryA versioned catalog of trained models — their versions, metrics, and lineage — the source of truth for what is deployed.Systems & BackendModel ServingThe infrastructure that runs a trained model behind an API for real-time or batch predictions.Systems & BackendMomentumAn optimizer trick: accumulate a running average of past gradients to speed up and smooth training.AI & LLMsMonolithAn application built and deployed as a single unit — simple to start, harder to scale a large team on.Systems & BackendMRRMean Reciprocal Rank: rewards how high the FIRST correct result appears, averaged over queries.Data & RetrievalmTLSMutual TLS: both client and server present certificates, so each cryptographically proves its identity to the other.Systems & BackendMulti-Head AttentionRunning several attention operations in parallel, each learning to focus on a different kind of relationship.AI & LLMsMulti-RegionRunning an application across multiple cloud regions for lower latency and resilience — at a big jump in complexity.Systems & BackendMultimodal ModelOne model over several modalities — text, images, audio, video — in a shared token space.AI & LLMsMVCCMulti-Version Concurrency Control: readers see a consistent snapshot while writers create new versions — so reads do not block writes.Systems & Backend
N
N+1 Query ProblemA performance bug where fetching a list then looping to fetch each item’s details fires 1 + N queries instead of one.Systems & BackendNaive BayesA fast probabilistic classifier that assumes features are independent — surprisingly effective for text.AI & LLMsNDCGA ranking metric that rewards putting the most relevant results highest, discounting relevance further down the list.Data & Retrieval
O
OAuthThe standard for letting an app act on your behalf on another service without ever seeing your password.Systems & BackendObservabilityUnderstanding a system’s internal state from its outputs — logs, metrics, and traces — especially for problems you did not predict.Systems & BackendOLAPDatabases optimized for heavy analytical queries over large datasets, not many small transactions.Data & RetrievalOLTP vs OLAPOnline Transaction Processing — the fast, small read/write workload of live apps, versus OLAP analytics.Data & RetrievalOn-Policy DistillationDistilling a teacher into a student using the student’s own sampled outputs, rather than a fixed teacher-generated dataset.AI & LLMsOne-Hot EncodingRepresenting a category as a vector of all zeros with a single one — the raw form before dense embeddings.AI & LLMsONNXAn open format for exchanging trained models between frameworks, so a model trained in one tool runs in another.Systems & BackendOptimistic LockingAssume conflicts are rare: do not lock, but check on write that nobody changed the row since you read it.Systems & BackendOutbox PatternWrite the event in the same DB transaction as the data; publish it afterwards — no lost, no phantom events.Systems & BackendOverfittingWhen a model memorizes the training data’s noise instead of the pattern, so it aces training but fails on new data.AI & LLMsOverride RateThe share of a system’s outputs that a human changes before acting on them — simultaneously a quality signal, a trust signal, and a source of labels.AI & LLMs
P
p99 LatencyThe response time your worst 1% of requests exceed — the tail users actually feel.Systems & BackendPagedAttentionManaging the KV-cache in fixed-size, non-contiguous memory pages — like an OS managing virtual memory — instead of one big reserved block per request.AI & LLMsPaginationReturning a big result set in bounded pages instead of all at once — cursor-based beats offset at scale.Systems & BackendPartition KeyThe field a system uses to decide which shard or partition a row lives on — choose it wrong and you get hotspots.Systems & BackendPCAPrincipal Component Analysis: reduces dimensions by projecting data onto the axes of greatest variance.AI & LLMsPerplexityHow surprised a model is by text — the classic intrinsic measure of language-model quality.AI & LLMsPessimistic LockingAssume conflicts are likely: lock the row up front so no one else can touch it until you are done.Systems & BackendPII RedactionStripping or masking personally identifiable information from data before it reaches a model, a log, or a third party.Systems & BackendPipeline ParallelismSplitting a model’s layers across GPUs, so one device holds the early layers and another holds the later ones.AI & LLMsPoolingDownsampling a feature map (usually taking the max or average over small regions) to shrink it and add invariance.AI & LLMsPositional EncodingHow a transformer is told the ORDER of tokens, since attention itself is order-blind.AI & LLMsPrecision and RecallTwo sides of accuracy: precision is how many of your hits were right; recall is how many of the right ones you caught.Data & RetrievalPrimary KeyThe column(s) that uniquely identify each row in a table — the anchor foreign keys point to.Systems & BackendPrinciple of Least PrivilegeGrant a component only the minimum access it actually needs to do its job — nothing more, so a compromise stays small.Systems & BackendPrompt CachingReusing the processed prefix of a prompt so you don’t re-pay to encode it every turn.AI & LLMsPrompt InjectionAn attack where malicious text in the input overrides the developer’s instructions.AI & LLMsPrompt LeakingTricking a model into revealing its own system prompt or hidden instructions that were never meant to be shown to the user.AI & LLMsPrompt TemplateA reusable prompt with placeholders you fill in per request — the unit of prompt reuse in an app.AI & LLMsProtocol BuffersA compact, schema-first binary serialization format from Google — the payload behind gRPC.Systems & BackendPruningRemoving unimportant weights or neurons from a trained model to shrink and speed it up with little accuracy loss.AI & LLMsPub/SubA messaging pattern where publishers broadcast events to topics and any number of subscribers receive them — sender and receiver never know each other.Systems & Backend
Q
QuantizationStoring model weights in fewer bits — smaller, faster, cheaper, with a small quality haircut.AI & LLMsQuery Fan-OutExpanding one query into many sub-queries issued in parallel, then merging the results.Data & RetrievalQuorum (N/R/W)Requiring a minimum number of replicas to agree on a read or write to tune consistency.Systems & Backend
R
RaftThe understandable consensus algorithm — how a cluster agrees on one log despite crashes.Systems & BackendRandom ForestAn ensemble of many decision trees trained on random data/feature subsets, averaged — accurate and hard to overfit.AI & LLMsRate LimitingCapping how many requests a client may make, protecting a service from overload and abuse.Systems & BackendRBAC (Role-Based Access Control)Granting permissions to roles, not individual users — a user gets access by being assigned a role, not by direct configuration.Systems & BackendRead ReplicaA read-only copy of a database that absorbs read traffic, scaling reads and easing load on the primary.Systems & BackendRecall@kOf all the relevant items, what fraction appear in the top-k results — the key metric for the retrieval stage of RAG.Data & RetrievalRed-TeamingDeliberately attacking your own model or system to find its failure modes before a real adversary does.AI & LLMsRegularizationTechniques that penalize complexity so a model generalizes instead of memorizing.AI & LLMsReinforcement LearningLearning by trial and error: an agent takes actions, gets rewards, and learns a policy that maximizes long-term reward.AI & LLMsReLUThe simplest activation function: pass positive values through, zero out negatives.AI & LLMsReplication LagThe delay between a write hitting the primary and showing up on a replica.Systems & BackendRequest CoalescingCollapse many identical in-flight requests into one — so a cache miss on a hot key hits the origin once, not thousands of times.Systems & BackendRerankingA second, smarter pass that reorders retrieved candidates before they reach the prompt.Data & RetrievalResidual ConnectionAdding a layer’s input directly to its output, so gradients have a direct path back through very deep networks.AI & LLMsRESTThe dominant style for web APIs: resources at URLs, manipulated with standard HTTP methods, stateless requests.Systems & BackendRetrieval-Augmented Generation (RAG)Fetching relevant documents and putting them in the prompt so the model answers from real sources.Data & RetrievalReverse ProxyA server in front of your backends that forwards client requests to them — the entry point for TLS, routing, caching, and load balancing.Systems & BackendRLHFTraining a model against a learned reward of human preferences — how raw LLMs become helpful assistants.AI & LLMsRNNRecurrent Neural Network: processes sequences one step at a time, carrying a hidden state — the pre-transformer sequence model.AI & LLMsROC-AUCA single number (0.5–1.0) summarizing how well a classifier ranks positives above negatives across all thresholds.AI & LLMsRolloutOne sampled attempt at a task, start to finish, used to compute a reward during RL training.AI & LLMsRoPE (Rotary Position Embedding)Encoding a token’s position by rotating its query/key vectors by an angle proportional to position, instead of adding a learned position vector.AI & LLMsRPCA style where calling a remote service looks like calling a local function.Systems & BackendRPO and RTOThe two DR targets: RPO = how much data you can lose; RTO = how long recovery may take.Systems & Backend
S
Saga PatternA distributed transaction as a chain of local ones — undo by compensation, not rollback.Systems & BackendScaling LawsThe empirical, roughly power-law relationship between a model’s size, its training data, its compute budget, and its loss.AI & LLMsSchema RegistryA central store of the schemas for messages in a streaming system, enforcing compatibility as producers and consumers evolve.Data & RetrievalSecrets ManagementStoring API keys, credentials, and certificates in a dedicated vault, outside source code, with rotation and audited access.Systems & BackendSelf-Supervised LearningLearning from unlabeled data by creating the labels from the data itself — predict a hidden part from the rest.AI & LLMsSemantic SearchSearch that matches on meaning via embeddings, not exact keywords.Data & RetrievalSerializable IsolationThe strictest isolation level: transactions behave as if run one at a time, ruling out all concurrency anomalies.Systems & BackendSerializationTurning an in-memory object into bytes to store or send it — and deserialization turns it back.Systems & BackendServer-Sent EventsA simple one-way stream: the server pushes a sequence of events to the browser over one long-lived HTTP connection.Systems & BackendServerlessRunning code without managing servers — the cloud provisions, scales, and bills per-execution, scaling to zero when idle.Systems & BackendService DiscoveryHow a service finds the current network address of another service, when instances are constantly starting, stopping, and moving.Systems & BackendService MeshAn infrastructure layer of sidecar proxies that handles service-to-service traffic — retries, mTLS, observability — without touching application code.Systems & BackendServices TrapWhen a product company’s deployment organisation drifts into consultancy economics: utilisation targets, bespoke builds, engagements that never end.Systems & BackendSessionServer-side state that remembers a logged-in user across requests, keyed by an ID stored in a cookie.Systems & BackendShardingSplitting data across many machines so no single node holds — or bottlenecks on — all of it.Systems & BackendSidecar PatternRunning a helper process alongside your main application container to handle a cross-cutting concern, independent of your app’s own code.Systems & BackendSigmoidThe classic S-shaped activation that squashes any number into (0,1) — used for probabilities and gates.AI & LLMsSLO / SLA / SLIThe reliability targets: an SLI measures it, an SLO is your goal for it, an SLA is the contract with penalties.Systems & BackendSnapshot IsolationEach transaction reads from a frozen snapshot of the database, so it sees a consistent world regardless of concurrent writes.Systems & BackendSOAPAn older, XML-based, contract-heavy protocol for web services — verbose but strictly standardized, still seen in enterprise/finance.Systems & BackendSoftmaxTurns a vector of raw scores (logits) into a probability distribution that sums to 1.AI & LLMsSOWThe contract document defining scope, deliverables and acceptance criteria for an engagement — including, often, your eval thresholds.Systems & BackendSparse RetrievalClassic keyword retrieval (like BM25) scoring documents by term overlap — sparse because most vocabulary entries are zero.Data & RetrievalSpec-Driven DevelopmentWriting an explicit specification an AI agent implements against, moving the human role to authoring the spec and reviewing the result.AI & LLMsSpeculative DecodingA small model drafts several tokens ahead; the big model verifies them in one pass — same output, faster.AI & LLMsSplit-BrainWhen a network partition makes two halves of a cluster each think they are in charge — risking divergent, conflicting writes.Systems & BackendSQL InjectionA classic attack: unescaped user input is treated as SQL code, letting an attacker read or destroy your database.Systems & BackendSSTableA Sorted String Table: an immutable, sorted-on-disk file of key-value pairs — the on-disk unit of LSM-tree stores.Systems & BackendStar SchemaA data-warehouse layout with one central fact table linked to descriptive dimension tables — simple and fast for analytics.Data & RetrievalStatelessnessA service that keeps no per-client state between requests, so any instance can handle any request — the key to easy scaling.Systems & BackendSticky SessionLoad-balancer routing that pins a user to the same backend instance — a workaround for stateful servers.Systems & BackendStop SequenceA string that tells the model to stop generating as soon as it produces it.AI & LLMsStream ProcessingComputing over data continuously as it arrives, event by event, rather than in scheduled batches.Data & RetrievalStrong ConsistencyEvery read reflects the most recent write — the opposite of eventual consistency, at a latency and availability cost.Systems & BackendStructured OutputsForcing model output to match a schema — the difference between text you parse and data you trust.AI & LLMsSupervised LearningLearning from labeled examples — the model sees inputs paired with correct answers and learns to map one to the other.AI & LLMsSVMSupport Vector Machine: finds the boundary that separates classes with the widest margin — strong for small, clean datasets.AI & LLMsSWE-benchThe benchmark behind every coding-agent claim: resolve real GitHub issues in real repos, graded by real tests.AI & LLMsSystem PromptThe standing instructions a model receives before any user input — identity, rules, output contract.AI & LLMs
T
t-SNEA technique for visualizing high-dimensional data in 2D/3D by preserving which points are neighbors.AI & LLMsTanhAn S-shaped activation like sigmoid but centered at zero, squashing inputs to (−1, 1).AI & LLMsTCPThe reliable, ordered, connection-based transport protocol most of the web runs on.Systems & BackendTemperatureThe decoding dial that flattens or sharpens the next-token distribution.AI & LLMsTensor ParallelismSplitting an individual weight matrix across GPUs so one layer’s math runs cooperatively on several devices at once.AI & LLMsTest-Time ComputeMaking a model smarter by letting it think longer at inference — the scaling axis behind o1/R1-class models.AI & LLMsThrottlingDeliberately slowing or capping a client’s request rate to protect a service — the enforcement side of rate limiting.Systems & BackendThundering HerdMany clients or workers waking up and retrying at the exact same moment, overwhelming the resource they’re all trying to reach.Systems & BackendTime Horizon (50%)The length of task an AI agent can complete at 50% reliability — a single trendable measure of capability.AI & LLMsTime to First TokenHow long until the first token of the reply appears — the latency number chat users actually feel.AI & LLMsTimeoutA cap on how long to wait for an operation before giving up — the most basic and most-forgotten resilience control.Systems & BackendTLSThe protocol that encrypts data in transit and authenticates the server — the S in HTTPS.Systems & BackendTokenThe unit a model actually reads and writes — a sub-word chunk, not a character or a word.AI & LLMsTokenizationSplitting text into tokens, usually with byte-pair encoding that merges frequent character pairs.AI & LLMsTombstoneA marker that records a deletion in a log-structured store, since you cannot erase in place — later removed by compaction.Systems & BackendTool CallingHow a model reaches beyond text: it emits a structured call and your code runs the function.AI & LLMsTop-p SamplingSampling from the smallest set of tokens whose probabilities sum to p — adaptive randomness control.AI & LLMsTransfer LearningReusing a model pretrained on a big general task as the starting point for a specific one — the foundation of modern AI.AI & LLMsTransformerThe neural-network architecture behind virtually every modern language model.AI & LLMsTwelve-Factor AppA set of 12 principles for building cloud-native apps that are portable, scalable, and disposable.Systems & BackendTwo-Phase CommitAll-or-nothing across systems: everyone votes, then everyone commits — at the price of blocking.Systems & Backend
U
UDPA lightweight, connectionless transport protocol that sends packets fast with no delivery guarantees.Systems & BackendUMAPA faster dimensionality-reduction method for visualization that preserves more global structure than t-SNE.AI & LLMsUnsupervised LearningFinding structure in unlabeled data — clustering, dimensionality reduction — with no correct answers to learn from.AI & LLMs
V
Vector ClockCounters that capture causality — decide whether two events are ordered or truly concurrent.Systems & BackendVector DatabaseA store built to index and search embeddings by nearest-neighbor similarity at scale.Data & RetrievalVerifierA program that scores whether an output is correct — the reward source in reinforcement learning from verifiable rewards.AI & LLMsVibe CodingDescribing what you want and accepting AI-written code without reading it closely — prototyping’s new superpower and production’s new risk.AI & LLMsVirtual MachineA software emulation of a full computer — its own OS and virtual hardware — running isolated on shared physical hardware.Systems & BackendVirtualizationRunning multiple isolated virtual machines on one physical server by abstracting the hardware — the foundation of cloud computing.Systems & BackendVPC DeploymentRunning your software inside the customer’s own cloud account and network, rather than in your SaaS.Systems & Backend
W
WebhookA reverse API: instead of you polling for events, the other service calls YOUR URL when something happens.Systems & BackendWebSocketA persistent two-way connection over one TCP socket, for real-time push both directions — unlike request/response HTTP.Systems & BackendWeight DecayA regularizer that shrinks weights toward zero each step, discouraging over-large parameters and overfitting.AI & LLMsWorld ModelA learned simulator of how an environment evolves — imagine outcomes before acting in the real world.AI & LLMsWrite SkewA subtle anomaly where two transactions read the same data, each writes something valid alone, but together break a rule.Systems & BackendWrite-Ahead LogAppend the change to a log before touching the data — the trick behind durability in nearly every database.Systems & BackendWrite-Through CacheWrite to the cache and the database together on every write, so the cache is always fresh — at a write-latency cost.Systems & Backend
X
Z
ZeRO / FSDPSharding optimizer state, gradients, and even parameters across GPUs instead of replicating them on every one.AI & LLMsZero-ShotGetting a model to do a task from instructions alone, with no examples in the prompt.AI & LLMsZero-Trust ArchitectureNever implicitly trust a request because of where it came from — verify identity and authorization on every single call.Systems & Backend
How this glossary works
A definition you can actually use, not just recite.
Plain English first
Every term opens with a one-sentence definition in words a beginner can follow — the idea before the jargon. No circular definitions that only make sense if you already knew the term.
A worked example and a gotcha
Most entries then show the concept with a concrete worked example and the mistake people actually make — the difference between recognizing a word and understanding what it does.
Linked to the real thing
Each term points to a hands-on lab, challenge, or handbook where the idea shows up, so you can go from the definition to doing it. Search by name, or filter by category to browse a whole area.