Handbooks / Encoder vs Decoder Models

AI~9 min readComparison

Head to Head

Encoder vs decoder: which way does attention flow?

Q: What is the difference between an encoder and a decoder model?

An encoder reads the entire input at once with bidirectional attention — every token can see every other token — and produces rich representations, so it is ideal for understanding tasks like classification, named-entity recognition and embeddings (BERT is the classic example). A decoder generates text one token at a time with causal attention — each token can only see the tokens before it — which is what makes it good at generation, chat and code (GPT-style models). Encoder-decoder models like T5 combine both.

Q: Is GPT an encoder or a decoder?

GPT-style models (GPT, Llama, Mistral, and most chat LLMs) are decoder-only. They use causal (left-to-right) attention and are trained to predict the next token, which is exactly what generation requires. BERT and its relatives are encoder-only, trained with masked language modeling to build representations rather than to generate.

Q: Why did decoder-only models take over?

Because generation, in-context (few-shot) learning and chat are all natural fits for next-token prediction, and decoder-only models scaled cleanly to enormous sizes. But encoders did not disappear: retrieval, semantic search, reranking and text classification are still dominated by encoder models, because those tasks want a representation of the whole input, not generated text.

Q: What are encoder-decoder models used for?

Encoder-decoder (sequence-to-sequence) models like T5 and BART fit tasks that transform one full sequence into another — translation and summarization are the classic cases. The encoder reads the whole source, and the decoder generates the target conditioned on it. Modern decoder-only LLMs now handle many of these tasks too, which is why pure encoder-decoder architectures are less common than they once were.

EncodervsDecoder

Both are made of the same Transformer blocks. The whole difference is one design choice — whether a token is allowed to look at the tokens after it. That single rule splits the family into models that understand (BERT) and models that generate (GPT), and decides what each is good for.

The one distinction that decides everything

An encoder uses bidirectional attention: every token can attend to every other token, before and after it, so the model reads the whole input at once and builds a representation of it. A decoder uses causal attention: a token can only attend to tokens before it, so the model generates left to right, one token at a time, never peeking at the future it hasn't written yet. Encoders understand; decoders generate.

→ The rule

If the task is to understand a fixed input — classify it, tag it, embed it, retrieve against it — an encoder's bidirectional view is the right tool. If the task is to produce text — chat, code, summaries, agent actions — a decoder's left-to-right generation is what you need.

Head to head

Dimension	Encoder (BERT-style)	Decoder (GPT-style)
Attention	Bidirectional — sees the whole input	Causal — sees only tokens before it
Training objective	Masked language modeling (fill in blanks)	Next-token prediction
Primary output	Representations / embeddings / labels	Generated text, token by token
Best at	Classification, NER, retrieval, reranking	Generation, chat, code, agents
Can it generate text?	Not naturally — it's built to understand	Yes — that's the whole point
In-context / few-shot learning	Weak	Strong — the basis of prompting
Typical examples	BERT, RoBERTa, DeBERTa, embedding models	GPT, Llama, Mistral, most chat LLMs
Inference cost pattern	One forward pass over the input	One forward pass per generated token

When to use each

Reach for an encoder

Text classification, sentiment, intent detection
Named-entity recognition and token tagging
Embeddings for semantic search and retrieval (the "R" in RAG)
Reranking retrieved candidates by relevance
Any task where you need a representation of a fixed input, not new text

Reach for a decoder

Chat assistants and open-ended generation
Code completion and code assistants
Summarization, rewriting, translation via prompting
Agents that plan and call tools
Anything few-shot: give examples in the prompt, get behavior

Why decoders took the spotlight — but encoders never left

The headline models of the LLM era are all decoder-only, and for good reason: next-token prediction is a perfect fit for generation and chat, it unlocked in-context (few-shot) learning, and decoder-only stacks scaled cleanly to enormous sizes. So when people say "LLM" today, they usually mean a decoder. But that's a spotlight effect, not the whole stage. Encoders quietly run some of the highest-volume workloads in production: every embedding for semantic search, every reranker, and a huge share of text classification is an encoder, because those tasks want a representation of the input, not a stream of generated tokens — and a bidirectional read is simply better at building one.

→ The cheap default

Building something that reads and sorts existing text (search, classify, route, tag)? An encoder is usually smaller, faster and better. Building something that writes text or takes actions? A decoder. Many real systems use both: an encoder to retrieve, a decoder to answer.

The encoder-decoder middle ground

There's a third design: encoder-decoder (sequence-to-sequence) models like T5 and BART. The encoder reads the whole source with bidirectional attention, and a separate decoder generates the target conditioned on that encoding. This shines for tasks that transform one full sequence into another — translation and summarization are the textbook cases, where you want a complete understanding of the source before you start writing the output.

So why isn't everything encoder-decoder? Because modern decoder-only LLMs turned out to handle translation and summarization very well by just conditioning on the source in the prompt, without a dedicated encoder — and a single decoder-only stack is simpler to scale and serve than a two-part architecture. Encoder-decoder models are still excellent at what they do; they're just less common now because a big decoder often does the same job with one architecture instead of two.

→ The trade you're actually making

Encoder-decoder buys a clean split between "understand the source" and "generate the target," which suits seq2seq tasks — at the cost of a more complex architecture that a large decoder-only model can often replace.

A worked scenario: a support-ticket classifier vs a support chatbot

You need to route incoming support tickets to the right team. That's an understanding problem: read the whole ticket, output one of a dozen labels. An encoder (a fine-tuned BERT-style model, or an embedding model plus a classifier) is the right call — it's small, fast, cheap to run at high volume, and its bidirectional read of the full ticket gives it every clue at once. Reaching for a giant generative LLM here is slower and more expensive for no accuracy gain.

Now you want a chatbot that actually answers the customer. That's a generation problem — and a decoder-only LLM is the tool, generating a fluent reply token by token, ideally grounded by retrieved docs. Notice the retrieval step in that sentence: the search that finds the right docs is almost certainly powered by an encoder embedding model. The mature system uses both — encoder to find, decoder to answer — which is exactly why "encoder vs decoder" is less a fight than a division of labor.

→ The pattern generalizes

Read-and-sort → encoder. Write-and-act → decoder. Transform-one-sequence-into-another → encoder-decoder (or a capable decoder). Real products often chain an encoder and a decoder together.

Frequently asked

Quick answers

What's the difference between an encoder and a decoder model?

An encoder reads the whole input at once with bidirectional attention and builds representations — ideal for classification, NER and embeddings (BERT). A decoder generates one token at a time with causal attention — ideal for generation, chat and code (GPT). Encoder-decoder models like T5 do both.

Is GPT an encoder or a decoder?

Decoder-only. GPT, Llama, Mistral and most chat LLMs use causal attention and next-token prediction. BERT and its relatives are encoder-only, trained with masked language modeling to build representations.

Why did decoder-only models take over?

Generation, few-shot learning and chat are natural fits for next-token prediction, and decoders scaled cleanly. But encoders still dominate retrieval, semantic search, reranking and classification, because those tasks want a representation of the input, not generated text.

What are encoder-decoder models used for?

Sequence-to-sequence tasks — translation and summarization are the classics. The encoder reads the whole source, the decoder generates the target. Modern decoder-only LLMs now handle many of these too, which is why pure encoder-decoder models are less common.

Finished this one? 0 / 139 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

LLM Engineering

More Handbooks

Cite this page

Reference it in your work, paper or an AI's context window.

APASingh, S. (2026). Encoder vs Decoder Models. Vibe Engines. https://vibeengines.com/handbook/encoder-vs-decoder-models

MLASingh, Saurabh. “Encoder vs Decoder Models.” Vibe Engines, 2026, vibeengines.com/handbook/encoder-vs-decoder-models.

BibTeX

@online{vibeengines-encoder-vs-decoder-models,
  author       = {Singh, Saurabh},
  title        = {Encoder vs Decoder Models},
  year         = {2026},
  organization = {Vibe Engines},
  url          = {https://vibeengines.com/handbook/encoder-vs-decoder-models}
}