Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

LABS · 10  /  PERCEPTRON

The Line- Drawer.

Rosenblatt's perceptron, 1958, was one neuron with one job: draw a straight line between two kinds of points. It learns the only way it knows how — every time it puts a point on the wrong side, it nudges the line and tries again.

THE GIST · 20 SECONDS

A perceptron scores a point with w·x + b and calls it class +1 if positive, −1 if negative — a straight-line boundary. It learns by fixing mistakes: for each misclassified point, w += lr·label·x (and bias too), tilting the line toward that point's side. Repeat until zero mistakes. If the classes are separable, it's guaranteed to converge.

  • Weightsthe line's tilt (w)
  • Biasthe line's offset (b)
  • Updatenudge on a mistake
  • Separablea line can split them

Hit Step to fix the next misclassified point — or Play and watch the line swing until every point is on its own side.

UNDER THE HOOD

What you just played, written down

Learning by correcting mistakes, one at a time. It's the whole idea of a trainable neuron, stripped to its bones — and the limit that pointed the way to deep networks.

How the perceptron thinks — four moves

  1. Score the point. Compute w·x + b — a weighted sum of the features plus a bias.
  2. Predict a side. Positive score → class +1; negative → −1. That threshold is the boundary line.
  3. On a mistake, nudge. If wrong, push the weights toward the point's true class: w += lr·y·x, and b += lr·y.
  4. Repeat to zero mistakes. Sweep the data again and again; each pass fixes errors until none remain.
WHY THE NUDGE WORKS

Adding lr·y·x tilts the weight vector toward a misclassified positive point (or away from a negative one), moving the boundary so that point ends up on the correct side. Do it for every mistake and the line settles where all points agree.

The whole thing in code

def train(points, labels, lr=0.1, epochs=100):
    w = [0.0, 0.0]; b = 0.0
    for _ in range(epochs):
        errors = 0
        for x, y in zip(points, labels):
            score = w[0]*x[0] + w[1]*x[1] + b
            pred = 1 if score >= 0 else -1
            if pred != y:                 # a mistake
                w[0] += lr * y * x[0]      # nudge the weights
                w[1] += lr * y * x[1]
                b   += lr * y
                errors += 1
        if errors == 0:                   # converged
            break
    return w, b
BOUNDARYw·x+b=0a straight line
CONVERGESif separableguaranteed, finite steps
THE XOR WALL

A single perceptron can only draw one straight line, so it can't solve XOR — no line separates its classes. That 1969 realization stalled neural nets, until stacking perceptrons into layers brought non-linear boundaries.

⚠ Only linear boundaries

One perceptron draws a single hyperplane. If the classes aren't linearly separable, it never converges — it just cycles. The cure is depth: a multi-layer network.

↔ Its descendants

Add a smooth activation and gradient descent and you get logistic regression; stack many and train with backprop and you get a deep network. The perceptron is the seed.

★ Why it matters

It's the single neuron every modern network is built from — weights, a bias, a boundary, and mistake-driven learning. Understand one and the rest is repetition.

STEP BY STEP

The perceptron, step by step

The learning rule as formulas, a four-point run worked by hand, the guarantee on how many mistakes it can make, and the numbers this page's widget reaches when you press Play.

The rule, with the formulas

  1. Fold the bias in. Append a constant 1 to each input, x̃ = (x₁, x₂, 1), and the bias becomes a third weight.
  2. Predict. Output +1 if w·x + b ≥ 0, otherwise −1.
  3. Update only on a mistake. If the prediction is wrong, add η·y·x to w and η·y to b. A correct point changes nothing.
  4. Stop after a clean pass. One full epoch with zero updates means every training point is on its side.
# predict
y_hat = +1 if w·x + b ≥ 0 else −1
# update, only when y_hat ≠ y
w = w + η·y·x
b = b + η·y
THE MISTAKE BOUND

Novikoff's theorem: if some unit-length weight vector gives every folded-in point a score y·w̃·x̃ of at least γ, and no point is longer than R, the perceptron makes at most (R/γ)² updates. In the example, R = √11 ≈ 3.32 and the best γ is 0.853, so at most 15 updates. It needed 3.

Worked example: four points, η = 1

  1. Data. A(2, 1) and B(1, 3) are +1; C(−2, 1) and D(1, −1) are −1. Start at w = (0, 0), b = 0. A score of exactly 0 counts as +1.
  2. Epoch 1, C. A and B score 0, correct. C also scores 0, so it is called +1: wrong. w = (0, 0) − (−2, 1) = (2, −1), b = −1.
  3. Epoch 1, D. D scores 2·1 + (−1)(−1) − 1 = 2: wrong again. w = (2, −1) − (1, −1) = (1, 0), b = −2.
  4. Epoch 2, B. A scores 0, correct. B scores 1 − 2 = −1: wrong. w = (1, 0) + (1, 3) = (2, 3), b = −1. C and D both score −2, correct.
  5. Epoch 3. Scores 6, 10, −2, −2: no mistakes. Three updates found the line 2x₁ + 3x₂ − 1 = 0.
  6. The widget's run. Its 20 points start with 7 on the wrong side of the vertical line w = (1, 0). With η = 0.15 it makes 8 updates, and epoch 3 is clean, at w ≈ (0.871, −0.519), b = 0.

⚠ η is nearly irrelevant

From zero weights every update is a multiple of η, so η = 0.1 gives exactly a tenth of the η = 1 weights, (0.2, 0.3) and −0.1 here: the same line after the same mistakes. It matters only when the weights start elsewhere, as in the widget.

↔ When no line exists

On non-separable data the weights never settle, so the last ones are arbitrary. Shuffle each epoch, cap the epochs, and keep either the best weights seen (the pocket algorithm) or the average of all of them (the averaged perceptron), which is far more stable.

★ In production

The averaged perceptron still trains fast linear models on sparse features and in one-pass online learning, and the structured perceptron tags sequences. When you need probabilities or a maximum margin, logistic regression or a linear SVM is the usual choice.

SOLVE IT YOURSELF

Solve it: one perceptron update

This one is yours to write — in Python or TypeScript, running for real in your browser. Given the weights, bias, a point, and its label, apply one perceptron update if it's misclassified. Edit the stub, hit Run (or ⌘/Ctrl + Enter), and watch the hidden tests. Stuck? the hints are below and Reveal solution is one click away.

YOUR TASK

Implement update(w, b, x, y, lr): compute the prediction (sign of w·x + b, +1 if ≥ 0 else −1). If it matches y, return (w, b) unchanged. Otherwise nudge: w += lr·y·x component-wise and b += lr·y, then return the new (w, b).

HINTS — 4 IDEAS
  1. Score = w[0]*x[0] + w[1]*x[1] + b; prediction is 1 if score >= 0 else -1.
  2. If prediction equals y, it's correct — return w, b as-is.
  3. Otherwise add lr*y*x[k] to each weight and lr*y to the bias.
  4. Return the updated weights and bias.
CPython · WebAssembly
THE PICTURE

The picture: three updates, three lines

1 2 final A B C D filled = +1, open = −1 updates, η = 1 1 · epoch 1, C wrong: w=(2,−1) b=−1 2 · epoch 1, D wrong: w=(1,0) b=−2 3 · epoch 2, B wrong: w=(2,3) b=−1 epoch 3: 0 mistakes, stop bound (R/γ)² allows 15, used 3
Each mistake moves the line. Fixing C gives line 1, which now puts D on the wrong side. Fixing D gives line 2, the vertical x₁ = 2, and A sits exactly on it (score 0, which counts as +1). Fixing B gives the final line, with every point on its own side. The rule never looks at the correct points, and it never looks back.
TRADE-OFFS

Cost and trade-offs

For n training points with d features. The perceptron is about as cheap as learning gets. What it gives up is any notion of the best line.

CaseTimeSpaceWhy
Predict one pointO(d)O(d) weightsOne dot product and a sign.
One training epochO(n·d)O(d)Every point is scored; only mistakes cost an extra O(d) update.
Reaching zero mistakes (separable)at most (R/γ)² updatesO(d)The bound does not depend on n or d. Here it allows 15 and the run used 3.
Non-separable data, capped at E epochsO(E·n·d)O(d)It never converges, so the cap is what stops it and the last weights are arbitrary.
Averaged perceptronO(n·d) per epochtwo weight vectorsKeep a running sum of w and predict with the average. Much steadier on noisy data.
Kernel perceptronO(m·d) per predictionO(m·d), m = mistakes so farStore the mistaken points instead of w. Curved boundaries, including XOR with a polynomial kernel, but every mistake makes prediction slower.

The trade against logistic regression or a linear SVM is the same at every row: those cost more per step but give you a probability or a maximum margin. The perceptron stops at the first line that works.

WATCH FOR

Common mistakes

  • Using 0/1 labels with the ±1 update

    The rule is w += η·y·x. With y = 0 that adds nothing, so a misclassified class-0 point is never fixed and training can loop forever. Map labels to −1 and +1 first.

  • Leaving out the bias

    Without b the line must pass through the origin. Put a +1 point at (3, 3) and a −1 point at (1, 1): every line through the origin gives them the same sign, so no weights can separate them. Fold in the constant 1, as the step-by-step section does.

  • Looping until zero mistakes on real data

    Real data is rarely separable, and then the perceptron cycles forever. Always cap the epochs, as the page’s train does with 100, and shuffle each epoch so the order does not decide the answer.

  • Feeding unscaled features

    R is the length of the longest input. One feature in the thousands makes R huge, the bound (R/γ)² explodes, and the constant 1 that carries the bias becomes negligible next to it. Standardise features first.

  • Treating the line it finds as the best line

    It stops at the first separating line. In the worked example, the final line’s worst-case margin (on unit-length weights) is 0.535; the best possible is 0.853. Its score is also not a probability. Use an SVM for margin, logistic regression for probabilities.

QUICK CHECK

Did it stick?

See every answer, with the reasoning
  1. How does a perceptron classify a point?

    Answer: By the sign of w·x + b — a straight-line boundary — A perceptron computes the weighted sum w·x + b and predicts +1 if it is non-negative, −1 otherwise. The set of points where w·x + b = 0 is the straight-line boundary between the classes.

  2. What does the perceptron do when it misclassifies a point?

    Answer: It nudges the weights toward that point’s correct class: w += lr·y·x — The learning rule adds lr·y·x to the weights (and lr·y to the bias) for each misclassified point, tilting the boundary so that point moves to the correct side. Repeat until no mistakes.

  3. When is a single perceptron guaranteed to converge?

    Answer: Only when the two classes are linearly separable — The perceptron convergence theorem guarantees a solution in finite steps only if a straight line can separate the classes. On non-separable data (like XOR) it never settles — which motivated multi-layer networks.

FAQ

Perceptron, answered

What is a perceptron?

The simplest neural network — a single neuron that scores an input with w·x + b and predicts one of two classes by the sign. Geometrically it draws a straight-line boundary between the classes.

What is the learning rule?

On each misclassified point it nudges the weights toward the correct class: w += lr·y·x and b += lr·y, repeating until no point is misclassified.

Does it always converge?

Only if the data is linearly separable — then the convergence theorem guarantees a solution in finite steps. Otherwise it cycles forever.

What is the XOR problem?

A dataset a single perceptron can't solve because no straight line separates its classes. It exposed the perceptron's limit; stacking neurons into layers fixes it.

How does it relate to modern nets?

It's the single neuron deep networks are built from. Add non-linear activations and backprop across many layers and you get today's deep learning.

RESULT

—

Next → one perceptron draws one line; stack them into layers with non-linear activations and train with backprop, and you can carve any boundary — that's a deep neural network.

Finished this one? 0 / 61 Labs done
NextThe Bend: Activation Functions Next on the “Training a network” ladder · rung 2 of 7

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

More Labs