Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

LABS · 06  /  CONVOLUTION

The Sliding Stencil.

Slide a tiny 3×3 stencil across an image. Everywhere it lands, it multiplies the pixels underneath by its weights and adds them up — one number, one output pixel. Change the stencil and the same image becomes edges, or blur, or sharpened.

THE GIST · 20 SECONDS

Convolution slides a small kernel (weight grid) over an image. At each spot it does a dot product: multiply each pixel by the overlapping weight, sum them, write one output pixel. The output is a feature map. Swap the kernel and the job changes — center-vs-surround weights find edges, equal weights blur. CNNs just learn these kernels.

  • Kernelthe 3×3 weight grid
  • Slidemove over every pixel
  • Multiply-sumdot product → 1 pixel
  • Feature mapthe filtered output
INPUT OUTPUT ✳

Pick a kernel on the right to transform the image instantly — or hit Play to watch the stencil slide and fill the output pixel by pixel.

UNDER THE HOOD

What you just played, written down

One operation — multiply the overlap and add it up — repeated at every pixel. Change the weights and the same machine becomes an edge finder, a blur, or a sharpener.

How convolution works — four moves

  1. Place the kernel. Center the 3×3 weight grid over an input pixel.
  2. Multiply the overlap. Multiply each pixel under the kernel by the weight sitting on top of it.
  3. Sum to one pixel. Add all nine products into a single number — that's one pixel of the output.
  4. Slide and repeat. Move one pixel over and do it again, across the whole image, to build the feature map.
WHY EDGES LIGHT UP

An edge kernel's weights sum to zero (big positive center, negative surround). Over a flat patch the parts cancel to near-zero (dark); at a boundary, the mismatch doesn't cancel, so the sum spikes (bright). Edges pop, smooth areas vanish.

The whole thing in code

def convolve(img, kernel):
    H, W = len(img), len(img[0])
    out = [[0] * W for _ in range(H)]
    for r in range(H):
        for c in range(W):
            s = 0.0
            for i in (-1, 0, 1):
                for j in (-1, 0, 1):
                    rr = min(H - 1, max(0, r + i))   # edge pad
                    cc = min(W - 1, max(0, c + j))
                    s += img[rr][cc] * kernel[i + 1][j + 1]
            out[r][c] = s                            # one pixel
    return out
EDGEsums to 0flat → dark, edge → bright
BLURsums to 1averages neighbors
CNNs LEARN THE KERNEL

Here you pick the weights by hand. A convolutional neural network learns them from data — early layers discover edge detectors on their own, later layers combine them into textures, parts, and whole objects.

⚠ Edges need padding

At the image border the 3×3 window hangs off the edge. Padding — zeros or replicated edge pixels — fills the gap so the output can stay the same size. Without it, the output loses the kernel's radius on every side: a 3×3 kernel turns 5×5 into 3×3.

↔ Stride & channels

Stride is how far the kernel jumps each step; a bigger stride downsamples. Real images have colour channels, so kernels are 3-D (e.g. 3×3×3) and a layer stacks many of them.

★ Where it's used

Photo filters, edge and blur in image editors, feature extraction, and every layer of a CNN for vision — the backbone of image classification and detection.

STEP BY STEP

Convolution, step by step

The four moves above, made exact: the formula for one output value, the arithmetic that sizes the output, a 5×5 patch worked by hand, and the knobs a real convolution layer exposes.

What a conv layer computes

  1. Size the output. Each side is floor((n + 2p − k) / s) + 1 for input size n, kernel k, padding p and stride s. A 5×5 input with a 3×3 kernel gives 3×3 at p = 0, stays 5×5 at p = 1, and drops to 3×3 again at p = 1, s = 2.
  2. Stack the channels. On a color image each kernel is 3×3×3. One output value is the sum of all 27 products plus a bias b.
  3. One kernel per output channel. A layer with C_out kernels writes C_out feature maps and holds k·k·C_in·C_out + C_out parameters: 73,856 for a 3×3 layer from 64 to 128 channels. The same weights are reused at every position, which is why a 3×3 kernel needs 9 weights where a dense map from a 32×32 image to a 32×32 output needs 1,048,576.
  4. No flip. Textbook convolution rotates the kernel 180° before sliding; deep-learning libraries skip the flip and compute cross-correlation, as the code above does. The symmetric edge-detect, blur and sharpen kernels give the same result either way; flipping Sobel-X negates every output.
# one output value of one kernel
y[r][c] = b + Σ_ch Σ_i Σ_j
    x[ch][r+i][c+j] · w[ch][i][j]
# output side length
out = floor((n + 2p − k) / s) + 1

Worked example: edge-detect on a 5×5 patch

  1. Input. Rows 1 and 2 are all 10. Rows 3 to 5 read 10, 10, 90, 90, 90: a bright 3×3 block in the bottom-right corner.
  2. Kernel. This page's edge-detect: 8 in the center, −1 on the eight neighbors, summing to 0. No padding, stride 1, so the output is 3×3.
  3. Top-left output. The window holds eight 10s and one 90: 8·10 − (7·10 + 90) = 80 − 160 = −80.
  4. Center output. The window sits on the block's corner pixel: 8·90 − (5·10 + 3·90) = 720 − 320 = 400.
  5. Full output. Rows read −80, −160, −240 / −160, 400, 240 / −240, 240, 0. The 0 sits wholly inside the flat bright block, the biggest values trace its corner and edges, and the dark pixels beside it go negative.
  6. Same patch, Sobel-X. Rows read 80, 80, 0 / 240, 240, 0 / 320, 320, 0: it fires only where brightness changes left to right, and its right column is 0 because that window is flat.

⚠ Outputs leave the input range

From inputs between 10 and 90 the edge kernel produced −240 and 400. Keep feature maps as signed floats; clip, take the absolute value, or rescale only for display. In 8-bit pixels a saturating cast clips −80 to 0, and a plain integer cast wraps it around to 176.

↔ Padding, stride, dilation

For odd k, p = (k − 1)/2 keeps the size. Zero padding darkens borders under a blur; replicate padding (this page) or reflect padding avoids that. Dilation 2 spreads a 3×3 kernel over a 5×5 area, and two stacked 3×3 layers also see 5×5 with 18 weights instead of 25.

★ In production

Libraries turn convolution into matrix multiplies (im2col) or Winograd transforms. Depthwise-separable layers split a 3×3, 128-to-128 convolution into a per-channel 3×3 plus a 1×1: 17,536 weights instead of 147,456, about 12%. A Gaussian blur runs as two 1-D passes, 10 multiplies per pixel for 5×5 instead of 25.

SOLVE IT YOURSELF

Solve it: one convolution step

This one is yours to write — in Python or TypeScript, running for real in your browser. Take a 3×3 patch and a 3×3 kernel and return the single output pixel. Edit the stub, hit Run (or ⌘/Ctrl + Enter), and watch the hidden tests. Stuck? the hints are below and Reveal solution is one click away.

YOUR TASK

Implement apply_kernel(patch, kernel): both are 3×3 grids (lists of lists). Multiply each patch value by the matching kernel weight and return the sum — the single output pixel for that window.

HINTS — 4 IDEAS
  1. Loop over rows i and columns j from 0 to 2.
  2. Multiply patch[i][j] by kernel[i][j] and add it to a running total.
  3. Return the total — that's the convolution output for this position.
  4. An edge kernel like [[-1,-1,-1],[-1,8,-1],[-1,-1,-1]] returns 0 on a flat patch.
CPython · WebAssembly
THE PICTURE

One output pixel, drawn out

10 10 10 10 10 10 10 10 10 10 10 10 90 90 90 10 10 90 90 90 10 10 90 90 90 −1 −1 −1 −1 8 −1 −1 −1 −1 −80 −160 −240 −160 400 240 −240 240 0 × input, 5×5edge kerneloutput, 3×3 centre: 8·90 − (5·10 + 3·90) = 720 − 320 = 400
The page’s worked patch under the edge kernel. The highlighted window sits on the corner of the bright block, so the eight −1 weights cover five 10s and three 90s while the 8 covers a 90. The positives win and the output spikes to 400. Slide the window one step at a time and the other eight outputs fill in the same way. The window fully inside the bright block cancels to exactly 0, because the kernel’s weights sum to 0.
WHAT IT COSTS

Cost and trade-offs of computing a convolution

Time counts multiply-adds. Space counts what has to sit in memory while they run.

CaseTimeSpaceWhy
Direct, one channelH·W·k² multiply-addsk² weights plus the outputThe four loops in the code above. A 3×3 kernel costs 9 multiply-adds per pixel; an 11×11 costs 121.
A conv layerH·W·k²·C_in·C_outk²·C_in·C_out + C_out parametersA 56×56 output, 3×3, 64 to 64 channels: 115,605,504 multiply-adds for only 36,928 parameters. Compute, not weights, is the bill.
Separable kernel, two 1-D passes2k per pixel instead of k²One intermediate imageA 7×7 drops from 49 to 14. Only for kernels that are an outer product of two vectors: box, Gaussian, Sobel. This page’s 8/−1 edge kernel is not one, so it cannot be split.
im2col + matrix multiplyThe same multiply-adds, as one big matrix productk²·C_in·H·W for the unrolled inputFor the layer above that is 1,806,336 floats, about 7.2 MB, against a 0.8 MB input. Fast because matrix libraries are heavily tuned; the price is a 9× memory blow-up.
FFT convolutionO(H·W·log(H·W)), whatever k isComplex buffers the size of the padded imagePays off for large kernels, like a wide blur. For a 3×3 the transforms cost more than the nine multiply-adds they replace.
Stride 2¼ of the output positions, so ¼ of the work¼ of the output memoryA downsampling choice, not a free speed-up: three of every four positions are never computed.

Read the table as a choice about where to pay. Direct convolution is lean on memory and slow for big kernels. im2col trades memory for speed. Separable and FFT tricks only apply to some kernels or sizes.

WATCH FOR

Common mistakes when writing a convolution

  • A blur that brightens the image

    An all-ones 3×3 kernel sums to 9, so a flat patch of 100 comes out as 900, and clips to 255 in an 8-bit image. A blur’s weights must sum to 1: divide by 9, or for a Gaussian, by the sum of its weights. The check is one line: sum(kernel) == 1 for a blur, 0 for an edge detector.

  • Writing the output into the input

    Filter in place and, by the time the window reaches row 2, the row above it already holds filtered values. Each output then mixes old and new pixels, and the result depends on the scan order. Always write to a separate buffer, as convolve() does with out. A separable blur needs its own buffer between the two passes for the same reason.

  • Swapping rows and columns

    Index with img[x][y] in one place and img[r][c] in another, and the kernel is applied transposed. Sobel-X becomes Sobel-Y. On the page’s patch the output reads 80, 240, 320 / 80, 240, 320 / 0, 0, 0 instead of 80, 80, 0 / 240, 240, 0 / 320, 320, 0. Symmetric kernels like blur hide the bug, so test with an asymmetric one.

  • Getting the output size wrong with stride

    A 224-pixel input, 3×3 kernel, stride 2 gives 112 with padding 1 but 111 with no padding, not the 112 that halving suggests. Hand-written code that assumes 112 reads past the end of the array, and a framework model fails its next shape check. Always compute floor((n + 2p − k) / s) + 1.

  • Expecting a deep stack to see the whole image

    Each stride-1 3×3 layer widens what one output can see by 2 pixels: 3 after one layer, 5 after two, 21 after ten. Ten such layers on a 224×224 image still give each output a 21×21 view. Networks grow that view quickly with strides and pooling, which is why depth alone does not buy global context.

QUICK CHECK

Did it stick?

See every answer, with the reasoning
  1. What single operation produces one output pixel in a convolution?

    Answer: Multiply each pixel by the overlapping kernel weight and sum them — Convolution is a dot product between the kernel and the image patch it covers: multiply each overlapping pair and add the nine products into one output pixel. (Max-pooling and median filters are different operations.)

  2. Why does an edge-detection kernel leave flat regions dark?

    Answer: Its weights sum to zero, so uniform neighbourhoods cancel out — Edge kernels have a positive center and negative surround that sum to zero. Over a flat patch the contributions cancel to near-zero (dark); only where neighbours differ — an edge — does the sum spike (bright).

  3. How is a CNN different from picking kernels by hand?

    Answer: It learns the kernel weights from data instead of hand-designing them — A convolutional neural network runs the same sliding multiply-and-sum, but the kernel weights are learned by training on labelled data — early layers end up as edge detectors, later layers build toward objects.

FAQ

Convolution, answered

What is image convolution?

Sliding a small kernel over an image; at each spot, multiply each pixel by the overlapping weight, sum the products into one output pixel, and repeat. The result is a filtered image (a feature map).

What is a kernel?

The small weight grid (usually 3×3) that defines the filter. Center-heavy-with-negative-surround detects edges; all-equal weights blur; a boosted center sharpens.

Why do edges light up?

An edge kernel's weights sum to zero. On a flat patch the positive and negative parts cancel (dark); at a boundary they don't, so the output spikes (bright).

How does this relate to CNNs?

A CNN uses convolution layers but learns the kernel weights from data instead of hand-picking them — early layers find edges, later ones build up to objects.

What are padding and stride?

Padding adds a border so edge pixels get a full window and the output keeps its size. Stride is the step size; a bigger stride skips pixels and downsamples.

RESULT

—

Next → stop hand-picking the weights and learn them from labelled images instead, stacking many layers — that's a convolutional neural network.

▶  Watch it explained

Convolution: how a neural net actually SEES an image

Finished this one? 0 / 61 Labs done
NextThe Gravity Wells: K-Means Next on the “Structure, and making new things” ladder · rung 2 of 6

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

More Labs