k-Nearest Neighbors
also: KNN
Classify or predict a point by looking at the k closest labeled examples — no training, just distance.
k-Nearest Neighbors (k-NN) makes a prediction for a new point by finding the k most similar labeled examples (by distance) and taking their majority vote (classification) or average (regression). It is a lazy, non-parametric method — there is no training phase; all the work happens at query time.
Worked example: to classify a new flower, k-NN with k=5 finds the 5 nearest known flowers by petal measurements and assigns whichever species is most common among them. Gotcha: k-NN’s simplicity hides real costs — prediction is O(N) over the whole dataset (which is why approximate-nearest-neighbor indexes like HNSW exist for scale), it degrades in high dimensions (the curse of dimensionality makes all points nearly equidistant), and features must be scaled or the largest-range feature dominates the distance.