no training at all: store everything, decide later

Every model built so far in this track does real work at training time — logistic regression fits weights, decision trees learn splits, SVMs find support vectors, naive Bayes computes priors and likelihoods — and then discards the raw training data, keeping only that compact summary. k-Nearest Neighbors (k-NN) is built the opposite way: "training" is just storing every training example, unchanged. All of the actual decision-making is deferred to prediction time, which is why this style is sometimes called lazy learning, in deliberate contrast to every "eager" model earlier in this track.

how k-NN predicts

To predict a new CI run, k-NN computes the distance from that run to every single stored historical run, finds the k closest ones, and predicts by majority vote among them (classification) or by averaging their outcomes (regression). A new run that closely resembles five historical runs — three of which failed — would be predicted "fail" with k=5, purely because it's geometrically close to mostly-failing neighbors, with no weights or splits or probability model involved anywhere.

picking k

k is yet another dial on the same bias/variance tradeoff from Overfitting vs Underfitting, expressed through a completely different mechanism than anything else in this track. k=1 predicts based on a single closest neighbor — maximally sensitive to noise, since one mislabeled or unusual training point can flip the prediction for anything near it (high variance). A very large k smooths predictions by blending in points that aren't really "local" anymore; taken to its extreme, k equal to the entire training set makes every single prediction the same — the overall majority-class rate, the exact baseline first introduced in The ML Pipeline — completely ignoring the query point's features (maximum bias). The useful range sits between those two extremes, found the same way as every other hyperparameter in this track: swept against a validation set.

distance-based, so scaling matters — a third time

This is the third and final time this track raises the same warning: k-means in Unsupervised Learning and the margin in Support Vector Machines both broke without scaled features, and k-NN's entire mechanism — literal distance between points — is exactly as dependent on it. An unscaled lines_changed would dominate every distance calculation over a 0/1 feature like touches_slow_suite, regardless of which one actually predicts failure better. The z-score standardization from Feature Engineering, fit on the training split only, is not optional here.

the cost of storing everything

k-NN inverts the usual cost tradeoff. Every parametric model in this track pays a real training cost up front and then predicts almost instantly, since prediction is just plugging numbers into a fixed, compact formula. k-NN's "training" is nearly free — just store the data — but every single prediction has to measure distance to every stored point, so prediction cost grows directly with how much training data exists. A CI gating system that needs an answer in milliseconds, backed by years of historical runs, is a genuinely poor fit for plain k-NN for exactly this reason — a cost that every other model in this track avoided by doing its work once, during training, instead of over and over at every single prediction.

the curse of dimensionality

k-NN's entire premise is that "close" points are meaningfully similar. That premise quietly breaks down as the number of features grows. In a high-dimensional feature space, the volume grows so fast that data points end up spread thin — and a well-known consequence is that the distance to a point's nearest neighbor and the distance to its farthest neighbor stop being meaningfully different from each other. Once every point is roughly equidistant from every other point, "nearest neighbor" stops encoding real similarity at all.
Concretely: k-NN on a handful of CI features (lines_changed, touches_slow_suite, hour_of_day) works fine. Naively one-hot encoding a high-cardinality feature like author into hundreds of extra columns — exactly the trap Feature Engineering warned about for a different reason — pushes the feature space into a regime where distances stop discriminating usefully, and k-NN's predictions degrade even though, in principle, more features should mean more information. The practical fix is the same tool introduced for a different purpose in Unsupervised Learning: reduce dimensionality with PCA before computing distances, rather than handing k-NN the full raw feature set.

practical notes

from sklearn.neighbors import KNeighborsClassifier| https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html | n_neighbors is k; always pair with a scaler in a Pipeline, exactly as with SVMs and k-means earlier in this track |'knn_sk1'
KNeighborsClassifier(weights='distance')| | weights closer neighbors more heavily than farther ones within the k found, rather than a flat majority vote -- often a small, cheap accuracy improvement |'knn_sk2'

where to go from here

This closes out ML Foundations — all 20 pages, from "what is machine learning" through the theory behind why models generalize at all. Every model in this track has been evaluated the same way: fit on training data, validated honestly, and understood well enough to know when it's the wrong tool for the job.
Deep Learning & PyTorch Engineering — where these same concepts (cost functions, gradient descent, regularization, overfitting) turn into trained, deployed neural networks in actual PyTorch code.
LLMs & Generative AI — the same foundations, scaled up and applied to language.
Time Series & Quant Trading — this track's ideas applied to sequential, non-i.i.d. data, with an eye toward trading.
What Is Machine Learning, Actually? — back to where this track started, worth another look with everything since then in hand.

reference

scikit-learn — Nearest Neighbors
Wikipedia — Curse of Dimensionality