Instance-Based Learning: k-NN and the Curse of Dimensionality
Every model in this track has been building something. This one doesn't build anything at all — it just remembers.
Advanced
k closest ones, and predicts by majority vote among
them (classification) or by averaging their outcomes (regression). A new run that closely
resembles five historical runs — three of which failed — would be predicted "fail" with
k=5, purely because it's geometrically close to mostly-failing neighbors, with no
weights or splits or probability model involved anywhere.
k is yet another dial on the same bias/variance tradeoff from
Overfitting vs Underfitting, expressed through a
completely different mechanism than anything else in this track. k=1 predicts
based on a single closest neighbor — maximally sensitive to noise, since one mislabeled or
unusual training point can flip the prediction for anything near it (high variance). A very
large k smooths predictions by blending in points that aren't really "local"
anymore; taken to its extreme, k equal to the entire training set makes every
single prediction the same — the overall majority-class rate, the exact baseline first
introduced in The ML Pipeline — completely ignoring the query
point's features (maximum bias). The useful range sits between those two extremes, found the
same way as every other hyperparameter in this track: swept against a validation set.
lines_changed would dominate every distance
calculation over a 0/1 feature like touches_slow_suite, regardless of which one
actually predicts failure better. The z-score standardization from
Feature Engineering, fit on the training split only, is
not optional here.
lines_changed,
touches_slow_suite, hour_of_day) works fine. Naively one-hot
encoding a high-cardinality feature like author into hundreds of extra columns —
exactly the trap Feature Engineering warned about for a
different reason — pushes the feature space into a regime where distances stop discriminating
usefully, and k-NN's predictions degrade even though, in principle, more features should mean
more information. The practical fix is the same tool introduced for a different purpose in
Unsupervised Learning: reduce dimensionality with PCA
before computing distances, rather than handing k-NN the full raw feature set.