K-Nearest Neighbors Lab
Instance-based classification with interactive decision regions, neighbor inspection, and weighted voting.
K-NN does not learn global model parameters. It predicts using the labels of nearby training points in feature space.
For a query point \(x\), select the closest \(K\) samples and aggregate their labels by majority vote (or weighted vote).
$$\hat{y}(x)=\arg\max_c \sum_{i\in \mathcal{N}_K(x)} w_i\,\mathbf{1}(y_i=c)$$
- Small
K: highly flexible boundary, sensitive to local noise. - Large
K: smoother boundary, lower variance, potentially higher bias. - Weighted voting (
w=1/d) gives stronger influence to very close neighbors.
Because K-NN relies on distance, feature scaling is critical in real datasets. Standardization usually improves reliability.
Prediction cost grows with dataset size, since distances must be computed against many samples at inference time.
Use validation to choose K and evaluate robustness under noisy or overlapping class distributions.
- Load a demo distribution (vertical, XOR, concentric, overlap, random).
- Adjust
Kand toggle weighted voting to compare decision boundaries. - Enable Test Mode and click to inspect nearest neighbors and class probability.
- Increase region density to see finer boundary details, then lower it for faster rendering.
Click to add training samples. In Test Mode, click to classify a query point and inspect its nearest neighbors.