k-Nearest Neighbors (k-NN)

k-Nearest Neighbors, or k-NN, is a simple supervised algorithm that compares a new data point with stored training examples. For classification, it assigns the class most common among the k closest neighbors; for regression, it combines their numerical values. The definition of distance, the value of k, feature scaling, and weighting all affect the result. k-NN requires little conventional training and can model irregular boundaries, but prediction becomes slower and more memory-intensive as the dataset grows. It also struggles when irrelevant features or many dimensions make distance less meaningful. Careful preprocessing and validation are therefore essential despite the method’s intuitive design.