next up previous contents
Next: Decision Tree Classification Up: Classification Previous: Classification   Contents

k-Nearest Neighbours Classification

The k-Nearest Neighbours classifier [43] (kNN) consists of the assignment of an unclassified vector using the closest $ k$ vectors found in the training set. In this classifier, the Euclidean distance is used. Due to the fact that kNN is based on distances between sample points in the feature space, features need to be normalized to avoid that some features are weighted more strongly than others. Hence, all features have been normalized to unit variance and zero mean. Moreover, kNN presents another inherent problem, which is the uniform weighting of features regardless their discriminant power. In order to solve this problem we have included a feature selection step which automatically selects the set of the most discriminant features. Here we have used the Sequential Forward Selection (SFS) algorithm [87], which is a widely known technique that selects a local optimum solution in a computationally attractive way. SFS starts by selecting the best single feature and, in an iterative process, subsequent features are selected one at a time which in combination with the already selected ones, maximizes an Euclidean distance based criterion function.


next up previous contents
Next: Decision Tree Classification Up: Classification Previous: Classification   Contents
Arnau Oliver 2008-06-17