10 - Finding Good Pattern Features

Summary

In this chapter we consider the problem of what features should be included when designing our classifier. We should make clear at the outset that this is an impossible problem; there may be no substitute for trying them all and seeing how well the resulting classifier works. However, this may be computationally impracticable, and unless a large test set is available it may be impossible to avoid selection effects, of choosing the best of a large class of classifiers on that particular test set and not for the population.

To illustrate the difficulty, consider a battery of diagnostic tests T₁, …, T_m for a fairly rare disease, which perhaps around 5% of all patients tested actually have. Suppose test T₁ correctly picks up 99% of the real cases and has a very low false positive rate. However, there is a rare special form of the disease that T₁ cannot detect, but T₂ can, yet T₂ is inaccurate on the normal disease form. If we test the diagnostic tests one at a time, we will never even think of including T₂, yet T₁ and T₂ together may give a nearly perfect classifier by declaring a patient diseased if T₁ is positive or T₁ is negative and T₂ is positive. This illustrates that considering features one at a time may not be sufficient.

Our aim in this chapter is to indicate single features which are likely to have good discriminatory power (feature selection) or linear combinations of features with the same aim (feature extraction).