4 - Feature selection

Summary

Introduction

Two primary techniques for dimension-reducing feature extraction are subspace projection and feature selection. Compared with subspace projection, feature selection has the following advantages.

• Owing to its simplicity, feature selection can quickly and effectively weed out a large fraction of the insignificant features and yield a manageable vector dimension for subsequent data analysis.

• Feature selection is appealing from the perspective of online processing. Processing speed and power consumption are usually the major concerns for online processing. Feature selection results in a reduced number of selected features. This means that fewer features will need to be acquired in the input acquisition phase, thus saving processing power in the raw-data acquisition phase. This saving does not apply to the subspace projection approach, for which all of the original features must be acquired first before proper linear combinations can be computed to generate principal components.

• There are application-specific roles of feature selection, which simply could not be replaced by subspace projection. For example, gene selection is vital to genomic study. It is known that a few genes are often responsible for the cause or cure of tumors while most of the remaining genes are just “housekeeping” genes that have no functional roles and should be discarded for the sake of performance. Moreover, if a specific subset of genes is specially effective for accurate prediction of certain tumors, those genes are also most likely to be the biomarkers for the design of effective drugs to cure the tumors.