Part II - Tools for fully data-driven machine learning

Summary

In Sections 3.2 and 4.5 we have discussed how understanding of regression and classification datasets can be used to forge useful features in particular instances. With regression we saw that by visualizing low-dimensional data we could form excellent features for particular datasets like e.g., data from Galileo's classic ramp experiment. Later, when discussing classification, we also saw how basic features can be designed for e.g., image data using our understanding of natural signals and the mammalian visual processing system. Unfortunately, due to our general ignorance regarding most types of phenomena in the universe, instances such as these are rare and we often have no knowledge on which to construct reasonable features at all. However, we can, as described in the next three chapters, automate the process of feature design itself by leveraging what we know strong features should accomplish for regression/classification tasks.