4 - Three examples and several machines

Summary

A ship in harbor is safe – but that is not what ships are for.

John Shedd

Introduction

In this chapter we apply several learning machines to three datasets. These examples show some of the range, strengths, and pitfalls of the machines. The datasets include those connected to two published studies, concerning lupus and stroke; see König et al. (2007, 2008), Ward et al. (2006, 2009). We also study an artificially created file of cholesterol data. This data was generated using known cardiac health factors; see Note 1.

Regarding methods, we begin with two linear schemes, logistic regression and linear discriminant, and apply them to the cholesterol data. Then we apply four learning machine methods – k-nearest neighbor, boosting, Random Forests, and neural nets – with several variations, to all three datasets.

Our goal in this chapter is to see how these learning machines operate on real, or nearly real, data. We evaluate the machines using methods such as out-of-bag resampling, techniques which are described more fully later; see Chapter 10. Other methods also make brief appearances, simply as part of the standard learning machine toolkit.

It is not our intention to declare a single winner among these machine methods over all three datasets. Theory tells us that there is no hope of finding a single machine that is provably better than all others for all datasets: the “super-classifier” does not exist.