10 - Industrial-Strength Evaluation

Summary

In Chapter 10, the book turns to practical considerations. In particular, it surveys the software engineering discipline with its rigorous software testing methods, and asks how these techniques can be adapted to the subfield of machine learning. The adaptation is not straightforward, as machine learning algorithms behave in non-deterministic ways aggravated by data, algorithm, and platform imperfections. These issues are discussed and some of the steps taken to handle them are reviewed. The chapter then turns to the practice of online testing and addresses the ethics of machine learning deployment. The chapter concludes with a discussion of current industry practice along with suggestions on how to improve the safety of industrial deployment in the future.