4 - Traditional Machine Learning Evaluation

Summary

Chapter 4 reviews frequently used machine learning evaluation procedures. In particular, it presents popular evaluation metrics for binary and multi-class classification (e.g., accuracy, precision/recall, ROC analysis), regression analysis (e.g., mean squared error, root mean squared error, R-squared error), clustering (e.g., Davies–Bouldin Index). It then reviews popular resampling approaches (e.g.,holdout, cross-validation) and statistical tests (e.g., the t-test and the sign test). It concludes with an explanation of why it is important to go beyond these well-known methods in order to achieve reliable evaluation results in all cases.