8 - Recent Developments

Summary

We reviewed the major components of the evaluation framework in the last chapters and described in detail the various techniques pertaining to each, together with their applications. The focus of this chapter is to complement this review by outlining various advancements that have been made relatively recently, but that have not yet become mainstream. We also look into some approaches aimed at addressing problems arising from the developments on the machine learning front in various application settings, such as ensemble classifiers.

Just as with the traditional developments in performance evaluation, recent attempts at improving as well as designing new performance metrics have led the way. These have resulted in both improvements to existing performance metrics, thereby claiming to ameliorate the issues with the current versions, and proposals for novel metrics aimed at addressing the areas of algorithm evaluation not satisfactorily addressed by current metrics. We discuss in brief some of these advancements in Section 8.1. In Section 8.2, we focus on the attempts at unifying these performance metrics as well as studying their interrelation in the form of both theoretical and experimental frameworks. A natural extension to such studies is the design of more general or broader measures of performance, either as a result of insights obtained from the theoretical framework or by combining existing metrics based on observations from the experimental frameworks. Such metric combinations for obtaining general measures are the focus of Section 8.3.