15 - Predicting Performance in Large-Scale Identification Systems by Score Resampling

Summary

Introduction

With the wider deployment of biometric authentication systems and the increased number of enrolled persons in such systems, the problem of correctly predicting the performance has become important. The number of available testing samples is usually smaller than the number of enrolled persons that the biometric system is expected to handle. The accurate performance prediction allows system integrators to optimally select the biometric matchers for the system, as well as to properly set the decision thresholds.

Research in predicting the performance in large-scale biometric systems is still limited and mostly theoretical. Wayman (1999) introduced multiple operating scenarios for biometric systems and derived the equations for predicted performance assuming that the densities of genuine and impostor scores are known. Jarosz et al. (2005) presented an overview of possible performance estimation methods including extrapolation of large-scale performance given the performance on smaller-scale databases, binomial approximation of performance, and the application of extreme value theory. Bolle et al. (2005) derived the performance of identification systems (CMC curve) assuming that the performance of the corresponding biometric verification system (ROC curve) is known. The major assumption used in all these works is that the biometric match scores are independent and identically distributed, that is, genuine scores are randomly drawn from a genuine score distribution, and impostor scores are randomly and independently drawn from an impostor score distribution. As we will show in this chapter, this assumption does not generally hold, and using it leads to the underestimation of identification performance.