7 - Benchmark programs

Summary

‘It is not simply how busy you are, but why you are busy. The bee is praised; the mosquito is swatted.’

Marie O'Conner

To measure the maximum speed of an automobile, it must be in motion. Similarly, a computer must be executing some sort of program when you attempt to measure any aspect of its performance. Since you are ultimately interested in how the computer performs on your application programs, the best program to run is, obviously, one of your own applications. Unfortunately, this is not always possible since a substantial amount of time and effort may be required to port your existing application to a new computer system. It will perhaps not be cost effective to port the application if the only goal is to measure the performance of the new system. Or, it may be that you are evaluating computer systems to determine which one is most appropriate for developing a completely new application. Since the application does not yet exist, it would be impossible to use it as your test program.

Owing to these practical and logistical difficulties in running your application program on the system or systems being evaluated, you instead are often forced to rely on making measurements while the computer system is executing some other program. This surrogate program is referred to as a benchmark program since it is used as a standard reference for comparing performance results. The hope is that this standardized benchmark program is in some way characteristic of the applications that you plan to execute on the machine you are evaluating.