10 - Map Quality Metrics

Summary

This research places great emphasis on practical experimentation and quantitative evaluation of the results. To do this it is essential to have a precise measure of map quality. It is then possible to tune the map-building algorithms or to evaluate an exploration strategy by monitoring the quality of the map as exploration progresses. This chapter examines the issue of measuring the quality of a robot's map.

As an introduction, Section 10.1 examines some of the properties which one would expect to find in a useful quality metric, illustrating with examples of quality measures used by other researchers. The properties are that:

1. metric must be clearly defined.

2. metric must be multi-valued.

3. metric must reflect the purpose of the map.

4. metric must balance coverage and detail.

5. metric must be applicable during the construction of the map.

Section 10.2 discusses the need for an ‘omniscient’ observer. Is it possible for the robot to determine the quality of its map independently or can quality only be judged by comparison with a perfect map held by an external observer? It concludes that some quality measures can indeed by created by the robot independently but that measures derived from an ideal map are the most useful for the current purpose.

Section 10.3 surveys previous research in map-building, documenting the types of metrics which have been used. No ready-made metric was found which could be used in the current research.