15 - From Classification to Full Object Interpretation

Summary

Introduction

In current classification schemes, the goal is usually to identify category instances in an image, together with their corresponding image locations. However, object recognition goes beyond top-level category labeling: when we see a known object, we not only recognize the complete object, but also identify and localize its parts and subparts at multiple levels. Identifying and localizing parts, called “object interpretation,” is often necessary for interacting with visible objects in the surrounding environment.

In this chapter I will describe a method for obtaining detailed interpretation of the entire object, by identifying and localizing parts at multiple levels. The approach has two main components. The first is the creation of a hierarchical feature representation that is constructed from informative parts and subparts, that are identified during a learning stage. The second is the detection and localization of objects and parts using a two-pass algorithm that is applied to the feature hierarchy. The resulting scheme has two main advantages. First, the overall recognition performance is improved compared with similar nonhierarchical schemes. Second, and more important, the scheme obtains reliable detection and localization of object parts even when the parts are locally ambiguous and cannot be recognized reliably on their own.

The second part of the chapter will discuss a possible future direction for improving the performance obtained by current classification methods, by the use of a continuous online model update, in an attempt to narrow the so-called performance gap between computational methods and human performance.