Position accuracy refers to the precision with which an object can be localized in space. Spatial resolution, on the other hand, is a measure of the ability to distinguish two separated point-like objects from a single object. The diffraction limit implies that optical resolution is ultimately limited by the wavelength of light. Before the advent of near-field optics it was believed that the diffraction limit imposes a hard boundary and that physical laws strictly prohibit resolution significantly better than λ/2. It was found that this limit is not as strict as assumed and that various tricks allow us to access the evanescent modes of the spatial spectrum. In this chapter we analyze the diffraction limit and discuss the principles of different imaging modes with resolutions near or beyond the diffraction limit.

The point-spread function

The point-spread function is a measure of the resolving power of an optical system. The narrower the point-spread function the better the resolution will be. As the name implies, the point-spread function defines the spread of a point source. If we have a radiating point source then the image of that source will appear to have a finite size. This broadening is a direct consequence of spatial filtering. A point in space is characterized by a delta function that has an infinite spectrum of spatial frequencies kx, ky. On propagation from the source to the image, high-frequency components are filtered out.

In the history of science, the first applications of optical microscopes and telescopes to investigate nature mark the beginning of new eras. Galileo Galilei used a telescope to see for the first time craters and mountains on a celestial body, the Moon, and also discovered the four largest satellites of Jupiter. With this he opened the field of astronomy. Robert Hooke and Antony van Leeuwenhoek used early optical microscopes to observe certain features of plant tissue that were called “cells”, and to observe microscopic organisms, such as bacteria and protozoans, thus marking the beginning of biology. The newly developed instrumentation enabled the observation of fascinating phenomena not directly accessible to human senses. Naturally, the question was raised whether the observed structures not detectable within the range of normal vision should be accepted as reality at all. Today, we have accepted that, in modern physics, scientific proofs are verified by indirect measurements, and that the underlying laws have often been established on the basis of indirect observations. It seems that as modern science progresses it withholds more and more findings from our natural senses. In this context, the use of optical instrumentation excels among ways to study nature. This is due to the fact that because of our ability to perceive electromagnetic waves at optical frequencies our brain is used to the interpretation of phenomena associated with light, even if the structures that are observed are magnified thousandfold.

The interaction of light with nanometer-sized structures is at the core of nano-optics. It is obvious that as the particles become smaller and smaller the laws of quantum mechanics will become apparent in their interaction with light. In this limit, continuous scattering and absorption of light will be supplemented or replaced by resonant interactions if the photon energy hits the energy difference of discrete internal (electronic) energy levels. In atoms, molecules and nanoparticles, like semiconductor nanocrystals and other “quantum confined” systems, these resonances are found at optical frequencies. Due to the resonant character, the light–matter interaction can often be approximated by treating the quantum system as an effective two-level system, i.e. by considering only those two (electronic) levels whose difference in energy is close to the interacting photon energy ħω0.

In this chapter we consider single-quantum systems that are fixed in space, either by deposition to a surface or by being embedded into a solid matrix. The material to be covered should familiarize the reader with single-photon emitters and with concepts developed in the field of quantum optics. While various theoretical aspects related to the fields emitted by a quantum system have been discussed in Chapter 8, the current chapter focuses more on the nature of the quantum system itself. We adopt a rather practical perspective since more rigorous accounts can be found elsewhere (see e.g. [1–4]).

Light embraces the most fascinating spectrum of electromagnetic radiation. This is mainly due to the fact that the energy of light quanta (photons) lies in the energy range of electronic transitions in matter. This gives us the beauty of color and is the reason why our eyes adapted to sense the optical spectrum.

Light is also fascinating because it manifests itself in the forms of waves and particles. In no other range of the electromagnetic spectrum are we more confronted with the wave–particle duality than in the optical regime. While long wavelength radiation (radiofrequencies, microwaves) is well described by wave theory, short wavelength radiation (X-rays) exhibits mostly particle properties. The two worlds meet in the optical regime.

To describe optical radiation in nano-optics it is mostly sufficient to adopt the wave picture. This allows us to use classical field theory based on Maxwell's equations. Of course, in nano-optics the systems with which the light fields interact are small (single molecules, quantum dots), which necessitates a quantum description of the material properties. Thus, in most cases we can use the framework of semiclassical theory, which combines the classical picture of fields and the quantum picture of matter. However, occasionally, we have to go beyond the semiclassical description. For example the photons emitted by a quantum system can obey non-classical photon statistics in the form of photon-antibunching (no two photons arriving simultaneously).

The interaction of metals with electromagnetic radiation is largely dictated by the free conduction electrons in the metal. According to the simple Drude model, the free electrons oscillate 180° out of phase relative to the driving electric field. As a consequence, most metals possess a negative dielectric constant at optical frequencies which causes, for example, a very high reflectivity. Furthermore, at optical frequencies the metal's free electron gas can sustain surface and volume charge density oscillations, called plasmon polaritons or plasmons with distinct resonance frequencies. The existence of plasmons is characteristic of the interaction of metal nanostructures with light. Similar behavior cannot be simply reproduced in other spectral ranges using the scale invariance of Maxwell's equations since the material parameters change considerably with frequency. Specifically, this means that model experiments with, for instance, microwaves and correspondingly larger metal structures cannot replace experiments with metal nanostructures at optical frequencies. The surface charge density oscillations associated with surface plasmons at the interface between a metal and a dielectric can give rise to strongly enhanced optical near-fields which are spatially confined near the metal surface. Similarly, if the electron gas is confined in three dimensions, as in the case of a small subwavelength-scale particle, the overall displacement of the electrons with respect to the positively charged lattice leads to a restoring force, which in turn gives rise to specific particle-plasmon resonances depending on the geometry of the particle.

Introduction

In this chapter we will consider how light is used to carry and process information. To perform correlation pattern recognition optically, we begin with coherent light with the right beam characteristics. We impress a signal upon the beam with one spatial light modulator (SLM), alter the propagation of the beam with a second SLM so that information is preferentially passed through the system and gathered at a location in the output plane, and detect and identify any information that might have been on the input beam. Accordingly we shall examine coherent light sources, SLMs, noise, and detection methods. Polarization of light is a particular point of interest, since many of the extant SLMs operate by altering the polarization of the light. Light polarization is a proper subset of statistical optics, and for a more complete description we refer the reader to Goodman [77] and to O'Neill [78]. Diffraction phenomena are also examined in this chapter, as they are responsible for the Fourier transforming properties needed for an optical correlator.

A knowledge of the physics of light and light modulators can be used to place information onto a beam and to cause its propagation to convert the information to a usable (i.e., detectable) form. The light typically used in the correlators discussed in this book has idealized properties. It is almost monochromatic (i.e., of single wavelength), almost fully polarized (this concept will be explained in later sections), and often enters the processing system as almost planar waves.

Correlation involves two signals or images. A reference image is correlated with a test image (also called a scene) to detect and locate the reference image in the scene. Thus the correlator can be considered as a system with an input (the scene), a stored template or filter (derived from the reference image), and an output (correlation). As we will see in this chapter, such a system is linear in the sense that a new input that is a weighted sum of original inputs results in an output that is an identically weighted sum of the original outputs. Thus a correlator can take advantage of the many properties of linear systems. The most important property is that a linear, time-invariant system can be characterized in terms of its frequency response. We use this and other related properties for the synthesis and use of correlation filters with attractive features such as distortion-tolerance and discrimination. In this chapter, we provide a review of some of the useful properties of signals and linear systems.

Basic systems

Strictly speaking, the signal is denoted s(·), and s(x) is the value of s(·) when the argument value is x. We will occasionally require the strict notation, but usually there is no confusion from writing s(x) to mean “s(·) with x being used as a general value for the argument.” Figure 3.1 is a simple block diagram of a system.

Ever since VanderLugt's pioneering work [5] on the implementation of matched filters (MFs) by coherent optical processing, there has been considerable interest in using correlators for recognizing patterns in images. The MF is of course optimal for finding a given pattern in the presence of additive white noise, and, as we have shown in Chapter 5, yields the highest output SNR. In radar signal processing and digital communications, matched filters have been very successful in many applications. For image processing, perhaps the greatest appeal of correlation filtering lies in its ability to produce shift-invariant peaks (because correlation filters are just a special class of LSI filter) and the resultant processing simplicity since we can avoid the need for image segmentation and registration. Unfortunately, MFs are not adequate for practical pattern recognition since their response degrades rapidly when the patterns deviate from the reference [57]. Such pattern variations can be induced by scale changes, rotations or signature differences, all of which are common phenomena associated with the general pattern recognition problem.

One straightforward approach to this problem would be to apply a large number of MFs, each tuned to a particular variation. However, the enormous storage and processing requirements of this approach make it impractical. The alternative is to design robust correlation filters that can overcome the limitations of the MFs.

In Chapter 2, we discussed the basics of probability theory, which helps us to model the randomness in signals (called noise) and thus allows us to extract the desired signal from unwanted noise. An example source of noise is the thermal noise induced in the voltage across a resistor by the motion of electrons. Similarly, when light is incident on a photodetector, the number of electrons released is random (although the mean is proportional to the incident light intensity), and this randomness leads to noise or uncertainty in the signal. When a signal is corrupted by such random noise, it is often important to extract or restore the original signal from the noisy version; this is known as signal restoration. In other instances, our task is to classify the signal as the noisy version of one of a few possible signals. This task of detecting the signal class is known as detection (or classification) and it is one of the foci of this chapter. It is not surprising that detection theory has a bearing on pattern recognition. A generalization of the notion of detection is estimation, where we try to estimate a parameter (which can assume a value in an interval rather than in a discrete set) from a noisy signal. Estimation theory is also relevant in tasks such as evaluating a correlator; e.g., estimating the error rate from a classifier.

There are many daily pattern recognition tasks that humans routinely carry out without thinking twice. For example, we can recognize those that we know by looking at their face or hearing their voice. You can recognize the letters and words you are reading now because you have trained yourself to recognize English letters and words. We can understand what someone is saying even if it is slightly distorted (e.g., spoken too fast). However, human pattern recognition suffers from three main drawbacks: poor speed, difficulty in scaling, and inability to handle some recognition tasks. Not surprisingly, humans can't match machine speeds on pattern recognition tasks where good pattern recognition algorithms exist. Also, human pattern recognition ability gets overwhelmed if the number of classes to recognize becomes very large. Although humans have evolved to perform well on some recognition tasks such as face or voice recognition, except for a few trained experts, most humans cannot tell whose fingerprint they are looking at. Thus, there are many interesting pattern recognition tasks for which we need machines.

The field of machine learning or pattern recognition is rich with many elegant concepts and results. One set of pattern recognition methods that we feel has not been explained in sufficient detail is that of correlation filters. One reason why correlation filters have not been employed more for pattern recognition applications is that their use requires background in and familiarity with different disciplines such as linear systems, random processes, matrix/vector methods, statistical decision theory, pattern recognition, optical processing, and digital signal processing.

The preceding chapters have described the techniques for designing correlation filters and their underlying mathematical foundations. The objective of this chapter is to provide a better understanding of the end-to-end process of designing and applying correlation filters to solve a pattern recognition problem. To facilitate this process we discuss two examples. The first is geared towards the recognition of targets in synthetic aperture radar (SAR) imagery. For this purpose, we use the public MSTAR SAR data set [3]. Details of a sample pseudo-code needed to construct and apply the filters to this data set are also provided. The second example discusses applications of correlation filters for face verification. Face recognition is just one example of a growing research field called biometric recognition [92] which includes other biometrics such as fingerprint, iris, etc. Correlation filters should prove useful in all such image recognition endeavors.

Correlation filters can be used to recognize patterns in images generated by many different types of sensors. Once the sensed information is converted into image pixels, the correlation pattern recognition algorithms can be applied in a fairly universal fashion. Thus correlation filters can find uses in all areas of automation including industrial inspection, security, robot vision, space applications, and defense. For instance, systems have been developed for fingerprint recognition [93]. Although the choice of sensor depends on the phenomenology associated with the pattern recognition/machine vision problem of interest, the approach for designing correlation filters generally remains the same.

In this chapter we shall optimize statistical correlation pattern recognition under the constraints of being implemented optically. We shall also trace their genealogy and look at some of their predecessors.

Introduction

The objective of this chapter is to treat optical correlation pattern recognition (OCPR) by considering signals, correlation metrics, noise, and limited filter domains.

Digital correlation is computationally more flexible and less noisy than optical correlation. On the other hand, optical correlation can be much faster, and have less weight and volume and power consumption, etc., which motivates us to give it a try. The constraints are very different in the two processes; in some digital correlation filter designs we have seen the necessity of introducing constraints (such as that the filter should have unit energy). In optics we are thoroughly constrained already, without introducing any artificial constraints. Unfortunately, in contrast to the digital version, the form of the optical constraint does not usually provide the solution for an optimizing filter. In this chapter we will nevertheless see how to operate optimally within the limitations imposed by optical practicalities.

The objective of pattern recognition is to recognize the presence of the reference object in the input signal or scene. Optical correlation aims to make'a comparatively bright spot of light that is detectable against a notably dimmer background when the desired object is present in the input image. We shall work with various criterion functions that measure the optical distinctness.

The basic concept of correlation is illustrated in Figure 1.4 with the help of a simple character recognition example. In this figure, black pixels take on a value of 1 and white pixels take on a value of 0. Suppose we are trying to locate all occurrences of the reference or target image (C in this example) in the test image (also called the input scene). One way to achieve this is to cross-correlate the target image with the input scene. The target image is placed in the upper left corner of the input scene and pixel-wise multiplication is carried out between the two arrays; all of the values in the resultant product array are summed to produce one correlation output value. This process is repeated by shifting the target image by various shifts to the right and down, thus producing a two-dimensional (2-D) output array called the correlation output. Ideally, this correlation output would have two large values corresponding to the two “C” letters in the input scene and zeros for other letters. Thus, large cross-correlation values indicate the presence and location of the character we are looking for. However, this will not always be achievable because some other letters may have high cross-correlation. For example, letter “C” and letter “O” have large cross-correlation. One of the goals of this book is to develop methods that preserve large cross-correlation with desired targets, while suppressing cross-correlation with undesired images (sometimes called the clutter), and reducing sensitivity to noise and distortions such as rotations, scale changes, etc.

Correlation filter theory relies heavily on concepts and tools from the fields of linear algebra and probability theory. Matrices and vectors provide succinct ways of expressing operations on discrete (i.e., pixelated) images, manipulating multiple variables and optimizing criteria that depend on multiple parameters. A vector representation also facilitates parallel operations on a set of constants or variables. Thus linear algebra provides powerful tools for digitally synthesizing and analyzing correlation filters.

If the world of interest contained only deterministic (i.e., non-random) signals and images, there would be no need for advanced pattern recognition methods in general and for correlation techniques in particular. In practice, reference images suffer from unpredictable disturbances such as noise, occlusion, illumination changes, rotation, and scale changes. Such unpredictability leads to randomness that can be characterized only by probabilistic models. We also need to understand what happens to such input randomness as it passes through signal processing systems such as correlation filters. Such knowledge will enable us to analyze the response of signal/image processing systems to noisy inputs, and to design systems that will preserve or enhance the desired signals while suppressing unwanted noise. This chapter aims to provide a quick review of the basics of matrix/vector techniques as well as the basics of probability theory and RVs.

While a comprehensive coverage of these fields is beyond the scope of this book, some topics relevant to correlation methods are discussed here for ease of reference.