In this chapter we illustrate the relevance of stochastic resonance to auditory neural coding. This relates to natural auditory coding and also to coding by cochlear implant devices. Cochlear implants restore partial hearing to profoundly deaf people by trying to mimic, using direct electrical stimulation of the auditory nerve, the effect of acoustic stimulation.

Introduction

It is not surprising that the study of auditory neural coding involves stochastic phenomena, due to one simple fact – signal transduction in the ear is naturally a very noisy process. The noise arises from a number of sources but principally it is the Brownian motion of the stereocilia (hairs) of the inner hair cells that has the largest effect (Hudspeth 1989). Although the Brownian motion of the stereocilia appears small – typically causing displacements at the tips of the stereocilia of 2–3 nm (Denk and Webb 1992) – this is not small compared with the deflection of the tips at the threshold of hearing. Evidence exists that suggests, at threshold, the tip displacement is of the order of 0.3 nm (Sellick et al. 1982, Crawford and Fettiplace 1986), thus yielding a signal-to-noise ratio (SNR) of about –20 dB at threshold. Of course, under normal operating conditions the SNR will be greater than this, but, at the neural level, is typically of the order of 0 dB (DeWeese and Bialek 1995).

Given the level of noise and the fact that neurons are highly nonlinear makes the auditory system a prime candidate for observing stochastic resonance (SR) type behaviour.

Quantization of a signal or data source refers to the division or classification of that source into a discrete number of categories or states. It occurs, for example, when analogue electronic signals are converted into digital signals, or when data are binned into histograms. By definition, quantization is a lossy process, which compresses data into a more compact representation, so that the number of states in a quantizer's output is usually far fewer than the number of possible input values.

Most existing theory on the performance and design of quantization schemes specifies only deterministic rules governing how data are quantized. By contrast, stochastic quantization is a term intended to pertain to quantization where the rules governing the assignment of input values to output states are stochastic rather than deterministic. One form of stochastic quantization that has already been widely studied is a signal processing technique called dithering. However, the stochastic aspect of dithering is usually restricted so that it is equivalent to adding random noise to a signal prior to quantization. The term stochastic quantization is intended to be far more general, and applies to the situation where the rules of the quantization process are stochastic.

The inspiration for this study comes from a phenomenon known as stochastic resonance, which is said to occur when the presence of noise in a system provides a better performance than the absence of noise. Specifically, this book discusses a particular form of stochastic resonance – discovered by Stocks – known as suprathreshold stochastic resonance, and demonstrates how and why this effect is a form of stochastic quantization.

The initial research into suprathreshold stochastic resonance described in Chapter 4 considers the viewpoint of information transmission. As discussed briefly in Chapter 4, the suprathreshold stochastic resonance effect can also be modelled as stochastic quantization, and therefore results in nondeterministic lossy compression of a signal. The reason for this is that the effect of independently adding noise to a common signal before thresholding the result a number of times, with the same static threshold value, is equivalent to quantizing a signal with random thresholds. This observation leads naturally to measuring and describing the performance of suprathreshold stochastic resonance with standard quantization theory. In a context where a signal is to be reconstructed from its quantized version, this requires a reproduction value or reproduction point to be assigned to each possible state of a quantized signal. The quantizing operation is often known as the encoding of a signal, and the assignment of reproduction values as the decoding of a signal. This chapter examines various methods for decoding the output of the suprathreshold stochastic resonance model, and evaluates the performance of each technique as the input noise intensity and array size change. As it is the performance criterion most often used in conventional quantization theory, the measure used is the mean square error distortion between the original input signal and the decoded output signal.

Introduction

We begin this chapter by very briefly reviewing the SSR model introduced in Chapter 4. We then introduce the concept of decoding the output of a quantizer's encoding to reconstruct the input signal, and consider measuring the performance of such a reconstruction.

To conclude this book, we summarize our main results and conclusions, before briefly speculating on the most promising areas for future research.

Putting it all together

Stochastic resonance

Chapter 2 presents a historical review and elucidation of the major epochs in the history of stochastic resonance (SR) research, and discussion of the evolution of the term ‘stochastic resonance’. A list of the main controversies and debates associated with the field is given.

Chapter 2 also demonstrates qualitatively that SR can actually occur in a single threshold device, where the threshold is set to the signal mean. Although SR cannot occur in the conventional signal-to-noise ratio (SNR) measure in this situation, if ensemble averaging is allowed, then the presence of an optimal noise level can decrease distortion.

Furthermore, Chapter 2 contains a discussion and critique of the use of SNR measures to quantify SR, the debate about SNR gains due to SR, and the relationship between SNRs and information theory.

Suprathreshold stochastic resonance

Chapter 4 provides an up-to-date literature review of previous work on suprathreshold stochastic resonance (SSR). It also gives numerical results, showing SSR occurring for a number of matched and mixed signal and noise distributions not previously considered. A generic change of variable in the equations used to determine the mutual information through the SSR model is introduced. This change of variable results in a probability density function (PDF) that describes the average transfer function of the SSR model.

As described and illustrated in Chapters 4–7, a form of stochastic resonance called suprathreshold stochastic resonance can occur in a model system where more than one identical threshold device receives the same signal, but is subject to independent additive noise. In this chapter, we relax the constraint in this model that each threshold must have the same value, and aim to find the set of threshold values that either maximizes the mutual information, or minimizes the mean square error distortion, for a range of noise intensities. Such a task is a stochastic optimal quantization problem. For sufficiently large noise, we find that the optimal quantization is achieved when all thresholds have the same value. In other words, the suprathreshold stochastic resonance model provides an optimal quantization for small input signal-to-noise ratios.

Introduction

The previous four chapters consider a form of stochastic resonance, known as suprathreshold stochastic resonance (SSR), which occurs in an array of identical noisy threshold devices. The noise at the input to each threshold device is independent and additive, and this causes a randomization of effective threshold values, so that all thresholds have unique, but random, effective values. Chapter 4 discusses and extends Stocks' result (Stocks 2000c) that the mutual information between the SSR model's input and output signals is maximized for some nonzero value of noise intensity. Chapter 6 considers how to reconstruct an approximation of the input signal by decoding the SSR model's output signal.

Stochastic resonance (SR), being an interdisciplinary and evolving subject, has seen many debates. Indeed, the term SR itself has been difficult to comprehensively define to everyone's satisfaction. In this chapter we look at the problem of defining stochastic resonance, as well as exploring its history. Given that the bulk of this book is focused on suprathreshold stochastic resonance (SSR), we give particular emphasis to forms of stochastic resonance where thresholding of random signals occurs. An important example where thresholding occurs is in the generation of action potentials by spiking neurons. In addition, we outline and comment on some of the confusions and controversies surrounding stochastic resonance and what can be achieved by exploiting the effect. This chapter is intentionally qualitative. Illustrative examples of stochastic resonance in threshold systems are given, but fuller mathematical and numerical details are left for subsequent chapters.

Introducing stochastic resonance

Stochastic resonance, although a term originally used in a very specific context, is now broadly applied to describe any phenomenon where the presence of internal noise or external input noise in a nonlinear system provides a better system response to a certain input signal than in the absence of noise. The key term here is nonlinear. Stochastic resonance cannot occur in a linear system – linear in this sense means that the output of the system is a linear transformation of the input of the system. A wide variety of performance measures have been used – we shall discuss some of these later.

By definition, signal or data quantization schemes are noisy in that some information about a measurement or variable is lost in the process of quantization. Other systems are subject to stochastic forms of noise that interfere with the accurate recovery of a signal, or cause inaccuracies in measurements. However stochastic noise and quantization can both be incredibly useful in natural processes or engineered systems. As we saw in Chapter 2, one way in which noisy behaviour can be useful is through a phenomenon known as stochastic resonance (SR). In order to relate SR and signal quantization, this chapter provides a brief history of standard quantization theory. Such results and research have come mainly from the electronic engineering community, where quantization needs to be understood for the very important process of analogue-to-digital conversion – a fundamental requirement for the plethora of digital systems in the modern world.

Information and quantization theory

Analogue-to-digital conversion (ADC) is a fundamental stage in the electronic storage and transmission of information. This process involves obtaining samples of a signal, and their quantization to one of a finite number of levels.

According to the Australian Macquarie Dictionary, the definition of the word ‘quantize’ is

1. Physics: a. to restrict (a variable) to a discrete value rather than a set of continuous values. b. to assign (a discrete value), as a quantum, to the energy content or level of a system. 2. Electronics: to convert a continuous signal waveform into a waveform which can have only a finite number (usually two) of values.

This chapter discusses the behaviour of the mutual information and channel capacity in the suprathreshold stochastic resonance model as the number of threshold elements becomes large or approaches infinity. The results in Chapter 4 indicate that the mutual information and channel capacity might converge to simple expressions of N in the case of large N. The current chapter finds that accurate approximations do indeed exist in the large N limit. Using a relationship between mutual information and Fisher information, it is shown that capacity is achieved either (i) when the signal distribution is Jeffrey's prior, a distribution which is entirely dependent on the noise distribution, or (ii) when the noise distribution depends on the signal distribution via a cosine relationship. These results provide theoretical verification and justification for previous work in both computational neuroscience and electronics.

Introduction

Section 4.4 of Chapter 4 presents results for the mutual information and channel capacity through the suprathreshold stochastic resonance (SSR) model shown in Fig. 4.1. Recall that σ is the ratio of the noise standard deviation to the signal standard deviation. For the case of matched signal and noise distributions and a large number of threshold devices, N, the optimal value of σ – that is, the value of σ that maximizes the mutual information and achieves channel capacity – appears to asymptotically approach a constant value with increasing N. This indicates that analytical expressions might exist in the case of large N for the optimal noise intensity and channel capacity.

Chapters 1 to 6 laid down the fundamental physical and mathematical concepts pertaining to thermodynamics. While we always kept the discussion close to our atmosphere, it was not until Chapter 7 that, in a more applied mood, we presented how these concepts are applied to yield quantities useful to atmospheric processes. However, even if we understand all the mathematics, we still need an efficient way to present and visualize thermodynamic processes in the atmosphere. Thermodynamic diagrams can do that. Up to this point we made an effort to visualize the thermodynamic process using a (p, V) or a (p, T) diagram. However, such diagrams, while simple, may not be very convenient when it comes to their utilization.

Since the purpose of a diagram is efficiently and clearly to display processes and estimate thermodynamic quantities, the following are very desirable in a thermodynamic diagram: (1) for every cyclic process the area should be proportional to work done or energy (area-equivalent transformations), (2) lines should be straight (easy to use), and (3) the angle between adiabats and isotherms should be as large as possible (easy to distinguish). The (p, V) diagram satisfies the first condition (pda = dw), but the angle between isotherms and adiabats is not very large (Figure 4.5(a)). Because of this, while it is used for illustration purposes, this diagram is not used in practice.

Conditions for area-equivalent transformations

When we are constructing a new diagram, in effect, we go from the x = a, y = -p domain to a new domain characterized by two new coordinates, say u and w.

The fundamental equations derived in the previous chapter can only be applied to closed systems (i.e. to systems that do not exchange mass) that are homogeneous (i.e. they involve just one phase). In such cases we do not need to specify how thermodynamic functions depend on the composition of the system. Only two independent variables (T and p, or p and V, or p and T) need to be known. Since the total mass (m) remains constant, if we know the values of the extensive variables per unit mass we can extend the equations to any mass by multiplying by m or by n (the number of moles).

A heterogeneous system involves more than a single phase. In this case we are concerned with the conditions of internal equilibrium between the phases. Even if the heterogeneous system is assumed to be (as a whole) a closed system the phases constitute homogeneous but open “subsystems” which can exchange mass between them. In this case the fundamental equations must be modified to include extra terms to account for the mass exchanges. These extra terms involve a function μ called the chemical potential, μ = μ(T, p). We will not go into the details of defining μ; we will only accept that in the case of open systems something else must be included to account for the heterogeneity of the system. In this book we are concerned with a heterogeneous system that involves dry air (N2, O2, CO2, Ar) and water, with the water existing in vapor and possibly one of the condensed phases (water or ice).