Introduction

Partial differential equations in complexity science

Partial differential equations (PDEs), that is to say equations relating partial derivatives of functions of more than one variable, are part of the bedrock of most quantitative disciplines and complexity science is no exception. They invariably arise when continuous fields are introduced into models. The classic example is the distribution of heat in a thermal conductor which typically varies continuously with time and with position. The continuous field in this case is T(x,t) the temperature at position x and time t which, in this case, satisfies a linear PDE called the diffusion equation.

In complexity science, PDEs often result from the process of coarse-graining whereby microscopically discrete processes are averaged over small scales to produce an effective continuous description of larger scales. Since much of complexity science focuses on the emergent properties of such coarse-grained descriptions, the analysis of PDEs is a key part of our complexity science toolkit. Some examples of coarse-graining as applied to interacting particle systems were discussed in Chapter 3. Another well-known example is the effective description of traffic flow using a coarse-grained fluid description described by a PDE known as Burgers' equation and variants of it, cf. Chapter 4. We will revisit this application in more detail later. Real fluids also provide a wealth of examples of complex behaviour, all described by the well-known Navier-Stokes equations or variants of them which are famous for being among the most mathematically intractable PDEs of classical physics.

Abstract

Dynamical systems are represented by mathematical models that describe different phenomena whose state (or instantaneous description) changes over time. Examples are mechanics in physics, population dynamics in biology and chemical kinetics in chemistry. One basic goal of the mathematical theory of dynamical systems is to determine or characterise the long-term behaviour of the system using methods of bifurcation theory for analysing differential equations and iterated mappings. Interestingly, some simple deterministic nonlinear dynamical systems and even piecewise linear systems can exhibit completely unpredictable behaviour, which might seem to be random. This behaviour of systems is known as deterministic chaos.

This chapter aims to introduce some of the techniques used in the modern theory of dynamical systems and the concepts of chaos and strange attractors, and to illustrate a range of applications to problems in the physical, biological and engineering sciences. The material covered includes differential (continuous-time) and difference (discrete-time) equations, first- and higher order linear and nonlinear systems, bifurcation analysis, nonlinear oscillations, perturbation methods, chaotic dynamics, fractal dimensions, and local and global bifurcation.

Readers are expected to know calculus and linear algebra and be familiar with the general concept of differential equations.

Abstract

Many examples exist of systems made of a large number of comparatively simple elementary constituents which exhibit interesting and surprising collective emergent behaviours. They are encountered in a variety of disciplines ranging from physics to biology and, of course, economics and social sciences. We all experience, for instance, the variety of complex behaviours emerging in social groups. In a similar sense, in biology, the whole spectrum of activities of higher organisms results from the interactions of their cells and, at a different scale, the behaviour of cells from the interactions of their genes and molecular components. Those, in turn, are formed, as all the incredible variety of natural systems, from the spontaneous assembling, in large numbers, of just a few kinds of elementary particles (e.g., protons, electrons).

To stress the contrast between the comparative simplicity of constituents and the complexity of their spontaneous collective behaviour, these systems are sometimes referred to as “complex systems”. They involve a number of interacting elements, often exposed to the effects of chance, so the hypothesis has emerged that their behaviour might be understood, and predicted, in a statistical sense. Such a perspective has been exploited in statistical physics, as much as the later idea of “universality”. That is the discovery that general mathematical laws might govern the collective behaviour of seemingly different systems, irrespective of the minute details of their components, as we look at them at different scales, like in Chinese boxes.

Abstract

Interacting particle systems (IPS) are probabilistic mathematical models of complex phenomena involving a large number of interrelated components. There are numerous examples within all areas of natural and social sciences, such as traffic flow on motorways or communication networks, opinion dynamics, spread of epidemics or fires, genetic evolution, reaction diffusion systems, crystal surface growth, financial markets, etc. The central question is to understand and predict emergent behaviour on macroscopic scales, as a result of the microscopic dynamics and interactions of individual components. Qualitative changes in this behaviour depending on the system parameters are known as collective phenomena or phase transitions and are of particular interest.

In IPS the components are modelled as particles confined to a lattice or some discrete geometry. But applications are not limited to systems endowed with such a geometry, since continuous degrees of freedom can often be discretized without changing the main features. So depending on the specific case, the particles can represent cars on a motorway, molecules in ionic channels, or prices of asset orders in financial markets (see Chapter 6), to name just a few examples. In principle, such systems often evolve according to well-known laws, but in many cases microscopic details of motion are not fully accessible. Due to the large system size these influences on the dynamics can be approximated as effective random noise with a certain postulated distribution. The actual origin of the noise, which may be related to chaotic motion (see Chapter 2) or thermal interactions (see Chapter 4), is usually ignored.

Abstract

In this chapter we introduce statistical mechanics in a very general form, and explore how the tools of statistical mechanics can be used to describe complex systems.

To illustrate what statistical mechanics is, let us consider a physical system made of a number of interacting particles. When it is just a single particle in a given potential, it is an easy problem: one can write down the solution (even if one could not calculate everything in closed form). Having two particles is equally easy, as this so-called “two-body problem” can be reduced to two modified one-body problems (one for the centre of mass, other for the relative position). However, a dramatic change occurs when the number of particles is increased to three. The study of the three-body problem started with Newton, Lagrange, Laplace and many others, but the general form of the solution is still unknown. Even relatively recently, in 1993 a new type of periodic solution has been found, where three equal mass particles interacting gravitationally chase each other in a figure-of-eight shaped orbit. This and other systems where the degrees of freedom is low belongs to the subject of dynamical systems, and is discussed in detail in Chapter 2 of this volume. When the number of interacting particles increases to very large numbers, like 1023, which is typical for the number of atoms in a macroscopic object, surprisingly it gets simpler again, as long as we are interested only in aggregate quantities. This is the subject of statistical mechanics.

Complexity Science is the study of systems with many interdependent components.

There is an urgent global need from industry, commerce, research institutions, academia, government and public services for a new generation trained to understand how complex systems behave, how to live with them, to control them and to design them well. We see this in public service management, transport, public opinion, epidemics, riots, terrorism, weather and climate. Relevant technological developments include distributed computing, data management, process control, personalised medicine, disease management, environmental sensor swarms, complex materials and nanobiotechnology.

Stimulated by problems from such a wide range of scientific disciplines, it presents great challenges and opportunities for Mathematics. Mathematics is essential for a deep understanding of complex systems and how to quantify their behaviour, for conclusions of genuine value to end-users, because of its powers for description, abstraction, deduction and prediction.

A range of Complexity Science concepts unify the field across disciplines: dynamics and diffusion, interacting agents and networks, coherent structures, emergence and self-organisation, upscaling and model reduction, quantification of complexity, scaling and extreme events, probabilistic modelling and statistical inference, feedback and control, diversity, optimisation and evolution.

This volume presents coherent introductions to the mathematical treatment of some areas of Complexity Science. It is based on some of the lecture modules of the Warwick EPSRC Doctoral Training Centre in Complexity Science.

Symmetries and wave functions

An important difference between the classical and quantum perspectives is their different criteria of distinguishability. Identical particles are classically distinguishable when separated in phase space. On the other hand, identical particles are always quantum mechanically indistinguishable for the purpose of counting distinct microstates. But these concepts and these distinctions do not tell the whole story of how we count the microstates and determine the multiplicity of a quantized system.

There are actually two different ways of counting the accessible microstates of a quantized system of identical, and so indistinguishable, particles. While these two ways were discovered in the years 1924–1926 independently of Erwin Schrödinger’s (1887–1961) invention of wave mechanics in 1926, their most convincing explanation is in terms of particle wave functions. The following two paragraphs may be helpful to those familiar with the basic features of wave mechanics.

A system of identical particles has, as one might expect, a probability density that is symmetric under particle exchange, that is, the probability density is invariant under the exchange of two identical particles. But here wave mechanics surprises the classical physicist. A system wave function may either keep the same sign or change signs under particle exchange. In particular, a system wave function may be either symmetric or antisymmetric under particle exchange.

Thermodynamics and entropy

The existence of entropy follows inevitably from the first and second laws of thermodynamics. However, our purpose is not to reproduce this deduction, but rather to focus on the concept of entropy, its meaning and its applications. Entropy is a central concept for many reasons, but its chief function in thermodynamics is to quantify the irreversibility of a thermodynamic process. Each term in this phrase deserves elaboration. Here we define thermodynamics and process; in subsequent sections we take up irreversibility. We will also learn how entropy or, more precisely, differences in entropy tell us which processes of an isolated system are possible and which are not.

Thermodynamics is the science of macroscopic objects composed of many parts. The very size and complexity of thermodynamic systems allow us to describe them simply in terms of a mere handful of equilibrium or thermodynamic variables, for instance, pressure, volume, temperature, mass or mole number, internal energy, and, of course, entropy. Some of these variables are related to others via equations of state in ways that differently characterize different kinds of systems, whether gas, liquid, solid, or composed of magnetized parts.

Boltzmann and atoms

The thermodynamic view of a physical system is the “black box” view. We monitor the input and output of a black box and measure its superficial characteristics with the human-sized instruments available to us: pressure gauges, thermometers, and meter sticks. The laws of thermodynamics govern the relations among these measurements. For instance, the zeroth law of thermodynamics requires that two black boxes each in thermal equilibrium with a third are in thermal equilibrium with each other, the first law that the energy of an isolated black box can never change, and the second law that the entropy of an isolated black box can never decrease. According to these laws and these measurements each black box has an entropy function S(E,V, …) whose dependence on a small set of variables encapsulates all that can be known of the black box system.

But we are not satisfied with black boxes – especially when they work well! We want to look inside a black box and see what makes it work. Yet when we first look into the black box of a thermodynamic system we see even more thermodynamic systems. A building, for instance, is a thermodynamic system. But so also is each room in the building, each cabinet in each room, and each drawer in each cabinet. But actual thermodynamic systems cannot be subdivided indefinitely. At some point the concepts and methods of thermodynamics cease to apply. Eventually the subdivisions of a thermodynamic system cease to be smaller thermodynamic systems and instead become groups of atoms and molecules.

The mathematician John von Neumann once urged the information theorist Claude Shannon to assign the name entropy to the measure of uncertainty Shannon had been investigating. After all, a structurally identical measure with the name entropy had long been an element of statistical mechanics. Furthermore, “No one really knows what entropy really is, so in a debate you will always have the advantage.” Most of us love clever one-liners and allow each other to bend the truth in making them. But von Neumann was wrong about entropy. Many people have understood the concept of entropy since it was first discovered 150 years ago.

Actually, scientists have no choice but to understand entropy because the concept describes an important aspect of reality. We know how to calculate and how to measure the entropy of a physical system. We know how to use entropy to solve problems and to place limits on processes. We understand the role of entropy in thermodynamics and in statistical mechanics. We also understand the parallelism between the entropy of physics and chemistry and the entropy of information theory.

But von Neumann’s witticism contains a kernel of truth: entropy is difficult, if not impossible, to visualize. Consider that we are able to invest the concept of the energy of a rod of iron with meaning by imagining the rod broken into its smallest parts, atoms of iron, and comparing the energy of an iron atom to that of a macroscopic, massive object attached to a network of springs that model the interactions of the atom with its nearest neighbors. The object’s energy is then the sum of its kinetic and potential energies – types of energy that can be studied in elementary physics laboratories. Finally, the energy of the entire system is the sum of the energy of its parts.

Messages and message sources

Information technologies are as old as the first recorded messages, but not until the twentieth century did engineers and scientists begin to quantify something they called information. Yet the word information poorly describes the concept the first information theorists quantified. Of course specialists have every right to select words in common use and give them new meaning. Isaac Newton, for instance, defined force and work in ways useful in his theory of dynamics. But a well-chosen name is one whose special, technical meaning does not clash with its range of common meanings. Curiously, the information of information theory violates this commonsense rule.

Compare the opening phrase of Dickens’s A Tale of Two Cities: It was the best of times, it was the worst of times … with the sequence of 50 letters, spaces, and a comma: eon jhktsiwnsho d ri nwfnn ti losabt,tob euffr te … taken from the tenth position on the first 50 pages of the same book. To me the first is richly associative; the second means nothing. The first has meaning and form; the second does not. Yet these two phrases could be said to carry the same information content because they have the same source. Each is a sequence of 50 characters taken from English text.