Although written with the benefit of hindsight, there is no doubt that, by the end of 1924, there was a major crisis in the attempts to create a system of ‘quantum mechanics’ which could encompass all the features of atoms and their spectra. At the heart of the problem was the wave–particle duality first enunciated by Einstein in 1905 and reinforced by de Broglie's remarkable association of ‘matter-waves’ with electrons in 1924. As recorded by Jammer (1989),

The physics community was now faced with two theories of quantum phenomena which could scarcely have differed more radically from one another and yet both had achieved remarkable successes in explaining precisely the same physical phenomena – the spectral lines of the hydrogen atom, the zero point energy of quantum systems, the quantisation of the harmonic oscillator, the quantum rotator and the Stark effect. Furthermore, both theories could account for the experimental data, unlike the predictions of the old quantum theory. Perhaps these quite different approaches are not so surprising when it is appreciated that matrix and wave mechanics started from the diametrically opposite poles of the wave–particle duality.

At the heart of the Heisenberg approach was the fundamental role played by the non-commutative behaviour of the quantum variables and the quantisation of both the momentum and spatial variables. To accommodate these features, a new mathematical calculus had been invented from the realisation that matrices followed precisely the correct algebraic rules. The elaboration of this scheme led to the concept of the energy levels of a quantum system being associated with the diagonalisation of matrices using the eigenvalue procedure. As Jammer remarks, the theory

How this book came about

This book is the outcome of a long cherished ambition to write a follow-up to my book Theoretical Concepts in Physics (TCP2) (Longair, 2003). In that book, I took the story of the development of theoretical concepts in physics up to the discovery of quanta and the acceptance by the physics community that quanta and quantisation are essential features of the new physics of the early twentieth century. There was neither space nor scope to take that story further – it was just too complicated and would have required more advanced mathematics than I wished to include in that volume.

This book is my attempt to do for quantum mechanics what I did for classical physics and relativity in TCP2. The objective is to try to reconstruct as closely as possible the way in which quantum mechanics was created out of a mass of diverse experimental data and mathematical analyses through the period from about 1900 to 1930. In my view, quantisation and quanta are the greatest discoveries in the physics of the twentieth century. The phenomena of quantum mechanics have no direct impact upon our consciousness which to all intents and purposes is a world dominated by classical physics. But quantum mechanics underlies all the phenomena of matter and radiation and is the basis of essentially all aspects of civilisation in the twenty-first century.

Heisenberg in Göttingen, Copenhagen and Helgoland

Werner Heisenberg studied under Sommerfeld in Munich and was present at the Bohr Festspiele held in Göttingen in 1922 (Sect. 8.2). Although aged only 20, he challenged Bohr's support of Kramers' analysis of the quadratic Stark effect, having studied the paper in detail for Sommerfeld's seminar in Munich. The result was a long walk with Bohr during which they discussed this topic and the more general problems of quantum physics. This encounter made a strong impression on Heisenberg. Much later Heisenberg stated:

Heisenberg spent the winter of 1922–1923 working in Göttingen as Born's assistant. The astronomers had made great progress in the use of perturbations techniques within the action–angle formulation of classical dynamics to study the gravitational perturbations of planetary orbits.

The triumph of nineteenth century physics

The nineteenth century was an era of unprecedented advance in the understanding of the laws of physics. In mechanics and dynamics, more and more powerful mathematical tools had been developed to enable complex dynamical problems to be solved. In thermodynamics, the first and second laws were firmly established, through the efforts of Rudolf Clausius and William Thomson (Lord Kelvin), and the full ramifications of the concept of entropy for classical thermodynamics were being elaborated. James Clerk Maxwell had derived the equations of electromagnetism which were convincingly validated by Heinrich Hertz's experiments of 1887 to 1889. Light and electromagnetic waves were the same thing, thus providing a firm theoretical foundation for the wave theory of light which could account for virtually all the known phenomena of optics.

Sometimes the impression is given that experimental and theoretical physicists of the 1890s believed that the combination of thermodynamics, electromagnetism and classical mechanics could account for all known physical phenomena and that all that remained was to work out the consequences of these recently won achievements. As remarked by Brian Pippard in his survey of physics in 1900, Albert Michelson's famous remark that

has often been quoted out of context and is better viewed in the light of Maxwell's words in his inaugural lecture as the first Cavendish Professor of Experimental Physics in 1871:

Bohr's success in accounting for the frequencies observed in the spectral series of hydrogen was rightly regarded as a triumph, despite the fact that it violated the classical laws of mechanics and electromagnetism. It could not account, however, for the spectra of helium and heavier elements. The Bohr model was the simplest possible model for the dynamics of a single electron in the electrostatic potential of a positively charged point nucleus, in that it involved only quantised circular orbits defined by a single quantum number n, what became known as the principal quantum number. At the 1911 Solvay Conference, before Bohr's announcement of his model for the hydrogen atom, Poincaré had raised the issue of how the quantisation conditions could be extended to systems of more than one degree of freedom. The problem was attacked by both Planck and Sommerfeld. Their approaches ended up being essentially the same, although expressed in somewhat different language. We will follow Sommerfeld's approach.

In 1891 Michelson had shown that the Hɑ and Hß lines of the Balmer series displayed very narrow splittings (Michelson, 1891, 1892). Although incompatible with Bohr's theory, the problem was set aside in the face of the other remarkable successes of the theory. Sommerfeld suspected that the explanation lay in the fact that Bohr's quantisation condition involved only a single degree of freedom. In his papers of 1915 and 1916, he extended the quantisation of the orbits of the electron to more than one degree of freedom and accounted for the splitting of the lines of the Balmer series once a special relativistic treatment of the model was adopted (Sommerfeld, 1915a,b, 1916a).

Göttingen and Copenhagen were the undoubted capitals of the new discipline of quantum mechanics. The expertise in experimental and mathematical physics and in pure mathematics made Göttingen the epicentre of the revolution which was taking place in the mathematical physics of quanta. Whilst this was to remain the case for the next few years, other actors soon appeared on the scene who were to contribute to Born's ‘tangle of interconnected alleys’. What was truly remarkable was how quickly the different approaches to the problems of quantum theory were developed and the rapid assimilation of all of them into a coherent and self-consistent theory of quantum mechanics. Whilst the theory itself was completed relatively quickly, the understanding of its physical content was to take many more years.

The new players on the scene included Paul Dirac at Cambridge, Erwin Schrödinger in Vienna and Norbert Wiener at the Massachusetts Institute of Technology. Each of them brought quite new approaches to the development of quantum theory – their innovations were to supersede the matrix mechanics of Born, Heisenberg and Jordan, but there can be no doubt that the success of that theory indicated clearly the route ahead. They were however to involve the introduction of new mathematical techniques into the description of quantum phenomena.

Dirac's approach to quantum mechanics

Paul Dirac was trained as an electrical engineer at Bristol University, but he had a very strong mathematical bent. He was a solitary character who was notoriously quiet and self-effacing. He simply worked things out on his own.

As expected when I started out on this project, this has proved to be a complex and, at times, difficult story. After all, what was involved was tearing up the foundations of classical physics, which had been extraordinarily successful in explaining the macroscopic world about us, and replacing it by something radically different and non-intuitive in terms of our everyday experience. But the effort involved has been more than repaid by the very much deeper appreciation I have gained of the extraordinary works of the pioneers of quantum mechanics, both the theorists and the experimenters. If the brilliant theoretical researches of Planck, Einstein, Bohr, Heisenberg, Born, Jordan, Schrödinger, Pauli, Dirac and many others form the central core of this story, it should be remembered that their researches were inspired by the equally brilliant achievements of experimental physics. Another huge bonus has been a deepened understanding of quantum mechanics itself – if only I had these insights more than 50 years ago when I first encountered the subject.

There is a great deal more that could be said. I must reiterate that I have presented a somewhat streamlined version of the story in order to ensure that there is some continuous pathway, however tortuous, to the way in which the new understandings came about. For a full appreciation of the complexity of the story and the numerous blind alleys and diversions which took place, there is no substitute for in-depth absorption in Mehra and Rechenberg's magisterial exposition of the history of quantum theory.

The discovery of the spin of the electron by Uhlenbeck and Goudsmit was a major advance in the understanding of physics at the atomic level. Its discovery coincided with the development of both matrix and wave mechanics and its incorporation into the scheme of quantum mechanics and statistics led to deeper understanding of the underlying structure of quantum mechanics. Almost immediately, Heisenberg and Jordan used the new scheme of matrix mechanics to derive the expression for the g-factor which Landé had derived empirically from a very close study of the anomalous Zeeman effect. An important consequence of these developments was that the different approaches of matrix and wave mechanics were brought together. In particular, the discovery of spin as a new quantum number suggested the possibility of understanding systems containing more than one electron. Heisenberg's analysis of the helium atom was to pave the way for the full incorporation of spin into quantum mechanics and quantum statistics.

Spin and the Landé g-factor

The story of the discovery of the spin of the electron by Uhlenbeck and Goudsmit (1925a) was told in Sect. 8.5. As discussed in that section, their discovery was based upon empirical studies of the regularities observed in the anomalous Zeeman effect, inspired by the intricate analyses of Landé. Although based originally upon the classical concept of a rotating electron, electron spin is a purely quantum mechanical property intrinsic to the electron.

Born's reaction

In his reminiscences, Born recounted his memories of these exciting days (Born, 1978):