“This exposition has fulfilled its purpose when it shows the reader how a life's efforts are related to one another and why they have led to expectations of a particular kind.” Thus end Albert Einstein's “Autobiographical notes,” as we begin our account. Einstein's critical self-assessment was published in a collection of essays entitled Albert Einstein: Philosopher-Scientist, and it appeared in 1949, only six years before Einstein's final passage. These “Notes” constitute a rich source of personal reflections on his life and science; in particular, they discuss in detail the ideas that dominated his later years. The latter are the subject of this book, and it is with the above sense of purpose that we wish to approach it.

Thus, this study will be about Einstein's search for a unified theory for all physical forces, and its relation to his larger oeuvre in science and philosophy. Our story starts with the events leading up to the 1915 discovery of general relativity and its scope extends to Einstein's passing away in 1955.We have a vast subject, and it will be impossible to address minutely all its elements; both some familiar and some less familiar themes of Einstein's later physics will have to remain undiscussed. What we will present, rather, is an attempt, from a historical perspective, at a synthesis, and our selection will be aimed to serve that synthesis.

To complete the picture of Einstein's later thought, we end by turning briefly to the critique of quantum theory that he formulated in the same years as he was elaborating unified field theories. We intend to address this critique in light of the historical and methodological perspective that we have taken with regard to the unified field theory work. In this way, we hope to come to a more complete understanding of Einstein's position.

In short, one can say that Einstein's attitude to quantum theory was conditioned by a conviction that the theory was at best a mere phenomenological description of reality. We argue that this attitude was in part due to the methods employed in formulating quantum theory, as these deviated from his gradually developing unified field theory methodology. In addition, it found its expression in the epistemological objections that Einstein formulated, such as the argument contained in the EPR article.

The chapter is organized as follows. We first give a characterization of quantum research in terms of its methods and practices before quantum theory had reached its more complete formulations. The continuously evolving description of quantum phenomena prior to 1925 is generally referred to as the “old quantum theory,” and we will be touching on Einstein's role in its development.

Introduction

Quantum information theory is the study of how the peculiar features of quantum mechanics can be exploited for the purposes of information processing and transmission. A central theme of such a study is the ways in which quantum mechanics opens up possibilities that go beyond what can be achieved classically. This has in turn led to a renewed interest in, and a new perspective on, the differences between the classical and the quantum. Although much of the work along these lines has been motivated by quantum information theory – and some of it has been motivated by the conviction that quantum theory is essentially about possibilities of information processing and transmission – the results obtained, and the frameworks developed, have interest even for those of us who are not of that conviction. Indeed, much of the recent work echoes, and builds upon, work that predates the inception of quantum information theory. The significance of such work extends beyond the setting of quantum information theory; the work done on distinguishing the quantum from the classical in the context of frameworks that embrace both is something worthy of the attention of anyone interested in the foundational issues surrounding quantum theory.

One of the striking features of quantum mechanics lies in its probabilistic character. A quantum state yields, not a definite prediction of the outcome of an experiment, but a probability measure on the space of possible outcomes. Of course, probabilities occur also in a classical context.

We live, we are told, in an information age. We are told this, perhaps, less often than once we were; but no doubt only because the phrase has become worn from use. If ours is an age of information, then quantum information theory is a field propitiously in tune with the spirit of the times: a rich and sophisticated physical theory that seeks to tame quantum mysteries (no less!) and turn them to ingenious computational and communication ends. It is a theory that hints, moreover, at the possibility of finally rendering the quantum unmysterious; or at least this is a conclusion that many have been tempted to draw.

Yet, for all its timeliness, some of the most intriguing of the prospects that quantum information science presents are to be found intertwining with some surprisingly old and familiar philosophical themes. These themes are immaterialism and instrumentalism; and in this chapter we shall be exploring how these old ideas feature in the context of two of the most tantalizing new questions that have arisen with the advent of this field. Does quantum information theory finally help us to resolve the conceptual conundrums of quantum mechanics? And does the theory indicate a new way of thinking about the world – one in which the material as the fundamental subject matter of physical theory is seen to be replaced by the immaterial: information?

Introduction

Many philosophers and physicists have expressed great hope that quantum information theory will help us understand the nature of the quantum world. The general problem is that there is no widespread agreement on what quantum information is. Hence, such pronouncements regarding quantum information theory as the savior of the philosophy of physics are hard to evaluate. Much work has been done producing and evaluating concepts of information.

In I have defended and articulated the Schumacher concept of quantum information. Roughly speaking, quantum information is construed as the statistical behavior associated with the measurement of a quantum system. Hence it is a coarse-grained operational description of quantum systems, with no recourse to the fundamental ontological features of quantum systems responsible for such behavior. From this perspective, construing quantum mechanics as a theory of quantum information departs from the traditional interpretive endeavors of philosophers and physicists. The question is whether there is any motivation for taking such a view.

The theorem of Clifton, Bub, and Halvorson (CBH) provides just such a motivation. The theorem guarantees that, if a theory T satisfies certain conditions, there will exist an empirically equivalent C*-algebraic theory that has a concrete representation in Hilbert space, which it is notoriously difficult to interpret as a constructive or mechanical theory. In such a case, any underlying ontologies philosophers develop that are compatible with T will be undermined by the C*-algebraic equivalent. Bub suggests in light of this in-principle uncertainty regarding ontology that we re-conceive of quantum mechanics as a theory about quantum information.

Recently there has emerged an exciting and rapidly growing field of research known as quantum information theory. This interdisciplinary field is unified by the following two goals: first, the possibility of harnessing the principles and laws of quantum mechanics to aid in the acquisition, transmission, and processing of information; and second, the potential that these new technologies have for deepening our understanding of the foundations of quantum mechanics and computation. Many of the new technologies and discoveries emerging from quantum information theory are challenging the adequacy of our old concepts of entanglement, non-locality, and information. This research suggests that the time is ripe for a reconsideration of the foundations – and philosophical implications – of quantum information theory.

Historically, apart from a small group of physicists working on foundational issues, it was philosophers of physics who recognized the importance of the concepts of entanglement and non-locality long before the mainstream physics community. Prior to the 1980s, discussions of the infamous “EPR” paper and John Bell's seminal papers on quantum non-locality were carried out more often by such philosophers than by ordinary physicists. In the 1990s that situation rapidly changed, once the larger community of physicists had begun to realize that entanglement and non-locality were not just quirky features of quantum mechanics, but physical resources that could be harnessed for the performance of various practical tasks. Since then, a large body of literature has emerged in physics, revealing many new dimensions to our concepts of entanglement and non-locality, particularly in relation to information. Regrettably, however, only a few philosophers have followed these more recent developments, and many philosophical discussions still end with Bell's work.

Entanglement can be understood as an extraordinary degree of correlation between states of quantum systems – a correlation that cannot be given an explanation in terms of something like a common cause. Entanglement can occur between two or more quantum systems, and the most interesting case is when these correlations occur between systems that are space-like separated, meaning that changes made to one system are immediately correlated with changes in a distant system even though there is no time for a signal to travel between them. In this case one says that quantum entanglement leads to non-local correlations, or non-locality.

More precisely, entanglement can be defined in the following way. Consider two particles, A and B, whose (pure) states can be represented by the state vectors ψA and ψB. Instead of representing the state of each particle individually, one can represent the composite two-particle system by another wavefunction, ΨAB. If the two particles are unentangled, then the composite state is just the tensor product of the states of the components: ΨAB = ψA ⊗ ψB; the state is then said to be factorable (or separable). If the particles are entangled, however, then the state of the composite system cannot be written as such a product of a definite state for A and a definite state for B. This is how an entangled state is defined for pure states: a state is entangled if and only if it cannot be factored: ΨAB ≠ ψA ⊗ ψB.

Introduction

More than a century after its birth, quantum mechanics (QM) remains mysterious. We still don't have general principles from which to derive its remarkable mathematical framework, as happened for the amazing Lorentz transformations, which were rederived by Einstein from the invariance of physical laws in inertial frames and from the constancy of the speed of light.

Despite the utmost relevance of the problem of deriving QM from operational principles, research efforts in this direction have been sporadic. The deepest of the early attacks on the problem were the works of Birkhoff, von Neumann, Jordan, and Wigner, attempting to derive QM from a set of axioms with as much physical significance as possible. The general idea in Ref. is to regard QM as a new kind of prepositional calculus, a proposal that spawned the research line of quantum logic, which is based on von Neumann's observation that the two-valued observables – represented in his formulation of QM by orthogonal projection operators – constitute a kind of “logic” of experimental propositions. After a hiatus of two decades of neglect, interest in quantum logic was revived by Varadarajan, and most notably by Mackey, who axiomatized QM within an operational framework, with the single exception of an admittedly ad hoc postulate, which represents the propositional calculus mathematically in the form of an orthomodular lattice. The most significant extension of Mackey's work is the general representation theorem of Piron.

We turn our attention again to Einstein's attempts to unify gravity and electromagnetism; in particular, we intend to address various aspects of his study of the Kaluza–Klein theory. Einstein studied this theory extensively on a number of occasions. His deepest involvement was in the late 1930s and early 1940s, producing two substantial papers. By this time Einstein had become firmly settled at the Institute for Advanced Study, where his Kaluza–Klein collaborators, Peter Bergmann and Valentin Bargmann, were employed as his assistants – again a short introduction to both of them is contained in the boxed texts of this chapter.

Particle solutions in field theory

Einstein was quite productive in the years between the semivector and these later Kaluza–Klein publications. In 1935, the “Einstein–Podolsky–Rosen” paper on the completeness of quantum mechanics appeared, and in 1938 he co-authored another important article on “the problem of motion” – the relation between the geodesic equation for test particles and general relativity's field equations; he collaborated with Leopold Infeld and Banesh Hoffmann on this subject. Einstein published a study on gravitational lensing in 1936 and the same year he and Nathan Rosen submitted an article to the Physical Review that called into question the existence of gravitational waves.

Einstein was much engaged with philosophy. His publications on epistemological issues are numerous and some have proven to be important on their own account. Like many of the leading physicists that were his contemporaries – men such as Erwin Schrödinger and Max Planck – he saw engagement with the philosophy of science as part of the intellectual project of physics. Einstein was deeply involved with questions concerning, for example, how scientific creativity works and how abstract thought relates to the actual world.

In Einstein's case it is impossible to disentangle the philosopher from the physicist. Throughout his work there is an evident overlap of the two, as well as a clear interaction between them. As already suggested in the previous chapter, Einstein's physics determined his philosophical outlook, and his philosophy inspired the directions he took in physics. Thus, in striving for a coherent understanding of Einstein's later oeuvre, one needs to address the exchanges between his practices in theoretical physics and his expressed philosophical beliefs – even if the latter were not intended to constitute an elaborated and consistent system.

In this chapter we turn in more detail to Einstein's declared methodological positions. The most important source for these is his 10 June 1933 Herbert Spencer lecture at the University of Oxford, “On the Method of Theoretical Physics.” We will study the various ideas laid out in that lecture and try to see how they developed gradually in the years following the discovery of general relativity.

Even though Einstein had proclaimed in Oxford that “all knowledge of reality starts from experience and ends in it,” we have seen that philosophically, more or less, he gradually moved from empiricism to realism, while his research became dominated by the unified field theory program. As we just saw, the program was criticized as it seemed to recede from the world of experience. This aspect, however, was identified by Einstein as a general trait of the search for unification. One may recognize it in the diagram that we drew in the preceding chapter, Figure 3.1: the distance between the concepts employed at the most fundamental level (A′) and direct experience (E) grows. The unified theory “pays for its higher logical unity by having elementary concepts […], which are no longer directly connected with complexes of sense experiences.” Einstein found this evolution perhaps to be regrettable, but nonetheless something that one can only resign oneself to.

A superficial glance at Einstein's professional career seems to reinforce further the image of the scholar who withdrew ever more into the ivory tower of mathematical principles: Einstein started off examining technological patents in the Bern patent office, yet ended as the sage who intuited unification axioms in the School of Mathematics of Princeton's Institute for Advanced Study.

In his 1933 Herbert Spencer lecture on the method of theoretical physics Einstein told his audience that the gravitational field equations can be found by looking for the mathematically simplest equations that a Riemannian metric can satisfy. However, the gravitational field equations were not the only example that Einstein gave in his lecture. His second example was taken from his recent work.

Einstein, together with Mayer, had in the course of 1932 turned to the Dirac theory of the electron. They had introduced the “semivector”: a generalization of the spinor, as the above suggests.