Preface

Summary

This book is partially based on the material covered in several Cambridge Mathematical Tripos courses: the third-year undergraduate courses Information Theory (which existed and evolved over the last four decades under slightly varied titles) and Coding and Cryptography (a much younger and simplified course avoiding cumbersome technicalities), and a number of more advanced Part III courses (Part III is a Cambridge equivalent to an MSc in Mathematics). The presentation revolves, essentially, around the following core concepts: (a) the entropy of a probability distribution as a measure of ‘uncertainty’ (and the entropy rate of a random process as a measure of ‘variability’ of its sample trajectories), and (b) coding as a means to measure and use redundancy in information generated by the process.

Thus, the contents of this book includes a more or less standard package of information-theoretical material which can be found nowadays in courses taught across the world, mainly at Computer Science and Electrical Engineering Departments and sometimes at Probability and/or Statistics Departments. What makes this book different is, first of all, a wide range of examples (a pattern that we followed from the onset of the series of textbooks Probability and Statistics by Example by the present authors, published by Cambridge University Press). Most of these examples are of a particular level adopted in Cambridge Mathematical Tripos exams. Therefore, our readers can make their own judgement about what level they have reached or want to reach.