This chapter first provides an overview of a general communication system and then shifts the focus to a digital communication system. It describes elements of a digital communication system and explains the functionalities of source coding, channel coding, and digital modulation blocks for communicating over a noisy channel. It also highlights the differences between analog and digital communication systems.

This chapter explores Age of Information (AoI) in the context of the timely source coding problem. In most of the existing literature, service (transmission) times are based on a given distribution. In the timely source coding problem, by using source coding schemes, we design the transmission times of the status updates. We observe that the average age minimization problem is different than the traditional source coding problem, as the average age depends on both the first and the second moments of the codeword lengths. For the age minimization problem, we first consider a greedy source coding scheme where all realizations are encoded. For this source coding scheme, we find the age-optimal real-valued code word lengths. Then, we explore the highest k selective encoding scheme, where instead of encoding all realizations, we encode only the most probable k realizations. For each source encoding scheme, we first determine the average age expressions and then, for a given pmf, characterize the age-optimal k value, and find the corresponding age-optimal codeword lengths. Through numerical results, we show that selective encoding schemes achieve lower average age than encoding all realizations.

The purpose of this chapter is to set the stage for the book and for the upcoming chapters. We first overview classical information-theoretic problems and solutions. We then discuss emerging applications of information-theoretic methods in various data-science problems and, where applicable, refer the reader to related chapters in the book. Throughout this chapter, we highlight the perspectives, tools, and methods that play important roles in classic information-theoretic paradigms and in emerging areas of data science. Table 1.1 provides a summary of the different topics covered in this chapter and highlights the different chapters that can be read as a follow-up to these topics.