14 - Distributed source coding

Summary

In this chapter, we introduce the concept that correlated sources need not be encoded jointly to achieve greater efficiency than encoding them independently. In fact, if they are encoded independently and decoded jointly, it is theoretically possible under certain conditions to achieve the same efficiency as when encoded jointly. Such a method for coding correlated sources is called distributed source coding (DSC). Figure 14.1 depicts the paradigm of DSC with independent encoding and joint decoding. In certain applications, such as sensor networks and mobile communications, circuit complexity and power drain are too burdensome to be tolerated at the transmission side. DSC shifts complexity and power consumption from the transmission side to the receiver side, where it can be more easily handled and tolerated. The content of this chapter presents the conditions under which DSC is ideally efficient and discusses some practical schemes that attempt to realize rate savings in the DSC paradigm. There has been a plethora of recent work on this subject, so an encyclopedic account is impractical and ill-advised in a textbook. The goal here is to explain the principles clearly and elucidate them with a few examples.

Slepian–Wolf coding for lossless compression

Consider two correlated, discrete scalar sources X and Y. Theoretically, these sources can be encoded independently without loss using H(X) and H(Y) bits, respectively, where H(X) and H(Y) are the entropies of these sources. However, if encoded jointly, both these sources can be reconstructed perfectly using only H(X, Y) bits, the joint entropy of these sources.