3.3 DATA COMPRESSION TECHNIQUES

The waveform coding techniques discussed in the previous section faithfully represent an analog signal by sampling and quantizing its instantaneous amplitude. Although these methods can produce excellent reproduction quality, they often require relatively high transmission bit rates because every sample is represented, regardless of whether successive samples contain new information. In practice, however, many real-world signals contain substantial redundancy. Adjacent speech samples are usually very similar, neighboring image pixels often have nearly identical values, and text contains predictable combinations of letters and words. By identifying and removing this redundancy, it is possible to represent the same information using significantly fewer bits.

The process of reducing the number of bits required to represent information is known as data compression, or more generally, source coding. Unlike channel coding, which deliberately adds redundancy to improve transmission reliability, data compression has the opposite objective: it removes redundancy wherever possible to improve transmission and storage efficiency. Effective compression allows more users to share a communications channel, reduces storage requirements, shortens download times, and enables applications such as digital television, video streaming, cloud storage, and mobile communications that would otherwise require impractically large bandwidths.

Compression techniques may be broadly classified as either lossless or lossy. Lossless methods preserve every bit of the original information, allowing the data to be reconstructed exactly. They are essential for applications such as computer software, executable files, databases, financial records, and text documents, where even a single incorrect bit may be unacceptable. Lossy methods, by contrast, deliberately discard information that is considered perceptually unimportant, allowing much greater compression ratios while maintaining acceptable subjective quality. Most image, audio, and video compression standards employ some degree of lossy compression because the human visual and auditory systems cannot perceive every detail present in the original signal.

Modern compression systems rarely rely on a single technique. Instead, they combine several complementary methods that exploit different forms of redundancy. Entropy coding removes redundancy arising from unequal symbol probabilities. Predictive coding exploits the correlation between successive samples or neighboring pixels. Transform coding represents signals in a domain where most of the information is concentrated into a relatively small number of coefficients, allowing less-important components to be discarded or represented with fewer bits. Dictionary coding replaces repeated sequences of symbols with compact references to previously encountered patterns. Individually these techniques provide useful reductions in bit rate; together they form the basis of almost every modern multimedia compression standard.

In this section we begin by introducing the concept of information and entropy, which establish the theoretical limits of data compression. We then examine the principal classes of compression techniques—entropy coding, predictive coding, transform coding, and dictionary coding—before concluding with model-based speech coding methods that exploit knowledge of human speech production to achieve extremely low transmission bit rates. Together, these techniques illustrate how communications engineers approach the fundamental challenge of representing information as efficiently as possible while preserving the level of quality required by the intended application.