4.2 BASIC ELEMENTS OF CODING

All forms of error control rely on the deliberate introduction of redundancy into the transmitted signal. Redundancy provides structure, and structure enables the receiver to detect and correct deviations caused by noise and other impairments. Redundancy is familiar in natural language. Written English, for example, contains substantial structural and statistical redundancy. Readers can often detect—and sometimes correct—typographical errors because only a limited subset of letter combinations form valid words.

Consider the three-letter word ‘fex’. It is immediately recognizable as invalid because few English words share that pattern. However, although the presence of an error is obvious, the intended word cannot be uniquely inferred. It might have been ‘hex’, ‘fed’, ‘fee’, and so on. Even when a single-character error is assumed and it is known to be a three-letter word, the error still can’t be corrected.

If the errored word ‘fex’ occurs in the sentence ‘The quick brown fex jumped over the lazy dog’, the additional context stored in memory allows the receiver to conclude unambiguously that the intended word is ‘fox’. The surrounding words strongly constrain the number of plausible corrections, providing the extra redundancy required to resolve the ambiguity.

This simple example illustrates the power of making use of two levels of redundancy—in our simple example, the redundancy in the English language has allowed the detection of an error in a three-letter word; the distributed redundancy across the sentence has allowed its correction.

So, in general, redundancy enables error detection and correction. If redundancy is present, a receiver can identify that a received message is not one of the valid combinations. With sufficient redundancy, the error can be detected and corrected. Without redundancy, however, every possible sequence is valid, so that a corrupted message is indistinguishable from an intended one.

If natural redundancy is so useful, why do we need to add redundancy through channel coding? The reason is that source coding (compression) deliberately removes redundancy to make transmission more bandwidth-efficient. This means that, after source coding, any bit error becomes significant, as there is no built-in redundancy to assist in detection or correction.

Furthermore, encryption reduces redundancy even further by randomizing the data. Encrypted messages have near-uniform statistical distributions, so that any bit change produces another plausible but incorrect message. Additionally, not all data sources have inherent redundancy. For example, in a computer program, changing a single instruction bit can lead to catastrophic system failure. To protect such data, channel coding reintroduces controlled redundancy—structured and mathematically defined—to detect and correct errors introduced during transmission.