4.2.2 Probability Of Error

In channel coding, we assume that small numbers of errors are much more probable than large numbers. This assumption is fundamental because it allows efficient codes to be designed that correct the most likely error patterns while accepting that very large error bursts occur so infrequently that attempting to correct every possible error would require excessive redundancy.

This assumption is well justified for many communication channels. Thermal noise, for example, perturbs each transmitted bit independently with a relatively small probability. Although burst errors can occur in fading or impulsive-noise environments, techniques such as interleaving distribute these bursts so that, from the decoder's perspective, they again appear largely as isolated random errors.

Suppose the probability that an individual bit is received incorrectly is p, where p << 1. Assuming independent bit errors, the probability that exactly m bits are corrupted in a block of n bits is given by the binomial distribution:

P_{e} (m) = \frac{n!}{m! (n - m)!} p^{m} {(1 - p)}^{n - m}

(4.14)

This expression shows that low-weight error patterns (small m) are much more probable than high-weight patterns. Single-bit errors are much more likely than double errors.

This rapidly decreasing probability is one of the principal reasons why practical channel codes are so successful. Rather than attempting to correct every conceivable error pattern, designers concentrate on correcting the relatively few low-weight errors that dominate overall system performance. A code capable of correcting all single-bit errors, for example, will successfully correct the overwhelming majority of error events encountered on many practical communication channels.

The decoder therefore operates according to the principle of maximum likelihood: among all possible transmitted codewords, it chooses the one that would require the smallest—and therefore most probable—error pattern to produce the received sequence. Although modern decoders employ sophisticated algorithms rather than explicitly evaluating every possibility, they all embody this same statistical principle.

The assumption that low-weight errors dominate practical channels underpins the design of virtually every error-control code, from the earliest Hamming codes to today's capacity-approaching LDPC and polar codes.