Communications Fundamentals FAQ / Chapter 4

4.18.2 What Is Shannon's Information Capacity Theorem?

1. Who Was Claude Shannon?
2. What Is Channel Capacity?
3. Why is it called Shannon–Hartley equation?
4. Why Not Just Call It Shannon's Equation?
5. Why Does Noise Limit Communication?
6. What Does the Equation Tell Us?
7. Why Is the Logarithmic Relationship Important?
8. What Is Signal-to-Noise Ratio?
9. What Is the Shannon Limit?
10. Does Shannon's Theorem Describe a Real System?
11. What Is Spectral Efficiency?
12. Why Is Spectral Efficiency Important?
13. What Is the Relationship Between Capacity and Eb/N₀?
14. What Is Special About −1.59 dB?
15. What Are Power-Limited Systems?
16. What Are Bandwidth-Limited Systems?
17. Can Capacity Be Increased Indefinitely?
18. How Close Do Modern Systems Come to Capacity?
19. What Is the Importance of Shannon's Theorem?
20. Where Is Shannon's Theorem Used?
21. Why Is Shannon's Information Capacity Theorem Important?

Few developments have had a greater impact on modern communications engineering than the work of Claude Shannon. Before Shannon's groundbreaking research in the late 1940s, engineers knew that noise limited the performance of communication systems, but they did not know the fundamental limits. Was there a maximum data rate that could be transmitted over a noisy channel? Could sufficiently clever engineering eliminate errors entirely? Was there a theoretical limit beyond which communication became impossible?

Shannon answered these questions in his landmark 1948 paper, A Mathematical Theory of Communication. In doing so, he established the foundations of modern information theory and introduced one of the most important concepts in communications engineering: channel capacity.

Shannon's Information Capacity Theorem, often called the Shannon–Hartley Theorem, defines the maximum rate at which information can be transmitted over a communication channel in the presence of noise. Remarkably, this limit depends only on the channel bandwidth and the signal-to-noise ratio. No amount of engineering ingenuity can exceed it.

Today, Shannon's theorem underpins the design of virtually every modern communications system, from mobile phones and fiber-optic networks to satellite systems, Wi-Fi, and deep-space communications.

Who Was Claude Shannon?

Claude Elwood Shannon (1916–2001) was an American mathematician, electrical engineer, and researcher widely regarded as the father of information theory.

Before Shannon's work, communications engineering focused largely on practical problems such as transmitters, receivers, antennas, and transmission lines. Shannon introduced a new perspective:

Communication could be analyzed mathematically, independent of the physical medium used to carry the information. His work established a theoretical framework that remains central to communications engineering more than seventy-five years later.

Shannon's contributions include:

• Information theory.
• Channel capacity.
• Entropy as a measure of information.
• Source coding theory.
• Channel coding theory.
• Digital communications principles.

Virtually every modern communication system is influenced by his work.

What Is Channel Capacity?

Channel capacity is the maximum rate at which information can be transmitted reliably through a communication channel.

The word "reliably" is important. A channel may be capable of transmitting information at higher rates, but if the error rate becomes unacceptable, communication is no longer reliable.

Channel capacity therefore represents: The highest achievable information rate for which arbitrarily low error probabilities are theoretically possible. It is measured in bits per second (bps).

Capacity represents an absolute upper limit. No communication system can exceed it.

Why is it called Shannon–Hartley equation?

The equation is called the Shannon–Hartley theorem or Shannon–Hartley law because it combines the contributions of two researchers:

In 1928, Ralph Hartley published a paper titled Transmission of Information. Hartley introduced one of the first quantitative measures of information and showed that the amount of information that can be transmitted increases with:

• Bandwidth.
• Transmission time.
• The number of distinguishable signal levels.

Hartley established the idea that communication capacity could be measured mathematically, but his analysis assumed a noiseless channel. He did not determine how noise limits communication.

Twenty years later, Claude Shannon developed modern information theory in A Mathematical Theory of Communication. Shannon solved the problem Hartley had left unanswered: how much information can be transmitted over a channel corrupted by noise?

By combining Hartley's measure of information with a statistical analysis of noise, Shannon derived the famous capacity relationship:

$C=B{\mathrm{l}\mathrm{o}\mathrm{g}}_2(1+S/N)$where:

• $C$= channel capacity (bps)
• $B$= bandwidth (Hz)
• $S/N$= signal-to-noise ratio

This equation gives the maximum achievable error-free information rate for an ideal AWGN channel.

Why Not Just Call It Shannon's Equation?

In practice, many engineers do exactly that and refer simply to:

• Shannon capacity.
• Shannon limit.
• Shannon theorem.
• Shannon's capacity formula.

However, historians of communications engineering generally recognize that the equation combines:

• Hartley's earlier measure of information.
• Shannon's analysis of noisy channels.

For that reason, the formal name is often Shannon–Hartley theorem.

Why Does Noise Limit Communication?

Every practical communication channel contains noise.

Common noise sources include:

• Thermal noise.
• Amplifier noise.
• Atmospheric noise.
• Cosmic noise.
• Interference from other systems.

Noise obscures the transmitted signal. As the information rate increases, symbols must be packed more closely together in time, frequency, or amplitude. Eventually, noise makes it impossible for the receiver to distinguish between them reliably. The result is an increase in error probability.

The central question is therefore: How much information can be transmitted before noise makes reliable communication impossible? Shannon's theorem provides the answer.

What Does the Equation Tell Us?

The equation reveals several important principles.

• Increasing bandwidth increases capacity. Larger bandwidth allows more information to be transmitted. Doubling bandwidth approximately doubles capacity when other factors remain constant.
• Increasing Signal Power Increases capacity. A stronger signal is easier to distinguish from noise. Increasing signal power therefore increases capacity.
• Capacity increases logarithmically with SNR. This is perhaps the most important insight. Doubling signal power does not double capacity. Instead, capacity increases according to a logarithmic relationship. As a result, enormous increases in power may produce only modest increases in capacity.

Why Is the Logarithmic Relationship Important?

Consider a channel with B = 1 MHz and S/N = 10.

The capacity is: C = 1,000,000 log₂(11) = 3.46 Mbps.

Now increase signal power by a factor of ten so the S/N = 100. The capacity becomes C = 1,000,000 log₂(101) = 6.66 Mbps. Although power increased tenfold, capacity increased by less than a factor of two.

This illustrates why simply increasing transmitter power is rarely an efficient solution.

What Is Signal-to-Noise Ratio?

Signal-to-noise ratio (SNR) compares signal power with noise power: SNR = S/N. A higher SNR indicates a cleaner channel.

SNR is often expressed in decibels: SNR_dB=10 log₁₀(S/N).

Higher SNR generally enables higher data rates.

What Is the Shannon Limit?

The Shannon limit represents the theoretical boundary between possible and impossible communication.

Below the limit reliable communication is theoretically achievable. Above the limit reliable communication is impossible regardless of coding complexity. This distinction is fundamental.

Engineers may develop better modulation schemes, coding systems, and signal-processing techniques. However, none can surpass the Shannon limit.

Does Shannon's Theorem Describe a Real System?

Not exactly.

The theorem describes the theoretical maximum performance achievable by an ideal communication system. Practical systems must contend with:

• Hardware imperfections.
• Finite code lengths.
• Processing limitations.
• Synchronization issues.
• Real-world channel impairments.

Consequently, practical systems operate below capacity.

The goal of communications engineering is to approach the Shannon limit as closely as possible.

What Is Spectral Efficiency?

Spectral efficiency measures how efficiently bandwidth is used.

It is defined as the ration of the data rate to the bandwidth. The units are bits/s/Hz.

Why Is Spectral Efficiency Important?

Bandwidth is a valuable and limited resource.

Governments carefully regulate spectrum allocation because only a finite amount is available. Higher spectral efficiency means:

• More users per channel.
• Higher throughput.
• Better spectrum utilization.

Modern communication systems place great emphasis on maximizing spectral efficiency.

What Is the Relationship Between Capacity and E_b/N₀?

Communications engineers often use E_b/N₀ rather than SNR.

Where:

E_b = energy per bit.

N₀ = noise spectral density.

This parameter directly relates signal energy to information transmission.

Using information theory, it can be shown that the theoretical minimum value required for reliable communication is:

E_b/N₀ = -1.59 dB

This value is often called the ultimate Shannon limit. No communication system can operate below it.

What Is Special About −1.59 dB?

This number surprises many. It implies that reliable communication is theoretically possible even when the signal power is less than the noise power.

The key is that reliable communication may require:

• Very low spectral efficiency.
• Extremely powerful coding.
• Very long codewords.

Although practical systems rarely operate close to this limit, it illustrates the extraordinary power of modern coding techniques.

What Are Power-Limited Systems?

Some systems have abundant bandwidth but limited power. Examples include:

• Deep-space communications.
• Satellite systems.
• Battery-powered devices.

In these systems:

• Power efficiency is critical.
• Bandwidth may be relatively plentiful.

Designers focus on minimizing required E_b/N₀.

What Are Bandwidth-Limited Systems?

Other systems face the opposite constraint.

Examples include:

• Cellular networks.
• Broadcast services.
• Wi-Fi systems.

Here:

• Spectrum is scarce.
• Additional bandwidth is expensive.

Designers focus on maximizing spectral efficiency.

The trade-off between power efficiency and bandwidth efficiency is one of the central themes of communications engineering.

Can Capacity Be Increased Indefinitely?

No.

The Shannon equation shows that capacity can be increased by:

• Increasing bandwidth.
• Increasing SNR.

However, both resources are finite. Consequently, every channel has a finite capacity.

This reality imposes unavoidable limits on communication-system performance.

How Close Do Modern Systems Come to Capacity?

For many decades, practical systems operated far below Shannon's theoretical limit.

During the 1990s and 2000s, powerful coding techniques dramatically reduced this gap.

Examples include:

• Turbo codes.
• LDPC codes.
• Polar codes.

Modern systems can operate within 1 dB and sometimes less than 0.5 dB of Shannon capacity.

This achievement is often regarded as one of the greatest successes in communications engineering.

What Is the Importance of Shannon's Theorem?

Shannon's theorem transformed communications engineering.

It established:

• A precise definition of information.
• The concept of channel capacity.
• The theoretical limits of communication.
• The possibility of reliable communication in noisy environments.

Most importantly, it provided engineers with a target.

Rather than guessing what might be achievable, designers now know the ultimate limits imposed by nature.

Where Is Shannon's Theorem Used?

The theorem influences virtually every communications technology.

• Mobile communications. 4G and 5G systems are designed to approach capacity.
• Satellite communications. Link budgets often include comparisons with Shannon limits.
• Fiber-optic networks. Capacity calculations guide system design.
• Wi-Fi systems. Modulation and coding choices reflect capacity considerations.
• Deep-space missions. Information theory plays a crucial role in spacecraft communications.
• Data storage. Capacity concepts influence error-correcting codes and storage density.

Few engineering theories have achieved such widespread practical impact.

Why Is Shannon's Information Capacity Theorem Important?

Shannon's theorem provides the fundamental benchmark against which all communication systems are measured.

It defines the maximum achievable information rate for a noisy channel and establishes the ultimate limits of communication performance. Although practical systems can never exceed these limits, advances in coding and signal processing have enabled engineers to approach them remarkably closely.

As a result, Shannon's Information Capacity Theorem remains one of the most influential and important results in all of engineering.

Summary

Shannon's Information Capacity Theorem defines the maximum rate at which information can be transmitted reliably over a noisy communication channel. The Shannon–Hartley equation shows that capacity depends only on channel bandwidth and signal-to-noise ratio, and establishes a fundamental limit that no communication system can exceed.

The theorem introduced the concepts of channel capacity, spectral efficiency, and the Shannon limit, providing the foundation for modern information theory and communications engineering. Today, virtually every communication system—from mobile phones and Wi-Fi networks to satellites and deep-space probes—is designed with Shannon's capacity limits in mind.

Return to Chapter 4 FAQ 4.18.2