(5 votes, average: 4.40 out of 5)

# Understanding Shannon’s capacity equation

by on June 15, 2008

Let us try to understand the formula for Channel Capacity with an Average Power Limitation, described in Section 25 of the landmark paper A Mathematical Theory for Communication, by Mr. Claude Shannon.

Further, the following writeup is based on Section 12.5.1 from Fundamentals of Communication Systems by John G. Proakis, Masoud Salehi

## Simple example with voltage levels

Let us consider that we have two voltage sources:
(a) Signal source which can generate voltages in the range $0$ to $V$ volts
(b) Noise source which can generate voltage levels $-\frac{V_n}{2}$ to $\frac{V_n}{2}$ volts.

Figure: Discrete voltage levels with noise

Let us now try to send information at discrete voltage levels $V_m$ from the source (thick black lines as shown in the above figure). It is intuitive to guess that the receiver will be able to decode the received symbol correctly if the received signal lies within $V_m \pm \frac{V_n}{2}$.

So, the number of different discrete voltages levels (information) which can be sent, while ensuring error free communication is the total voltage level divided by the noise voltage level i.e.
$M=\frac{V+V_n}{V_n}$.

## Extending to Gaussian channel

Let us transmit $k$ randomly chosen discrete voltage levels $V_m$ meeting the average power constraint,

$\frac{1}{k}\sum_{i=1}^{k}V_{m,i}^2 \le Ps$, where $P_s$ is the signal power.

The noise signal $V_n$ follows the Gaussian probability distribution function

$p(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{\frac{-(x-\mu)^2}{2\sigma^2}$ with mean $\mu=0$ and variance $\sigma^2 = \frac{N_0}{2}$.

The noise power $P_n$ is,

$\frac{1}{k}\sum_{i=1}^{k}V_{n,i}^2 = Pn$.

The average total (signal plus noise) voltage over $k$symbols is $\fs1 \sqrt{\sum_{i=1}^k(V_{m,i}+V_{n,i})^2}$.

Similiarly, the average noise voltage over $k$symbols is $\fs1 \sqrt{\sum_{i=1}^kV_{n,i}^2}$.

Combining the above two equations, the number of different messages $M$ which can be ‘reliably transmitted‘ is,

$\begin{eqnarray}M&=&\sqrt{\frac{\sum_i(V_{m,i}+V_{n,i})^2}{\sum_iV_{n,i}^2}}\\&=&\sqrt{\frac{\sum_iV_{m,i}^2+\sum_iV_{n,i}^2+2\sum_iV_{m,i}V_{n,i}}{\sum_iV_{n,i}^2}}\\&=&\sqrt{\frac{\sum_iV_{m,i}^2+\sum_iV_{n,i}^2}{\sum_iV_{n,i}^2}}\\&=&\sqrt{\frac{Ps+P_n}{P_n}}\end{eqnarray}$.

Note:

1. The product of the signal and noise accumulated over many symbols average to zero, i.e

$\sum_iV_{m,i}V_{n,i}=0$.

2. Since the noise is Gaussian distributed, the noise can theoretically go from $-\infty$ to $+\infty$. So the above result cannot ensure zero probability of error in receiver, but only arbitrarily small probability of error.

## Converting to bits per transmission

With $M$ different messages, the number of bits which can be transmitted per transmission is,

$\begin{eqnarray}C&=&\log_2(M)\\&=&\frac{1}{2}\log_2\left(1+\frac{P_s}{P_n}\right)\end{eqnarray}$ bits/transmission.

## Bringing bandwidth into the equation

Let us assume that the available bandwidth is $B$.

Noise is of power spectral density $\frac{N_0}{2}$ spread over the bandwidth $B$. So the noise power in terms of power spectral density and bandwidth is,

$P_n = \int_{-B}^{+B}\frac{N_0}{2}df=N_0B$.

From our previous post on transmit pulse shaping filter that minimum required bandwidth for transmitting symbols with symbol period $T$ is$+\frac{1}{2T}$Hz. Conversely, if the available bandwidth is $B$, the maximum symbol rate (transmissions per second) is $2B$.

Multiplying the equation for bits per transmission with transmission per second of $2B$ and replacing the noise term $P_n$, the capacity is

$\Huge\begin{eqnarray}C&=&B\log_2\left(1+\frac{P_s}{N_oB}\right)\end{eqnarray}$ bits/second.

Voila! This is Shannon’s equation for capacity of band limited additive white Gaussian noise channel with an average transmit power constraint.

## References

D id you like this article? Make sure that you do not miss a new article by subscribing to RSS feed OR subscribing to e-mail newsletter. Note: Subscribing via e-mail entitles you to download the free e-Book on BER of BPSK/QPSK/16QAM/16PSK in AWGN.

nad January 1, 2010 at 10:42 pm

Hi,
Can someone explain this claim (Note 1 above)
“The product of the signal and noise accumulated over many symbols average to zero”.
Thanks

Abhishek April 16, 2010 at 1:58 pm

It means that noise and signal are independent. Since the mean of noise is zero
E[XY]=E[X]E[Y]=0
if X and Y are independent and E[Y]=0

Krishna Sankar April 18, 2010 at 2:25 pm

@Abhishek: Thanks.

shadat December 6, 2009 at 3:13 pm

hi
thank you.

Krishna Sankar December 7, 2009 at 5:29 am

@shadat: For BPSK with convolutional coding with hard/soft Viterbi, please refer to
http://www.dsplog.com/tag/viterbi

Arif Iqbal April 30, 2009 at 12:53 pm

Please can u tell me how to change the data rate using spreading factor in matlab while using the same modulation scheme for different data rates.

BR,
Arif
AIT, Thailand

Krishna Pillai May 12, 2009 at 4:41 am

@Arif: Well, this depends on your simulation model. If you sampling rate is constant, but has a bigger spreading code, then the data rate reduce. Agree?

giri December 25, 2008 at 6:09 pm

Hello…

consider 4 lvl system
thermometric – 000, 001, 011, 111 … 4 levels & 3 bits
coded as 2′s – 00, 01, 10, 11 …. 4 levels & 2 bits
quick look at the paper shows there is an assumption of coding in binary after some point, will read it later.

giri December 24, 2008 at 5:33 pm

Chankaraaaa,

why is bits – messages related as log2(M)? if info is send as thermometric sequence will the no. of bits be = M / will the capacity linearly increase with snr for that case?
or suggest a good ref for me to understand these

Krishna Pillai December 25, 2008 at 7:50 am

@giri: Well, if a may bring in the analogy with ADC. With a k bit ADC one can split a voltage V into M=2^k regions, with each region is of width V/2^k (LSB size). Each message level can be positioned in the middle of each region with -LSB/2 to +LSB/2 on either sides as noise margin. So, if we assume that each message level can be used for transmission, then with M message levels we can have k = log2(M) bits Makes sense?

Ofcourse, you can also look at the paper A Mathematical Theory for Communication, by Mr. Claude Shannon

Btw, what is a thermometric sequence?

Krishna Pillai July 10, 2008 at 6:09 am

If you wish, you can guest post a brief article on the new phase modulation scheme in dsplog.com. Further, I can forward the informtion to faculty of IISc, Bangalore (hopefully, after understanding your proposed scheme).

Jagan Narayanan July 10, 2008 at 3:50 am

Hi Mr Pillai,

Do you have access to a good lab, not just a matlab terminal?

I want to try out a new abrupt phase modulation scheme. I am based in California, SF bay area.

jagan