Digital Communications: Basic Coding Techniques
Digital Communications: Basic Coding Techniques
Universidade do Minho
Escola de Engenharia 2
Introduction
• Unlike in block coding, the n elements that form the
encoded segment do not depend only on the segment of
k elements that are input at a given instant i
• They depend also on the previous segments input at
instants i − 1, i − 2, . . . , i − K
– K is the memory of the encoder
• The higher the level of memory, the higher the
complexity of the convolutional decoder, and the
stronger the error correction capability of Cconv
Universidade do Minho
Escola de Engenharia 3
Introduction
• A convolutional code with parameters n, k and K will be
denoted as Cconv(n, k, K)
• They are constructed by using basic memory units, or
delays, combined with adders and scalar multipliers
• These linear sequential circuits are also known as finite
state sequential machines (FSSMs)
• The number of memory units, or delays, defines the level
of memory of a given convolutional code Cconv(n, k, K),
determining also its error-correction capability
Universidade do Minho
Escola de Engenharia 4
Introduction
• FSSM analysis is usually performed by means of a
rational transfer function
– G(D) = P(D)/Q(D)
• of polynomial expressions in the D domain, called the
delay domain, where message and code sequences adopt
the polynomial form M(D) and C(D), respectively
• For multiple input – multiple output FSSMs, the
relationship between the message sequences and the
code sequences is described by G(D)
Universidade do Minho
Escola de Engenharia 5
Convolutional Codes and Encoders
• A convolutional encoder takes a k-tuple mi of
message elements as the input
• Generates the n-tuple ci of coded elements as
the output at a given instant I
• Coded elements depends not only on the
input k-tuple mi of the message at instant i but
also on previous k-tuples, mj present at
instants j < i
Universidade do Minho
Escola de Engenharia 6
Examples of Codes and Encoders
+ c(1) n=3
k=2
m(1) K=1
m(2)
+ c(3)
n=2
c(1) k=1
K=2
m
+ c(2) Rc = ½
• Input sequence:
– mi = (m0, m1, m2, . . .)
• Output sequences:
– c(1) = (c(1)0 , c(1)1 , c(1)2 , . . .)
– c(2) = (c(2)0 , c(2)1 , c(2)2 , . . .)
Universidade do Minho
Escola de Engenharia 8
Outputs and Impulse response
t1 100 1 1
t2 010 0 1
t3 001 1 1
___________________________________________
Input response = 11 01 11
Universidade do Minho
Escola de Engenharia 10
Outputs and Impulse response
m = (1101)
t1 1 11 01 11
t2 1 11 01 11
t3 0 00 00 00
t4 1 11 01 11
__________________________________________________
g(2) = (1 1 1)
Universidade do Minho
Escola de Engenharia 13
D-Transform Domain
• In the field of signals and their spectra, convolution
in the time domain becomes multiplication in the
spectral domain
• This suggests a better description of convolutional
codes based on expressions given in the D-transform
domain
• D-transform domain Delay domain D
• The sequences adopt a polynomial form expressed in
the variable D
Universidade do Minho
Escola de Engenharia 14
D-Transform Domain
• Delay D can be interpreted as a shift parameter, and
it plays the same role as the term Z−1 in the Z
transform
g(1) = (1 0 1)
g(2) = (1 1 1)
Universidade do Minho
Escola de Engenharia 16
D-Transform Domain
• In a more general FSSM structure for which there
are k inputs and n outputs, there will be kn
transfer functions that can be arranged in matrix
form as:
Universidade do Minho
Escola de Engenharia 17
D-Transform Domain
• A convolutional code Cconv(n, k, K) produces an
output sequence expressed in polynomial form
as:
– C(D) = M(D)G(D)
• where:
– M(D) = M(1)(D), M(2)(D), . . . , M(k)(D)
• and:
– C(D) = C(1)(D), C(2)(D), . . . , C(n)(D)
Universidade do Minho
Escola de Engenharia 18
Example
• For the convolutional code Cconv(2, 1, 2)
g(1) = (1 0 1)
g(2) = (1 1 1)
Universidade do Minho
Escola de Engenharia 19
Example
• The input sequence in polynomial form is
– M(D) = 1 + D4 + D5
• The generator matrix is of the form
– G(D) = [1 + D2 1 + D + D2]
• Then
– C(D) = (C(1)(D) C(2)(D)) = [1+D4+D5] [1+D2 1+D+D2]
– C(D) = [1+D2+D4+D5+D6+D7 1+D+D2+D4+D7]
– c(1) = (10101111); c(2) = (11101001)
• The output sequence is
– c = (11, 01, 11, 00, 11, 10, 10, 11)
Universidade do Minho
Escola de Engenharia 20
Another example
• For the encoder of the Cconv(3, 2, 1)
Universidade do Minho
Escola de Engenharia 22
Convolutional Encoder Representations
Universidade do Minho
Escola de Engenharia 23
Representation of Connections
• One form of describing a Cconv(n, k, K) is by means of
a vector description of the connections that the
FSSM has
• Directly described by the vector representation of
the generator polynomials g(1) and g(2) that
correspond to the upper and lower branches of the
FSSM
• In this description, a ‘ 1 ’ means that there is
connection, and a ‘0’ means that the corresponding
register is not connected
Universidade do Minho
Escola de Engenharia 24
Representation of Connections
• For a given input sequence, this code description can
provide the corresponding output sequence. This can
be seen by implementing a table
g(1) = (1 0 1)
g(2) = (1 1 1)
m = (100011)
c = (1101110011101011)
Universidade do Minho
Escola de Engenharia 25
Convolutional Encoder Representations
• Another table is a useful tool for constructing the
corresponding state diagram
g(1) = (1 0 1) g(2) = (1 1 1)
Universidade do Minho
Escola de Engenharia 26
State Diagram Representation
• The state diagram representation is obtained by the
previous table:
Universidade do Minho
Escola de Engenharia 27
State Diagram Representation
• There are only two transitions emerging from a given
state
• If the FSSM is in a given state, it is not possible to go
to any other state in an arbitrary manner
• This sort of memory will be useful in determining
that some transitions are not allowed in the decoded
sequence, thus assisting the decisions required for
error correction
Universidade do Minho
Escola de Engenharia 28
Trellis Representation
• A way of representing systems based on FSSMs is the
tree diagram (see references)
• This representation is useful to indicate the start of
the state sequence, but however the repetitive
structure of state evolution is not clearly presented
• One of the interesting characteristics of the state
evolution of a convolutional code is precisely that
after K + 1 initial transitions, the state structure
becomes repetitive, where K + 1 is the constraint
length of the code
Universidade do Minho
Escola de Engenharia 29
Trellis Representation
• State diagram clearly shows the repetitive structure of
the state evolution of a convolutional code, but it is not
clear for describing the initial evolution
• A representation that clearly describes these two
different parts of the state structure of a given
convolutional code is the so-called trellis diagram
• This trellis diagram is a state versus time instant
representation
• There are two branches emerging from and arriving at a
given state, which correspond to transitions produced by
the two possible inputs to the FSSM
Universidade do Minho
Escola de Engenharia 30
Trellis Representation
• There are 2K possible states in this diagram
• The state structure becomes repetitive after time
instant t4
Universidade do Minho
Escola de Engenharia 31
Convolutional Codes in Systematic Form
• In a systematic code, message information can be
seen and directly extracted from the encoded
information
• In the case of a convolutional code, the transfer
function for a systematic convolutional code is of the
form:
Universidade do Minho
Escola de Engenharia 32
Convolutional Codes in Systematic Form
Universidade do Minho
Escola de Engenharia 33
Convolutional Codes in Systematic Form
Universidade do Minho
Escola de Engenharia 34
Finite Impulse Response FSSMs
• The FIR general structure and the Transfer Function
are:
– ai = {0 , 1}
Universidade do Minho
Escola de Engenharia 35
Infinite Impulse Response FSSM
• The IIR general structure and the Transfer Function are:
– ai = {0 , 1}
– fi = {0 , 1}
feedback
Universidade do Minho
Escola de Engenharia 36
Relationship Between the Systematic
and the Non-Systematic Forms
• The transfer function description of an FSSM encoder can
be used to obtain the equivalent systematic form of a given
non-systematic encoder
• This conversion method consists of converting the transfer
function of a non-systematic form into an expression of
systematic form by means of matrix operations
• Example: Determine the equivalent systematic version of
the convolutional encoder generated by the transfer
function: G(D) = Gns(D) = [1+D2 1+D+D2]
Universidade do Minho
Escola de Engenharia 37
Relationship Between the Systematic and
the Non-Systematic Forms
• The transfer function should adopt the form bellow to
correspond to a systematic convolutional encoder
Universidade do Minho
Escola de Engenharia 39
Relationship Between the Systematic and
the Non-Systematic Forms
• It can be verified that transitions in the trellis have the
same output assignments as the trellis of convolutional
encoder in systematic form
• The difference between the systematic and the non-
systematic forms of the same convolutional code is in
the way the input is assigned a given output
• As in the case of block codes, the systematic
convolutional encoder generates the same code as its
corresponding non-systematic encoder, but with
different input–output assignments
Universidade do Minho
Escola de Engenharia 40
Relationship Between the Systematic and
the Non-Systematic Forms
• Trellis
Systematic
Universidade do Minho
Escola de Engenharia 41
Relationship Between the Systematic and
the Non-Systematic Forms
• The shortest state sequence using IIR transfer
functions, does not correspond to the unit impulse
input
– in this case generates an infinite output or state
sequence
– M(D) = 1 + D2
– (S1 S2) = (00, 10, 01, 00, 00, . . .)
• that is, the shortest sequence in the trellis
Universidade do Minho
Escola de Engenharia 42
Relationship Between the Systematic and
the Non-Systematic Forms
• The corresponding output sequence is:
Universidade do Minho
Escola de Engenharia 43
Distance Properties of Convolutional Codes
• One of the most significant parameters of an error-
correcting or error-detecting code is the minimum
distance of the code, normally evaluated as the minimum
value of the distance that exists between any two code
vectors of the code
• When the code is linear, it is sufficient to determine the
distance between any code vector and the all-zero vector
• As seen for block codes, the minimum distance can be
interpreted as the minimum-weight error pattern that
converts a given code vector into another code vector in
the same code
Universidade do Minho
Escola de Engenharia 44
Distance Properties of Convolutional Codes
• In the case of convolutional codes, this becomes the
number of errors that convert a given code sequence into
another valid code sequence
• Since almost all convolutional codes of practical use are
linear, the minimum distance of the code can be
determined by finding the code sequence of minimum
weight
• The above analysis implies a search for the minimum
number of errors that convert the all-zero sequence into
another code sequence
Universidade do Minho
Escola de Engenharia 45
Distance Properties of Convolutional Codes
• This can be seen in the corresponding trellis of the
convolutional code as a sequence that emerges from the all-
zero state, normally labelled Sa, and arrives back at the same
state after a certain number of transitions
• Then the Hamming or minimum distance of the code can be
determined by calculating the minimum weight among all the
sequences that emerge from and arrive back at the all-zero
state Sa after a finite number of transitions
• A tool for analysing the distance properties of a convolutional
code is obtained by modifying the traditional state diagram, in
such a way that the modified diagram starts and ends in the all-
zero state Sa
Universidade do Minho
Escola de Engenharia 46
Distance Properties of Convolutional Codes
• Using the previous example of the Cconv(2, 1, 2)
df = 5
Universidade do Minho
Escola de Engenharia 47
Minimum Free Distance of a
Convolutional Code
• The minimum free distance determines the properties of a
convolutional code, and it is defined as:
– df = min { d (ci , cj) : mi ≠ mj}
– where ci , cj are two code sequences that correspond to the
message sequences mi ≠ mj
• The minimum free distance is defined as the minimum distance
between any two code sequences of the convolutional code
• Assuming that the convolutional code is linear, the calculation of
the minimum distance between any two code sequences is the
same as determining the weight of the sum of these two code
sequences
Universidade do Minho
Escola de Engenharia 48
Minimum Free Distance
• The calculation of the minimum distance between any two
code sequences is the same as determining the weight of the
sum of these two code sequences as:
– df = min { w (ci cj) : mi ≠ mj } = min { w(c) : mi ≠ 0 }
• This also implies that the minimum free distance of a
convolutional code is the minimum weight calculated among
all the code sequences that emerge from and return to the
all-zero state, and that are not the all-zero sequences, c ≠ 0
• Example: Determine the minimum free distance of the
convolutional code Cconv(2, 1, 2) used in previous examples, by
employing the above procedure, implemented over the
corresponding trellis
Universidade do Minho
Escola de Engenharia 49
Minimum Free Distance
• The sequence corresponding to the path described by the
sequence of states SaSbScSa, seen in bold in Figure, is the
sequence of minimum weight, which is equal to 5
Universidade do Minho
Escola de Engenharia 51
Minimum Free Distance
• The error-correction capability of the code is
defined as the number t of errors that can be
corrected, which is equal to:
Universidade do Minho
Escola de Engenharia 52
Maximum Likelihood Detection
• For a given code sequence c generated by the
encoder of a convolutional code, the channel noise
converts this sequence into the received sequence sr,
which is essentially the code sequence c with errors
produced in the transmission
• An optimal decoder is one that is able to compare
the conditional probabilities P(sr/c ’ ) that the
received sequence sr corresponds to a possible code
sequence c’, and decide upon the code sequence
with the highest conditional probability
Universidade do Minho
Escola de Engenharia 53
Maximum Likelihood Detection
• The maximum likelihood criterion is given by:
Universidade do Minho
Escola de Engenharia 54
Maximum Likelihood Detection
• If the channel is memoryless, and the noise is additive,
white and Gaussian (AWGN), each symbol is
independently affected by this noise
• For a convolutional code of rate 1/n, the probability of
being alike to the received sequence is measured as:
Universidade do Minho
Escola de Engenharia 56
The Viterbi Algorithm
• The decoding procedure consists of calculating the
cumulative distance between the received sequence at
an instant ti at a given state of the trellis, and each of all
the code sequences that arrive at that state at that
instant ti
• This calculation is done for all states of the trellis, and for
successive time instants, in order to look for the
sequence with the minimum cumulative distance
• The sequence with the minimum cumulative distance is
the same as the sequence with the highest probability of
being alike to the received sequence if transmission is
done over the AWGN channel
Universidade do Minho
Escola de Engenharia 57
Example of VA
• Example: Apply the VA to the convolutional code, whose
trellis and encoder are represented bellow, if the received
sequence is sr = 11 01 01 00 11 . . . .
Universidade do Minho
Escola de Engenharia 58
Example of VA
• The first step is to determine the Hamming distance
between the received sequence and the outputs at the
different states and time instants, over the trellis
Universidade do Minho
Escola de Engenharia 59
Example of VA
• When two or more paths arrive at a given time instant
and state of the trellis, only one of them would have the
minimum cumulative distance, and should be selected
from among the others as the survivor
Universidade do Minho
Escola de Engenharia 60
The Viterbi Algorithm
• After a given number of time instants, the procedure is
truncated, and the sequence with the minimum
cumulative Hamming distance is selected as the decoded
sequence
• J is defined as the decoding length, measured in time
instants
• It can be heuristically shown that the error-correction
capability of the convolutional code is maximized if J is
approximately equal to five times the constraint length of
the code; that is, J ≈ 5(K + 1)
– In the example: J = 15
Universidade do Minho
Escola de Engenharia 61
The Viterbi Algorithm
• VA leads directly to the estimated message
sequence, and performs error correction without the
need of decoding table look-up, or of algebraic
equation solution, as in the case of traditional
syndrome decoding of block codes
Universidade do Minho
Escola de Engenharia 62
TCM - Trellis-Coded Modulation
• Channel coding increases the bandwidth occupied by the signal
because, in the same time interval, we go from a number k of
message symbols to a higher number, n, of coded symbols
– That is, the bandwidth increases by a factor n/k
• This increase was one of the important reasons for not using
channel coding for a long time in digital communications via
the telephone line, where, as is well known, bandwidth is not
plentiful
• It is essential to use channel coding on the telephone line if we
want to obtain high transmission rates
Universidade do Minho
Escola de Engenharia 63
TCM - Trellis-Coded Modulation
• But how, if the telephone lines are the same as they used to be and
the encoded signal takes up more bandwidth?
– Because in the meantime a new way of improving transmission (through
encoding) was invented without the cost of increasing the required
bandwidth
• Austrian scientist Gottfried Ungerboeck in 1976 and again in 1982
proposed a practical method of combining digital encoding and
modulation operations
– Ungerboeck's method became known as “Trellis-Coded Modulation”, or TCM
– The idea of using encryption without increasing bandwidth was something “heretical” or “revolutionary” at the
time: in the December 1989 issue of the IEEE Journal on Selected Areas in Communications it is mentioned in an
invited editorial that at the international conference telecommunications company ICC '78 a communication
whose title contained the phrase “encoding without bandwidth expansion” was greeted with laughter from the
audience
Universidade do Minho
Escola de Engenharia 64
TCM - Trellis-Coded Modulation
• The transmission of m bits in each signaling interval usually
requires digital modulation with a signal constellation of 2m points
– modulation that can be multilevel or multiphase, as QAM and MPSK
• What did Ungerboeck propose anyway?
– That to transmit m bits/symbol, 2m+1 dot constellations would be used
instead of the 2m constellations
– That is, he suggested increasing the size of the signal constellation instead
of expanding the bandwidth
• The idea is to consider encoding and modulation as a single
integrated operation, with the aim of maximizing the minimum
Euclidean distance between points in the constellation
Universidade do Minho
Escola de Engenharia 65
TCM - Trellis-Coded Modulation
• In a modulated signal, each block of bits in the original message
corresponds to a point in the constellation
• An arbitrary message thus corresponds, in the signal space, to an
equally arbitrary “jumping” between any points in the
constellation, all hops being allowed
– In 16QAM we can jump from any point to any of the remaining 15
Universidade do Minho
Escola de Engenharia 66
TCM - Trellis-Coded Modulation
• What coded modulation does is prevent hops from
occurring without restrictions, allowing only a few as
per the coding rule instituted
• Using the terminology of convolutional codes:
– only some trellis paths are allowed
• The decoding-demodulation work is normally
performed with the Viterbi algorithm
– try to find, among all the possible or valid routes, the one that
most resembles the “route” received
Universidade do Minho
Escola de Engenharia 67
TCM - Trellis-Coded Modulation
• Block diagram of a generic TCM encoder
Universidade do Minho
Escola de Engenharia 68
TCM - Trellis-Coded Modulation
• To each group of m + 1 bits the mapper assigns an
integer number (for example from 0 to 2m+1-1), which
will correspond to a point of the constellation used
• System input bits may not all be applied to encoder
– If it did not exist, the m bits in the input would be applied to
the mapper, giving rise to 2m constellation symbols
– As, with encoder, there is one more bit in the mapper input,
we now have a constellation with twice as many points, 2m+1 ,
but, note, without increasing the bandwidth
Universidade do Minho
Escola de Engenharia 69
TCM - Trellis-Coded Modulation
• The expanded constellation has twice the points of the original
constellation
– it was concluded that the coding gain thus obtained is practically the
maximum possible and no significant benefits are obtained if we use more
points
• Convolutional encoder is usually 1/2 or 2/3 rate
Universidade do Minho
Escola de Engenharia 70
TCM - Example
• 4-state Ungerboeck code for 8PSK
Universidade do Minho
Escola de Engenharia 71
TCM - Example
• We have m = 2 and m = 1
• the uncoded bit will cause parallel transitions in the trellis
• As the constraint length of the convolutional encoder is
K+1 = 3 there will be 2K = 4 states
• From each state it is possible to reach 2 states
• Between every 2 consecutive states there are 2 parallel
transitions
• The constellation to use has 2m+1 = 23 = 8 points, each
corresponding to a group of 3 bits
Universidade do Minho
Escola de Engenharia 72
TCM - Example
• Code Trellis
Universidade do Minho
Escola de Engenharia 73
TCM - Example
• It is observed that, being in any state, it is only possible to
obtain one of four possible outputs
• For example, if at a given moment it is in state 00, the
possible outputs are 000, 100, 010 and 110
• As each branch, or exit, will be associated with 1 of the 8
points of the constellation, we conclude that from each
one of these points it is not possible to move to any
other point, but only to one of the 4 allowed
• It is in this interdependence between signal points that
one of the main positive characteristics of TCM resides
Universidade do Minho
Escola de Engenharia 74
TCM
• The Euclidean distance between valid paths traversed in the trellis
must be as large as possible
• Ungerboeck proposed the following rules for assigning
constellation points to trellis branches to obtain the best TCM
schemes:
– 1. The parallel transitions of the trellis correspond to the maximum possible
Euclidean distance between points in the constellation
– 2. The branches that diverge or converge in the same state of the trellis
correspond to the next largest possible Euclidean distance
– 3. The constellation signals (dots) must be used the same number of times
• The most common decoding method is the Viterbi maximum
likelihood algorithm
Universidade do Minho
Escola de Engenharia 75
Reading Material
• S. Lin and D. J. Costello. Error Control Coding. Prentice-
Hall, Upper Saddle River, NJ, 2nd edition, 2004 (ISBN 0-
13-017973-6)
• Sílvio A. Abrantes. Códigos Correctores de Erros em
Comunicações Digitais, FEUP Edições, Porto, 2010
(ISBN 978-972-752-127-2)
• Simon Haykin. Communication Systems, McMaster
Univ, 4th Edition, 2004. (ISBN: 978-0-471-17869-9)
Universidade do Minho
Escola de Engenharia 76
Reading Material
Universidade do Minho
Escola de Engenharia 77