AGC

DSP A Prediction Problem

 Problem: Given a sample set of a stationary
processes
{x[n], x[n  1], x[n  2],..., x[n  M ]}
to predict the value of the process some time into
the future as

x[n  m]  f ( x[n], x[n  1], x[n  2],..., x[n  M ])

 The function may be linear or non-linear. We
concentrate only on linear prediction functions

Professor A G Constantinides© 1
AGC
DSP A Prediction Problem
 Linear Prediction dates back to Gauss in the
18th century.
 Extensively used in DSP theory and
applications (spectrum analysis, speech
processing, radar, sonar, seismology, mobile
telephony, financial systems etc)
 The difference between the predicted and
actual value at a specific point in time is
caleed the prediction error.
Professor A G Constantinides© 2
AGC
DSP A Prediction Problem
 The objective of prediction is: given the
data, to select a linear function that
minimises the prediction error.
 The Wiener approach examined earlier
may be cast into a predictive form in
which the desired signal to follow is the
next sample of the given process

Professor A G Constantinides© 3
AGC
Forward & Backward
DSP Prediction
 If the prediction is written as

xˆ[n]  f ( x[n  1], x[n  2],..., x[n  M ])

 Then we have a one-step forward prediction
 If the prediction is written as
xˆ[n  M ]  f ( x[n], x[n  1], x[n  2],..., x[n  M  1])
 Then we have a one-step backward
prediction

Professor A G Constantinides© 4
AGC
DSP Forward Prediction Problem
 The forward prediction error is then
e f [n]  x[n]  xˆ[n]
 Write the prediction equation as
M
xˆ[n]   w[k ]x[n  k ]
k 1
 And as in the Wiener case we minimise
the second order norm of the prediction
error
Professor A G Constantinides© 5
AGC
DSP Forward Prediction Problem
 Thus the solution accrues from
2 2
J  min E{(e f [n]) }  min E{( x[n]  xˆ[n]) }
w w

Expanding we have


 2 2
J  min E{( x[n]) }  2 E{( x[n]xˆ[n])  E{( xˆ[n]) }
w

 Differentiating with resoect to the
weight vector we obtain
J xˆ[n] xˆ[n]
 2 E{( x[n] )  2 E{xˆ[n] }
wi wi wi
Professor A G Constantinides© 6
AGC
DSP Forward Prediction Problem
 However xˆ[n]
 x[n  i ]
wi
 And hence
J
 2 E{( x[n]x[n  i ])  2 E{xˆ[n]x[n  i ]}
wi
 or
J M
 2 E{( x[n]x[n  i ])  2 E{  w[k ]x[n  k ]x[n  i ]}
wi k 1
Professor A G Constantinides© 7
AGC
DSP Forward Prediction Problem
 On substituting with the correspending
correlation sequences we have
J M
 2r[i ]  2  w[k ]rxx [i  k ]
wi k 1
 Set this expression to zero for
minimisation to yield
M
 w[k ]rxx [i  k ]  rxx [i ] i  1,2,3,..., M
k 1
Professor A G Constantinides© 8
AGC
DSP Forward Prediction Problem
 These are the Normal Equations, or Wiener-
Hopf , or Yule-Walker equations structured
for the one-step forward predictor

 In this specific case it is clear that we need

only know the autocorrelation propertities of
the given process to determine the predictor
coefficients

Professor A G Constantinides© 9
AGC
DSP Forward Prediction Filter
1 m0
 Set
aM [ m]   w[ m] m  1,.., M
0 mM
 And rewrite earlier expression as
M rxx [0] k 0
 aM [m]rxx [m  k ] 
m 0 0 k  1,2,..., M
 These equations are sometimes known as the
augmented forward prediction normal equations

Professor A G Constantinides© 10
AGC
DSP Forward Prediction Filter
 The prediction error is then given as
M
e f [n]   aM [k ]x[n  k ]
m 0

 This is a FIR filter known as the

prediction-error filter
1 2 M
A f ( z )  1  a1[1]z  aM [2]z  ...  aM [ M ]z

Professor A G Constantinides© 11
AGC
DSP Backward Prediction Problem
 In a similar manner for the backward
prediction case we write
eb [n]  x[n  M ]  xˆ[n  M ]
 And
M
~[k ]x[n  k  1]
xˆ[n  M ]   w
k 1
 Where we assume that the backward
predictor filter weights are different from the
forward case

Professor A G Constantinides© 12
AGC
DSP Backward Prediction Problem
 Thus on comparing the the forward and
backward formulations with the Wiener least
squares conditions we see that the desirable
signal is now
x[n  M ]
 Hence the normal equations for the backward
case can be written as
M
 w~[m]rxx [m  k ]  rxx [ M  k  1] k  1,2,3,..., M
m 1


Professor A G Constantinides© 13
AGC
DSP Backward Prediction Problem
 This can be slightly adjusted as
M
 w~[ M  m  1]rxx [k  m]  rxx [k ] k  1,2,3,..., M
m 1

 On comparing this equation with the

corresponding forward case it is seen that the
two have the same mathematical form and
w[m]  w~[ M  m  1] m  1,2,..., M
 Or equivalently
w~[m]  w[ M  m  1]  m  1,2,..., M
Professor A G Constantinides© 14
AGC
DSP Backward Prediction Filter
 Ie backward prediction filter has the same
weights as the forward case but reversed.
Ab ( z )  aM [ M ]  aM [ M  1]z 1  aM [ M  2]z  2  ...  z  M
 This result is significant from which many
properties of efficient predictors ensue.
 Observe that the ratio of the backward
prediction error filter to the forward
prediction error filter is allpass.
 This yields the lattice predictor structures.
 More on this later
Professor A G Constantinides© 15
AGC
DSP Levinson-Durbin
 Solution of the Normal Equations
 The Durbin algorithm solves the following
R m w m  rm
 Where the right hand side is a column of R
as in the normal equations.
 Assume we have a solution for
R k w k  rk 1 k  m
 Where rk  [ r1 , r2 , r3 ,..., rk ]T

Professor A G Constantinides© 16
AGC
DSP Levinson-Durbin
 For the next iteration the normal equations can
be written as
 Rk *
J k rk 
 T  w k 1  rk 1
rk J k r0 
 Where
Jk
 rk  Is the k-order
rk 1   
counteridentity
rk 1 
Set zk 

w k 1   
 k 
Professor A G Constantinides© 17
AGC
DSP Levinson-Durbin
 Multiply out to yield
1 1
z k  R (rk   k J r )  w k   k R J r
k
*
k k
*
k k k
 Note that R k1J k  J k R k1
 Hence zk  wk  k Jk w *
k

 Ie the first k elements of w k 1 are adjusted

versions of the previous solution

Professor A G Constantinides© 18
AGC
DSP Levinson-Durbin
 The last element follows from the
second equation of
 Rk J k rk*  w k   rk 
 T      r 
rk J k r0   k   k 1 
 Ie
1
 k  (rk 1  rkT J k z k )
r0

Professor A G Constantinides© 19
AGC
DSP Levinson-Durbin

 The parameters  k are known as the

reflection coefficients.
 These are crucial from the signal
processing point of view.

Professor A G Constantinides© 20
AGC
DSP Levinson-Durbin

 The Levinson algorithm solves the

problem
R my  b
 In the same way as for Durbin we keep
track of the solutions to the problems
R k y k  bk

Professor A G Constantinides© 21
AGC
DSP Levinson-Durbin

 Thus assuming w k , y k to be known

at the k step, we solve at the next step
the problem
 Rk J r *
  vk   bk 
     b 
k k
 T
rk J k r0   k   k 1 

Professor A G Constantinides© 22
AGC
DSP Levinson-Durbin

 vk 
 Where y k 1   
 k 
 Thus
1
 v k  R k (b k   k J k rk )  y k   k J k y k
* *

T
bk 1  rk J k y k
k  T *
r0  rk y k
Professor A G Constantinides© 23
 AGC
DSP Lattice Predictors
 Return to the lattice case.
 We write
Ab ( z )
TM ( z ) 
Af ( z)
 or
aM [ M ]  aM [ M  1]z 1  aM [ M  2]z 2  ...  z  M
TM ( z ) 
1  a1[1]z 1  aM [2]z 2  ...  aM [ M ]z  M
Professor A G Constantinides© 24
AGC
DSP Lattice Predictors
 The above transfer function is allpass of order M.
 It can be thought of as the reflection coeffient of
a cascade of lossless transmission lines, or
acoustic tubes.
 In this sense it can furnish a simple algorithm for
the estimation of the reflection coefficients.
 We strat with the observation that the transfer
function can be written in terms of another
allpass filter embedded in a first order allpass
structure

Professor A G Constantinides© 25
AGC
DSP Lattice Predictors
 This takes the form
 1  z 1TM 1 ( z )
TM ( z ) 
1   1 z 1TM 1 ( z )
 Where  1 is to be chosen to make TM 1 ( z )
of degree (M-1) .
 From the above we have
TM ( z )   1
TM 1 ( z )  1
z (1   1TM ( z ))
Professor A G Constantinides© 26
AGC
DSP Lattice Predictors
 And hence
(aM 1[ M ]  aM 1[ M  1]z 1  ...  z  M
TM ( z )  1
z (1  aM 1[1]z 1  aM 1[2]z 2  ...  aM 1[ M ]z  M )
 Where aM [r ]   1aM [ M  r ]
aM 1[r ] 
1   1aM [ M ]
 Thus for a reduction in the order the constant
term in the numerator, which is also equal to
the highest term in the denominator, must be
zero.
Professor A G Constantinides© 27
AGC
DSP Lattice Predictors
 This requirement yields  1  aM [ M ]
 The realisation structure is

1
TM (z )

1
TM 1 ( z )
z
Professor A G Constantinides© 28
AGC
DSP Lattice Predictors
 There are many rearrangemnets that can be
made of this structure, through the use of Signal
Flow Graphs.
 One such rearrangement would be to reverse
the direction of signal flow for the lower path.
This would yield the standard Lattice Structure
as found in several textbooks (viz. Inverse
Lattice)
 The lattice structure and the above development
are intimately related to the Levinson-Durbin
Algorithm
Professor A G Constantinides© 29
AGC
DSP Lattice Predictors
 The form of lattice presented is not the usual
approach to the Levinson algorithm in that we
have developed the inverse filter.
 Since the denominator of the allpass is also
the denominator of the AR process the
procedure can be seen as an AR coefficient to
lattice structure mapping.
 For lattice to AR coefficient mapping we
follow the opposite route, ie we contruct the
allpass and read off its denominator.
Professor A G Constantinides© 30
AGC
DSP PSD Estimation
 It is evident that if the PSD of the prediction
error is white then the prediction transfer
function multiplied by the input PSD yields a
constant.
 Therefore the input PSD is determined.
 Moreover the inverse prediction filter gives us
a means to generate the process as the
output from the filter when the input is white
noise.
Professor A G Constantinides© 31