68 Adaline and Madaline

' a system has converged to its minimum error solution. Alternatively, observing
individual errors is often necessary, since the system performance may have
a requirement that no error exceed a certain amount. Nevertheless, a mean
squared error that falls as the iteration number increases is probably your best
indication that the system is converging toward a solution.
We usually assume that the input signals are statistically stationary, and,
therefore, (d .) is essentially a constant after the optimum weigh! values have
been dete1111i11ed. During training, (c I, ) will hopefully decrease toward a stable
solution. Suppose, however, that the input signal statistics change somewhat
over time, or undergo some discontinuity: Additional training would be required
to compensate.
One way to deal with this situation is to cease or resume training con-
ditionally, based on the current value of (cU . If the signal statistics change,
training can be reinitiated until (d. ) is again reduced to an acceptable value.
This method presumes that a method of error measurement is available.
Provided that the input signals are statistically stationary, choosing the num-
ber of input vectors to use during training may be relatively simple. You can
use real, time-sequenced inputs as training vectors, provided that you know the
desired output for each input vector. If it is possible to identify a sample of
input vectors that adequately reproduces the statistical distribution of the ach1al
inputs, it may be possible to train on this set in a shorter time. The accuracy
of the training depends on how well the selected set of training vectors models
the distribution of the entire input signal space.
The other, related question is how to go about determining the desired
output for a given input vector. As with many questions discussed in this
section, this depends on the specific details of the problem. Fortunately, for
some problems, knowing the desired result is easy compared to finding an
algorithm for transforming the inputs into the desired result. The ALC will
often solve the difficult part. The "easy" part is left to the engineer.
Exercise 2.4: A lowpass filter can be constructed with an Adaline having two
weights. Consider a simple case of the removal of a random noise from a
constant signal. The constant signal level is C = 3, and the random noise
signal has a constant power, (r 2 ) - n - 0.025. Assume that the random noise
is completely uncorrelated with the constant input signal. Calculate the optimum
weight vector and the mean squared error in the output after the optimum weight
vector has been found. By finding the eigenvalues of the matrix, R, determine
the maximum value of the constant µ for use in the LMS algorithm.

2.3 APPLICATIONS OF ADAPTIVE

SIGNAL PROCESSING
Up to now, we have been concerned with the Adaline minus the threshold
condition on the output. In Section 2.4, on the Madaline, we will replace the
threshold condition and examine networks of Adalines. In this section, we will
 2.3 Applications of Adaptive Signal Processing 69

look at a few examples of adaptive signal processing using only the ALC portion
of the Adaline.

2.3.1 Echo Cancellation in Telephone Circuits

You may have experienced the phenomenon of echo in telephone conversations:
you hear the words you speak into the mouthpiece a fraction of a second later
in the earphone of the telephone. The echo tends to be most noticeable on long-
distance calls, especially those over satellite links where transmission delays can
be a significant fraction of a second.
Telephone circuits contain devices called hybrids that are intended to
isolate incoming signals from outgoing signals, thus avoiding the echo effect.
Unfortunately, these circuits do not always perform perfectly, due to causes such
as impedance mismatches, resulting in some echo back to the speaker. Even
when the echo signal has been attenuated by a substantial amount, it still may
be audible, and hence an annoyance to the speaker.
Certain echo-suppression devices rely on relays that open and close circuits
in the outgoing lines so that incoming voice signals are not sent back to the
speaker. When transmission delays are long, as with satellite communications,
these echo suppressors can result in a loss of parts of words. This choppy-
speech effect is perhaps more familiar than the echo effect. An adaptive filter
can be used to remove the echo effect without the choppiness of the relays used
in other echo suppression circuits [9, 71.
Figure 2. 15 is a block diagram of a telephone circuit with an adaptive
filter used as an echo-suppression device. The echo is caused by a leakage of
the incoming voice signal to the output line through the hybrid circuit. This
leakage adds to the output signal coming from the microphone. The output of
the adaptive filter, :i;, is subtracted from the outgoing signal, s + n', where s is
the outgoing pure voice signal and n' is the noise, or echo caused by leakage of
the incoming voice signal through the hybrid circuit. The success of the echo
cancellation depends on how well the adaptive filter can mimic the leakage
through the hybrid circuit.
Notice that the input to the filter is a copy of the incoming signal, 11, and
that the error is a copy of the outgoing signal,
E =s + n' - y (2.16)
We assume that y is correlated with the noise, n', but not with the pure voice
signal, s. If the quantity, n ' - y, is nonzero, some echo still remains in the out-
going signal. Squaring and taking expectation values of both sides of Eq. (2. I 6)
gives
(c: 2 ) (.,/ ) + ((n' - 2
·y ) ) + 2(s(n' - y)) (2.17)
- (:/) + ((n' - y}2 ) (2.18)
Equation {2.18) follows, since s is not correlated with either y or n', resulting
in the last term in Eq. (2.17) being equal to zero.

L
 70 Adaline and Madaline
r
Voice 5 +n' E ==S +n'- >".,----~
signal, ~_;__-•+~ L. 1----~--1~\ay 1',..,.....-----~---► To
s A earphone
noise, n' I Y
Hybrid Adaptive Adaptive Hybrid
circuit filter filter circuit

Voice
To
signal
earphone

Figure 2.15 This figure is a schematic of a telephone circuit using an

adaptive filter to cancel echo. The adaptive filter is depicted
as a box; the slanted arrow represents the adjustable weights.

The signal power, {.s2}, is determined by the source of the voice signaJ-
say, some amplifier at the telephone switching station local to the sender. Thus,
(.ri) is not directly affected by changes in (c:2 ). The adaptive filter attempts
to minimize (c 2 ), and, in doing so, minimizes ((11 1 - y)2 ), the power of the
uncanceled noise on the outgoing line.
Since there is only one input to the adaptive filter, the device would be
configured as a transverse filter. Widrow and Steams [9] suggest sampling the
incoming signal at a rate of 8 KHz and using 128 weight values.

2.3.2 Other Applications

Rather than go into the details of the many applications that can be addressed by
these adaptive filters, we refer you once again to the excellent text by Widrow
and Stearns. In this section, we shall simply suggest a few broad areas where
adaptive filters can be used in addition to the echo-cancellation application we
have discussed.
Figure 2. 16 shows an adaptive filter that is used to predict the future value of
a signal based on its present value. A second example is shown in Figure 2.17.
In this example, the adaptive filter learns to reproduce the output from some
plant based on inputs to the system. This configuration has many uses as an
adaptive control system. The plant could represent many things, including a
human operator. In that case, the adaptive filter could learn how to respond to
changing conditions by watching the human operator. Eventually, such a device
might result in an automated control system, leaving the human free for more
.
unportan t tas ks. 2
Another useful application of these devices is in adaptive beam-forming
antenna arrays. Although the term antenna is usually associated with electro-

2Such as training another adapti, e filter with the Standard & Poors 500.
 2.3 Applications of Adaptive Signal Processing 71

Current signal

Prediction of
y I-----+-----►
current signal
Delay
Adaptive
filter

Past signal
Figure 2.16 This schematic shows an adaptive filter used to predict signal
values. The input signal used to train the network is a delayed
value of the actual signal; that is, it is the signal at some past
time. The expected output is the current value of the signal.
The adaptive filter attempts to minimize the error between its
output and the current signal, based on an input of the signal
value from some time in the past. Once the filter is correctly
predicting the current signal based on the past signal, the
current signal can be used directly as an input without the
delay. The filter w ill then make a prediction of the future
signal value.

Input signals ~-~

Plant

Prediction of
y I-----+---~~
plant output
Adaptive
filter

Figure 2.17 This example shows an adaptive filter used to model the
output from a system, called the plant. Inputs to the filter are
the same as those to the plant The filter adjusts its weights
based on the difference between its output and the output of
the plant
I