International Journal of Innovative

Computing, Information and Control ICIC International ⃝2011

c ISSN 1349-4198
Volume 7, Number 6, June 2011 pp. 3407–3417

SPATIAL-BASED ADAPTIVE ITERATIVE LEARNING CONTROL

OF NONLINEAR ROTARY SYSTEMS WITH SPATIALLY
PERIODIC PARAMETRIC VARIATION

Yen-Hsiu Yang1 and Cheng-Lun Chen2

1
Inventec Corporation
66 Hou-Kang Street, Shin-Lin District, Taipei 111, Taiwan
yenhyang@gmail.com
2
Department of Electrical Engineering
No.250 Kuo Kuang Rd, Taichung 40227, Taiwan
chenc@dragon.nchu.edu.tw
Received January 2010; revised August 2010

Abstract. A new spatial-based iterative learning control design for a class of rotary
systems with unknown spatially periodic parameters is proposed in this paper. The concept
of parametric adjustment in conventional adaptive control is modified and interfaced with
spatial-based iterative learning control, i.e., a periodic parametric update law is identified
for minimization of the tracking error. Convergence property and stability proof of the
overall system are analyzed and discussed. Feasibility and effectiveness of the proposed
scheme is justified by an illustrative example with numerical simulation. Compared with
existing spatial-based iterative learning designs, the proposed approach applies to a more
generic class of nonlinear and high-order systems.
Keywords: Adaptive control, Iterative learning control, Motion control

1. Introduction. Motion systems play important roles in various industry, e.g., pack-
aging, printing, assembly, fabrication, semiconductor, robotics, etc. A conspicuous con-
figuration of motion control systems comprises actuators and sensors to regulate and
measure the position and velocity of the load, and a so-called controller to interface mea-
sured/command signals from/to the sensors and actuators. Most applications require
a motion system to operate at variable speeds whereas following pre-specified periodic
trajectories and rejecting/reducing periodic fluctuations. For example, a two-degree-of-
freedom control configuration for a novel omnidirectional wheelchair is proposed in [1]; an
obstacle avoidance control for a two wheeled mobile robot based on an improved support
vector regression technique was introduced by [2]. A category of motion systems is rotary
system, which features a typical repetitive operation. The rotary components within a
rotary system will return to the same angular position after revolving integer cycles or
revolutions. An immediate impact is that most uncertainties (disturbances or unknown
parameters) are position-invariant (as opposed to time-invariant) and become spatially
periodic with respect to the angular position. Note that such periodicity is invariant to
the operating speed of the system.
The increasing complexity in architecture and the high-performance requirement of
recent motion systems have posed major challenge on conceiving and synthesizing a de-
sirable control algorithm. Various types of temporal-based motion control algorithms have
been developed for decades. Adaptive control [3-6] is capable of dealing with systems sub-
ject to unknown but time-invariant parameters. Iterative learning control [7-12], on the
other hand, has been applied to systems subject to periodic disturbances or required to

3407
3408 Y.-H. YANG AND C.-L. CHEN

track periodic references. Lately, merging of adaptive and iterative learning control has
been studied by several researchers (e.g., [13] and the references therein), which integrates
the parametric adaptation (in adaptive control) with the learning algorithm (in iterative
learning control). Advantages such as attaining perfect tracking over finite time, tackling
time-varying parameters, and non-resetting of initial condition, are immediately acquired
through the integration. As pointed out by [15-16], for rotary systems operating at vari-
able speeds, most temporal-based control algorithms do not utilize the fact that most
uncertainties are spatially periodic. When analyzing and designing the control system in
time domain, those spatially periodic disturbances or parameters can be falsely recognized
as non-periodic time-varying ones. This inevitably leads to a system design with either
complicated time-varying controller or with compromised performance.
Recently, spatial-based control algorithms have been developed by several researchers,
which reformulate the open-loop systems into the ones in spatial domain and then per-
form the corresponding spatial-based controller design. Since those spatially related un-
certainties are rendered stationary in spatial domain regardless of the operating speed,
position-invariant controllers suffice in achieving the desired performance. After an ini-
tiative design by [14] and the subsequent advancement by [15-16], spatial-based repetitive
control is sophisticated enough to tackle a class of uncertain nonlinear systems. On the
contrary, existing spatial-based iterative learning controls [17-18] are still rudimentary
and focus on linear open-loop systems. It is not perspicuous whether those design meth-
ods are upgradable or applicable to generic nonlinear and high-order systems. Moreover,
spatially related uncertainties in rotary systems may be modeled as periodic disturbances
or periodic parameters. Despite that modeling the uncertainties as periodic parameters
are not uncommon [19-21], treating the uncertainties as disturbances seem to be more
prevailing in existing control literatures.
In this paper, we propose a new spatial-based iterative learning control, which is appli-
cable to a generic class of spatial-based nonlinear rotary systems with unknown spatially
periodic parameters. In a fundamental sense, the proposed design incorporates parametric
adaptation in adaptive control into iterative learning control. However, the integration is
not trivial since stability property and tracking performance of the overall spatial-based
control system need to be further analyzed and justified. A desired control input and
periodic parametric update law are determined based on specifying a sensible Lyapunov-
Krasovskii functional and rendering its derivative negative semi-definite. Note that the
proposed integration is distinct from that in [13], which is a (discrete) time-based design
and applicable to a different class of systems. Design of the control input and paramet-
ric update law, and stability/convergence analysis adopted in this study also differ from
that in [13]. Furthermore, unlike conventional adaptive control, the suggested periodic
parametric update law can deal with unknown parameters of constant or arbitrarily fast
variation.
The next section starts with transforming the model of a class of nonlinear time-
invariant rotary systems from time domain to spatial domain. Assumptions and defi-
nitions are then given in Section 3. Derivation of the control law and the parametric up-
date law along with the main theorem with stability and convergence proof is described
in Section 4. An example is provided in Section 5 to illustrate the design procedure.
Conclusions with discussion are given in Section 6.

2. Problem Statement. Consider a nonlinear time-invariant (NTI) system described

by
ẋ(t) = ft (x, φf (θ)) + Bc gt (x, φg (θ)) u(t), y(t) = Ψx(t) (1)
 CONTROL OF NONLINEAR ROTARY SYSTEMS 3409

where
[ ]T [ ]
x(t) = x1 (t) · · · xn (t) , Ψ= 0 ··· 0 1
[ ]T
Bc = 0 ··· 0 1
[ ]
y(t) is the system output, u(t) is the control input, φf (θ) = φ1 (θ) · · · φp (θ) and
φg (θ) are system parameters which are periodic with respect to angular position θ (i.e.,
spatially periodic). Instead of using time (t), we choose angular displacement (φ) as the
independent variable, i.e.,
∫ t
φ(t) = |ω (τ )| dτ +φ (0)
0

where ω(t) is the angular velocity of the system. Alternatively, if the system always
rotates in the same direction, angular position is equal to the angular displacement so
that
∫ t
θ = λ(t) = ω (τ ) dτ +θ (0)
0

Note that monotonically increasing of λ(t) with respect to time ensures the existence
of the inverse mapping, i.e., t = λ−1 (θ). We may define change of coordinate as follows
( ) ( )
x̂ (θ) = x λ−1 (θ) , ŷ (θ) = y λ−1 (θ)
( ) ( )
û (θ) = u λ−1 (θ) , dˆ(θ) = d λ−1 (θ)
( ) dx̂ (θ) dθ dx̂ (θ) dx(t)
ω̂ (θ) = ω λ−1 (θ) , ω̂ (θ) = =
dθ dt dθ dt
Hence, all variables in the time domain now have their corresponding variables in
the angular position domain (or θ-domain). Furthermore, we may define the Laplace
transform in θ-domain as ∫ ∞
Ĝ (s̃) = ĝ (τ ) e−s̃θ dθ
0

Therefore, the concept of transfer function stays valid for linear position-invariant (LPI)
system.
Using the aforementioned change of coordinate, we may transform (1) in the time
domain into
ω̂ (θ) x̂˙ (θ) = ft (x̂, φf (θ)) + Bc gt (x̂, φg (θ)) û (θ)
(2)
ŷ (θ) = Ψx̂ (θ)
in the θ-domain. If ω̂ (θ) is equal to one of the state variables, (2) can be regarded as a
nonlinear position-invariant (NPI) system in the θ-domain.
Remark 2.1. As mentioned previously, uncertainties for rotary systems may be modeled
as periodic disturbances or periodic parameters. Due to capability of most control algo-
rithms, treating the uncertainties as disturbances seem to be more prevailing in existing
literatures. However, literatures which modeled the uncertainties as periodic parameters
are not uncommon. Hence, periodic parametric variation is, in fact, a sensible assump-
tion.

3. Definitions and Assumptions. In this section, we list and present the definitions
and assumptions to be used in the subsequent sections.
3410 Y.-H. YANG AND C.-L. CHEN

Definition 3.1. (Lie derivative) The Lie derivative is defined as

∂h
L0f h(x) = h(x), Lf h(x) = f (x),
∂x
∂ (Lf h)
L2f h(x) = Lf Lf h(x) = f (x), · · ·
∂x
∂ (Lf h)
Lg Lf h(x) = g(x), · · ·
∂x

Definition 3.2. (Diffeomorphism) A diffeomorphism is considered as a mapping T (·) :

D ⊂ Rn → Rn which is continuously differentiable on D and has a continuously differen-
tiable inverse T −1 (·).
Definition 3.3. (Adaptation rate) Instead of using constant adaptation rate in conven-
tional adaptive control, a varying adaptation rate will be used. Consider a matrix Γ (θ, φc )
defined by



0, θ = 0
Γ (θ, φc ) = α (θ) , 0 < θ < φc (3)


β, φc ≤ θ
{ }
where φc is the lowest common multiple of the parametric periods, β = diag β1 ··· β
{ } ℓ
with nonzero positive constant βi , and α (θ) = diag α1 (θ) · · · αℓ (θ) with αi (θ) a
strictly increasing function, αi (0) = 0, and αi (φc ) = βi .
Assumption 3.1. The desired trajectory (or reference command signal) ym is sufficiently
(n) (n−1)
smooth or ym , ym , · · · , ẏm exist.
Assumption 3.2. For a θ-domain NPI system described by
x̂˙ (θ) = f (x̂ (θ) , φf (θ)) + Bc g (x̂ (θ) , φg (θ)) û (θ)
ŷ (θ) = Ψx̂ (θ)
the nonlinear functions f (x̂ (θ)) and g (x̂ (θ)) are assumed to be linearly related to the
system parameters φf and φg , i.e.,
∑
p
f (x̂ (θ) , φf (θ)) = φi (θ) fi (x̂ (θ)) ,
i=1
g (x̂ (θ) , φg (θ)) = φg (θ) g (x̂ (θ))

Remark 3.1. Assumption 1 may be easily satisfied by specifying a reference trajectory

which does not have sudden change of slope (e.g., the one in Figure 2), and Assumption
2 can actually be satisfied by various systems, i.e., all linear time-invariant systems and
a class of nonlinear systems.

4. Spatial-based Adaptive Iterative Learning Control. In this section, we proceed

to design the spatial-based adaptive iterative learning control (AILC) system. Figure
1 illustrates the block configuration for the proposed spatial-based AILC system. The
unknown parameters will be estimated periodically by a parametric adaptation law. The
estimated parameters and the tracking error are then utilized in generating the control
input. An appropriate Lyapunov-Krasovskii functional will be specified when deriving
the control input and the parametric update law. This functional also facilitates the
subsequent stability analysis and proof.
 CONTROL OF NONLINEAR ROTARY SYSTEMS 3411

ŷ m
Reference

xˆɺ = f (xˆ , φf ) + Bc g (xˆ , ϕg )uˆ ŷ

Controller
yˆ = ω = Ψxˆ
Plant

Iterative
parametric
update

Adaptive Iterative learning control

Figure 1. Control structure of the overall AILC system

6.5

rev/s

5.5

4.5
0 20 40 60 80 100
time (sec)

Figure 2. The reference command

For tidy presentation, the θ notation will be dropped from most of the equations in the
sequel. Rewrite (2) as
x̂˙ = f (x̂, φf ) + Bc g (x̂, φg ) û, ŷ = ω̂ = Ψx̂ (4)
where the output ŷ is equal to the angular velocity ω̂, which is set to be the first state of
the system. Also note that
f (x̂, φf ) = ft (x̂, φf )/x̂1 and g (x̂, φg ) = gt (x̂, φg )/x̂1
The system (4) is valid within the set D0 = { x̂ ∈ R| x̂1 ̸= 0}. Within this set, a
diffeomorphism T (x̂) : D0 ⊂ D (as defined previously) exists and may be described by
[ ]T
ẑ = T (x̂) = L0f h (x̂) Lf h (x̂) · · · Lfn−1 h (x̂) (5)
[ ]T
where ẑ = ẑ1 · · · ẑn . Further properties regarded T (x̂) can be found in [22]. Using
(5), we may transform (4) into
[ ]
ẑ˙ = Ac ẑ + Bc Lnf h (x̂) + Lg Ln−1
f h (x̂) û x̂=T −1 (ẑ) , ŷ = ẑ1 (6)
3412 Y.-H. YANG AND C.-L. CHEN

where [ ]
0 I(n−1)×(n−1)
Ac =
0 01×(n−1)
According to Assumption 3.2, we may rewrite (6) as
[ ]
ẑ˙ = Ac ẑ + Bc ρ (ẑ) + ΘT Wf (ẑ) + φg Wg (ẑ) û , ŷ = ẑ1 (7)
[ ]T
where Θ = φ1 · · · φp φg · · · corresponds to the actual parametric vector, φg is
a parameter mapped via the diffeomorphism, Wf (ẑ) is a vector of nonlinear terms, ρ (ẑ)
and Wg (ẑ) are two nonlinear functions.
Consider a desired or reference trajectory ym (t) satisfying Assumption 3.1. Similarly,
ym (t) may be transformed into its counterpart in the θ-domain, i.e.,
( )
ŷm (θ) = ym λ−1 (θ) = ym (t)
Define another state or coordinate transformation:
ẑ1r = ŷm (θ) , ẑ2r = ŷ˙ m (θ) , · · · , ẑnr = ŷm
(n−1)
(θ)
We may form a state space model, which produces the reference trajectory, as
ẑ˙r = Ac ẑr + Bc ŷm
(n)
(8)
[ ]T
where ẑr = ẑ1r · · · ẑnr . Define the tracking error as ê = ẑ − ẑr . Then, the error
dynamics can be obtained using the first equation of (7) and (8), i.e.,
[ ]
ê˙ = Aê + Bc ΘT Wf + ρ + σ − ŷm (n)
+ φg Wg û (9)
[ ]
where σ = cê with c = c1 · · · cn−1 1 , and
[ ]
0(n−1)×1 I(n−1)×(n−1)
A=
c
Next, specify a Lyapunov-Krasovskii functional (LKF) as
∫ θ
/
2
V = σ 2φg + 1/2 ΦT (τ ) β −1 Φ (τ )dτ (10)
θ−φc

where Φ = Θ̄ − Θ̄˜ and Θ̄ is a vector of parameters to be defined later. Θ̄

˜ is the estimate
of Θ̄. The objective for the subsequent steps is to determine a suitable control input and
parametric update law such that the derivative of the LKF becomes negative semi-definite.
Calculating the derivative of V , we have
V̇ = V1 + V2 (11)
where
σ ˙ σ2
V1 = cê − 2 φ̇g
φg 2φg
[ T ]
V2 = 1/2 Φ (θ) β −1 Φ (θ) − ΦT (θ − φc ) β −1 Φ (θ − φc )
Substituting the error dynamics (9) into V1 and recalling that σ = cê, we have
[ ]
1 ( ) ΘT φ̇g σ 2
V1 = σ c̄ê + ρ − ŷm +
(n)
Wf − + Wg û (12)
φg φg φg 2
[ ]
where c̄ = 0 c1 · · · cn−1 . Hence, we may specify û as
( )
û = −1/Wg kσ + Θ̄ ˜TW (13)
 CONTROL OF NONLINEAR ROTARY SYSTEMS 3413

˜ is the corresponding estimate of

where k is a positive variable, Θ̄
[ T
]T [ ( ) ]T
φ̇ (n) 2
Θ̄ = φ1g Θφg φgg , and W = c̄ê + ρ − ŷm Wf − σ2
This will simplify V1 , i.e.,
V1 = −kσ 2 + σΦT W (14)
Using the periodicity of Θ̄ (θ) = Θ̄ (θ − φc ), We may rewrite V2 as
[( )T ( ) ( )T ( )]
˜
V2 = 1/2 Θ̄ − Θ̄ β −1 ˜ ˜
Θ̄ − Θ̄ − Θ̄ − Θ̄ (θ − φc ) β −1 ˜
Θ̄ − Θ̄ (θ − φc ) (15)

According to the following algebraic relationship,

(a − b)T β (a − b) − (a − c)T β (a − c) = 1/2 (c − b)T β [2 (a − b) + (b − c)]
where a, b, and c are vectors, (15) implies that
( ) [ ( ) ( )]
V2 = 1/2 Θ̄ ˜ T β −1 2 Θ̄ − Θ̄
˜ (θ − φ ) − Θ̄ ˜ + Θ̄˜ − Θ̄
˜ (θ − φ ) (16)
c c

Therefore, we may specify a periodic parametric update law as

˜ (θ) = Θ̄
Θ̄ ˜ (θ − φ ) + Γ (θ, φ ) W σ
c c
(17)
˜ (θ) = 0,
Θ̄ −φc ≤ θ ≤ 0
Recall that Γ (θ, φc ) is the adaptation rate as defined in (3). For φc ≤ θ, V2 becomes
V2 = −σΦT W − 1/2σ T W T βW σ (18)
With (14) and (18), we conclude that
V̇ = −kσ 2 − 1/2σ T W T βW σ ≤ −kσ 2 (19)
The objective is achieved. We summarize the main results of this paper in the following
theorem.
Theorem 4.1. Consider a spatial-based nonlinear system (4) with spatially periodic pa-
rameters satisfying Assumption 3.2. Under Assumption 3.1, the error dynamics as de-
scribed by (9) exist. Suppose that the control input is specified by (13) along with the
periodic parametric adaptation law by (17). Then the tracking error ê will converge to 0
with the following performance characteristics:
∫ θ
∥ê∥2 dτ → 0, as θ → ∞
θ−φc

Proof: The stability proof consists of two steps. The first step is to show that the
tracking error is bounded within the interval 0 ≤ θ ≤ φc . The second step is to show
that the tracking error will converge to zero as θ → ∞ (for φc ≤ θ) and abides by the
indicated performance characteristics.
Step 1 0 ≤ θ ≤ φc
Within the interval 0 ≤ θ ≤ φc , the parametric adaptation law (17) reduces to
˜ (θ) = α (θ) W σ
Θ̄ (20)
Showing the boundedness of the tracking error ê within this interval is equivalent to
showing that V is finite or V̇ < 0 with the aforementioned control input and parametric
adaptation laws. Define the LKF for 0 ≤ θ ≤ φc as
∫ φc
/
2
V = σ (2φg ) + 1/2 ΦT (τ ) β −1 Φ (τ )dτ (21)
0
3414 Y.-H. YANG AND C.-L. CHEN

With (14) and (20), the derivative of V simplifies to

˜ + 1/2ΦT β −1 Φ
V̇ = −kσ 2 + ΦT α−1 Θ̄ (22)
˜ , (22) becomes
With the definition of Φ = Θ̄ − Θ̄
( )
V̇ = −kσ 2 − ΦT α−1 − 1/2β −1 Φ + ΦT α−1 Θ̄ (23)
where α−1 − 1/2β −1 > 0 (since αi is a strictly increasing function with upper bound βi ).
Applying the following inequality
ΦT α−1 Θ̄ ≤ ΦT Cα−1 Φ + 1/4Θ̄T Cα−1 Θ̄ (24)
{ }
where C = diag c1 · · · cℓ is an adjustable constant matrix, to (23), we have
( )
V̇ ≤ −kσ 2 − ΦT α−1 − 1/2β −1 − Cα−1 Φ + 1/4Θ̄T Cα−1 Θ̄
Choose a suitable C such that
/
α−1 − β −1 2 − Cα−1 < 0
Then, V̇ < 0 will hold in the following region
2
−k ∥σ∥2 − λ1 ∥Φ∥2 + λ2 Θ̄ ≤0
where λ1 is the minimum eigenvalue of α−1 − β −1 /2 − Cα−1 , and λ2 is the maximum
eigenvalue of Cα−1 /4. Hence, we conclude that V is finite for 0 ≤ θ ≤ φc , i.e., ê is
bounded for 0 ≤ θ ≤ φc .
Step 2 φc ≤ θ
Form a sequence θi → ∞, i = 1, 2, · · · with θi > φc and θi+1 ≥ θi + φ. From (19), we
have
∑i ∫θj
lim V (θ) ≤ V (φc ) − lim ∥kσ∥2 dτ (25)
θ→∞ i→∞
j=1
θj −φc

Since V (φc ) is finite as having been proved in Step 1, and V (θ) is positive, (25) implies
that
∫θ
lim ∥ê∥2 dτ → 0
θ→∞
θ−φc

Hence, the tracking error will converge to zero as θ → ∞ and adhere to the indicated
performance characteristics.

5. Illustrative Example. A simulation configuration is set up as shown in Figure 3, for

which the controller and parametric adaptation will operate in the θ-domain whereas the
open-loop system will operate in the time-domain. We assume that an optical encoder
is mounted on the rotary component of the open-loop system. The encoder pulses not
only provides measurement of the angular velocity, but also serves as triggering interrupt
for operating the controller and parametric adaptation in the θ-domain. The trigger
frequency (i.e., spatial sampling frequency) depends on the number of encoder pulses per
revolution and the angular velocity. We adopt an encoder of 1000 pulses/rev for this
example.
The proposed spatial-based AILC scheme is applied to a rotary system modeled by
b0 (θ)
Y (s) = U (s) (26)
s3 + a1 (θ) s2 + a2 (θ) s + a3 (θ)
 CONTROL OF NONLINEAR ROTARY SYSTEMS 3415

Time domain

Ӱ
Encoder
Trigger

Controller Plant
û (θ ) u (t ) y (t )

Reference
Ӱ- domain y m (t )

Figure 3. Configuration for simulation verification

where Y (s) is the output angular velocity, U (s) is control input voltage, and a1 (θ), a2 (θ),
a3 (θ), and b0 (θ) are spatially periodic parameters with
a1 (θ) = 3 + 0.01 sin(0.5θ), a2 (θ) = 4 + 0.01 sin(θ)
a3 (θ) = 7 + 0.01 sin(0.25θ), b0 (θ) = 5 + 0.01 sin(θ)
(26) may be further represented by an observable state space model, i.e.,
ẋ(t) = A (θ) x(t) + B (θ) u (t) , y = Ψx(t) (27)
[ ]T [ ]
where x(t) = x1 x2 x3 , Ψ= 1 0 0 .
   
−a1 (θ) 1 0 0
A (θ) =  −a2 (θ) 0 1  , B (θ) =  0 
−a3 (θ) 0 0 b0 (θ)
First, follow the steps described in Section 2 and transform (27) into an open-loop
system in spatial domain, i.e.,
x̂˙ = f (x̂) + Bc g (x̂) u(t)
(28)
ŷ = Ψx̂
where
 
[ ]T −a1 + x̂x̂21
b0
x̂ (θ) = x̂1 (θ) x̂2 (θ) x̂3 (θ) , f (x̂) =  −a2 + x̂x̂3  , g (x̂) =
1 x̂1
−a3
Using the diffeomorphism defined by (5), we change (28) into
ẑ˙1 = ẑ2 , ẑ˙2 = ẑ3 , ẑ˙3 = ΘT Wf + ρ + b0 Wg û
[ ]T
where Θ = ä1 ȧ1 a1 ȧ2 a2 a3 ,
 
−1
 −2ẑ −1 ẑ − z Λ 
 1 2 1 
 
 −2ẑ1 ẑ2 −
−1 2 d(z 1 Λ) 
Wf =   −1
dθ  , Λ = ẑ1−2 ẑ2

 −ẑ1 
 −2 
 −ẑ1 ẑ2 
−2
−ẑ1
3416 Y.-H. YANG AND C.-L. CHEN

ρ = −2ẑ1−2 ẑ23 − 2ẑ1−1 ẑ2 ẑ3 − ẑ1 ẑ2 Λ̇ − ẑ22 Λ + ẑ1 ẑ3 Λ
and Wg = ẑ1−3 . Thus, the control input and the parametric adaptation law may be
specified by (13) and (17). The reference trajectory ym (t), a periodic square wave with
smooth corner, is depicted in Figure 2. The tracking performance and the control input
are shown in Figure 4. We see that the proposed AILC system is stable and leads to small
tracking error (with magnitude order of 10−3 ) and requires a control input of acceptable
level (less than 15).

-3
x 10
0

-0.5
y-y (rev/s)

-1
m

-1.5

-2
0 50 100 150 200 250 300 350 400 450

15
u (voltage)

10

0
0 50 100 150 200 250 300 350 400 450
angular displacement (revolution)

Figure 4. Tracking performance and control input

6. Conclusions. This paper proposed a novel spatial-based AILC scheme for a class
of nonlinear rotary systems as given by (1). Stability and convergence properties of
the overall system are proved and analyzed. For future work, the control scheme may
be modified and extended to cope with other types of nonlinear systems, e.g., systems
with aperiodic parameters or subject to external and state-dependent disturbances (e.g.,
friction). Despite mentioning that rotary systems are the main application, this study
focuses more on theoretical derivation and development of the proposed control scheme. It
should be noted that the proposed algorithm is not immediately applicable to a practical
system before certain issues are tackled properly.

Acknowledgment. This work is supported by the National Science Council (Taiwan)

under grant NSC98-2221-E-005-043.

REFERENCES

[1] S. Wang, A new omnidirectional wheelchair and its motion control method, ICIC Express Letters,
vol.4, no.1, pp.289-294, 2010.
[2] L. Jiang, M. Deng and A. Inour, Obstacle avoidance and motion control of a two wheeled mo-
bile robot using SVR technique, International Journal of Innovative Computing, Information and
Control, vol.5, no.2, pp.253-261, 2009.
[3] R. H. Middleton and G. C. Goodwin, Adaptive control of time-varying linear systems, IEEE Trans.
Automatic Control, vol.33, pp.150-155, 1988.
[4] A. M. Annaswamy and K. S. Narendra, Adaptive control of simple time-varying systems, Proc. of
the 28th IEEE Conf. Decision and Control, Tampa, FL, pp.1014-1018, 1989.
 CONTROL OF NONLINEAR ROTARY SYSTEMS 3417

[5] R. Marino and P. Tomei, Adaptive control of linear time-varying systems, Automatica, vol.39,
pp.651-659, 2003.
[6] P. Qin, X. Han and Y. Lin, Convergence analysis about improved model-free adaptive controller
based on multi-innovation theory, vol.3, no.3, pp.301-306, 2009.
[7] K. L. Moore, Iterative Learning Control for Deterministic System, Springer, New York, 1992.
[8] Y. Chen and C. Wen, Iterative Learning Control: Convergence, Robustness and Application,
Springer, New York, 1999.
[9] J.-X. Xu and Y. Tan, Linear and Nonlinear Iterative Learning Control, Springer, New York, 2003.
[10] H.-S. Ahn and C. H. Choi, Iterative learning controller for linear system with a periodic distur-
bances, Electronics Letters, vol.26, no.18, pp.1542-1544, 1990.
[11] T. Sugie and T. Ono, Iterative learning control law for dynamical systems, Automatica, vol.27,
no.4, pp.729-732, 1991.
[12] K. L. Moore et al., Iterative learning control: a survey and new results, Journal of Robotic Systems,
vol.9, no.5, pp.563-594, 1992.
[13] R. Chi et al., A new adaptive iterative learning control motivated by discrete-time adaptive control,
International Journal of Innovative Computing, Information and Control, vol.4, no.6, pp.1267-1274,
2008.
[14] M. Nakano, J.-H. She and T. Hino, Elimination of position-dependent disturbances in constant-
speed-rotational control systems, Control Engineering Practice, vol.4, pp.1241-1248, 1996.
[15] C.-L. Chen and G. Chiu, Spatially periodic disturbance rejection with spatially sampled robust
repetitive control, ASME Journal of Dynamic Systems, Measurement and Control, vol.130, no.2,
pp.11-21, 2008.
[16] C.-L. Chen and Y.-S. Yang, Position-dependent disturbance rejection using spatial-based adaptive
feedback linearization repetitive control, International Journal of Robust and Nonlinear Control,
vol.19, pp.1337-1363, 2009.
[17] H.-S. Ahn, Y.-Q. Chen and H. Dou, State-periodic adaptive compensation of cogging and coulomb
friction in permanent-magnet linear motors, IEEE Trans. Magnetics, vol.41, no.1, pp.90-98, 2005.
[18] K. L. Moore, M. Ghosh and Y.-Q. Chen, Spatial-based iterative learning control for motion control
applications, Meccanica, vol.42, no.2, pp.167-175, 2007.
[19] M. Fardad, M. R. Jovanovic and B. Bassam, Frequency analysis and norms of distributed spatially
periodic systems, IEEE Trans. Automatic Control, vol.53, pp.2266-2279, 2008.
[20] A. Al-Shyyab and A. Kahraman, Non-linear dynamic analysis of a multi-mesh gear train using
multi-term harmonic balance method: Period-one motions, Journal of Sound and Vibration, vol.284,
pp.151-172, 2005.
[21] T. H. Young and M. Y. Wu, Dynamic stability of disks with periodically varying spin rates subjected
to stationary in-plane edge loads, Journal of Applied Mechanics, vol.71, pp.450-458, 2004.
[22] H. K. Khalil, Nonlinear System, Third Edition, Prentice Hall, Upper Saddle River, 2002.
The 33rd Annual Conference of the IEEE Industrial Electronics Society (IECON)
Nov. 5-8, 2007, Taipei, Taiwan

Spatially Periodic Disturbance Rejection for

Uncertain Rotational Motion Systems Using
Spatial Domain Adaptive Backstepping
Repetitive Control
tCheng-Lun Chen *Yen-Hsiu Yang
Department of Electrical Engineering, National Chung Hsing University, Taichung, Taiwan
tchenc@dragon.nchu.edu.tw *s9264034@mail.nchu.edu.tw

Abstract-In this paper, we propose a new design of rejection or repetitive tracking are required. Typical repetitive
spatial-based repetitive control system for rotational motion controllers are synthesized in temporal domain. To ensure
systems required to operate at various speeds, and subject to effectiveness of the design, an underlying assumption is that
structured parameter uncertainty and spatially periodic the tracking command or disturbance is stationary, i.e., the
disturbances. To synthesize a repetitive controller in spatial frequency spectrum ofthe tracking or disturbance signal does
domain, a linear time-invariant system is reformulated with
respect to a spatial coordinate (e.g., angular displacement), not vary with time. This assumption might be seriously
which results in a nonlinear system. Adaptive backstepping is violated for disturbance rejection problems where the
applied to stabilize the system while adapting the parameters at frequency spectrum ofthe disturbance depends on the system
the same time. Then, a spatial-based reduced-order repetitive operating speed. For example, for the motor/gear motion
controller along with a stabilizing controller is designed and subsystem in laser printing systems, the disturbances due to
operates in parallel with the adaptive backstepping system. The gear eccentricity or tooth profile error are inherently spatially
overall spatial-based adaptive backstepping repetitive control
system is thus robust to structured parameter uncertainty. Also periodic. As has been explained in the literature, gear
its capability of rejecting spatially periodic disturbances is eccentricity usually induces disturbances that repeat once per
invariant to process speed. Feasibility and effectiveness of the revolution, and tooth profile error induces disturbances with
proposed scheme is verified by simulation. fundamental frequency equal to the number of teeth per
revolution. This implies that the temporal frequency values
for these two types of disturbances will be proportional to the
I. INTRODUCTION operating speed of the printers and vary accordingly.
D OTATIONAL motion systems have found their Therefore, in this particular application, a repetitive controller
lXapplication in many industry products. In most synthesized based on the temporal frequencies of
application, the systems are required to operate at different disturbances might not perform well if it does not adapt or
nominal speed while following periodic trajectories and/or tune its parameters according to current operating speed in
resisting periodic fluctuations. For example, the brushless dc real-time. If the period fluctuation is small, methods have
motor in a typical laser printer may need to operate at been shown to improve the robustness of the repetitive
different speed when driving the image formation component controller by increasing the notch width in the frequency
for printing tasks of different media or resolution. Also, laser domain of the repetitive controller at the cost of reduced
printing systems often suffer from a type of print artifacts attenuation at the periodic disturbance [5]. When the period
(known as banding), which is mostly due to periodic variation is large, there are two approaches to address the
disturbances affecting the angular velocity of the image varying period in a repetitive control framework. For
formation component (see e.g., [1]). Repetitive control based situation where the period variation is not measurable or
system has been shown to be effective for tracking periodic unknown, adaptive control approaches have been shown to be
reference commands or for rejecting periodic disturbances. effective in adapting the period of the repetitive controller at
Although the idea has been verified as early as 1981 [2], the expense of response time and transient response [6].
rigorous analysis and synthesis of repetitive controllers for When the period variation is known or measurable, such as
continuous-time and discrete-time systems were not proposed the case in gear noise induced disturbance, better trade-off
until 1989, by Hara et al. [3] and Tomizuka et al [4], between period adaptability and effectiveness of repetitive
respectively. Since then, repetitive control has gained control can be made. Recent researches started studying
popularity in applications where periodic disturbances control problems of rejecting position-dependent
disturbances or tracking periodic time-varying references [7].

1-4244-0783-4/07/$20.00 C 2007 IEEE 638

It was suggested that an existing linear time-invariant (LTI) II. PROBLEM STATEMENT
system be reformulated with respect to the spatial coordinate, Suppose a 2nd order LTI model for a typical rotational
e.g., angular displacement. An immediate advantage is that motion system can be expressed as
the spatial-based repetitive controller would be robust to
time-varying but spatially periodic references or Y(s) b1s+bo U(s) +d(s) (1)
disturbances. Unfortunately, such reformulation produces a s + als + ao
class of nonlinear systems in spatial domain, which poses a where a1, ao, b1, and bo are coefficients whose values
challenge for designing the non-repetitive part of the overall depend on system parameters. In real applications, values of
repetitive control system. In one of the author's previous those coefficients are usually time-varying (but might have
work [8], it was proposed that the nonlinear system be upper/lower bounds). Y(s) and U(s) correspond to system
linearized around the operating velocity, which resulted in a
linear position-invariant system. A linear position-invariant input and output, respectively. d(s) represents output
robust repetitive controller was then synthesized for this disturbances which are spatially periodic. For a brush dc
linear system. In another of the author's previous work [9], it motor system with parameters Jm (inertia), Dm (viscous
was proposed that the nonlinear system be treated as a coefficient), Kt (torque constant), Lm (winding inductance),
quasi-linear parameter varying (quasi-LPV) system. An LPV Rm (winding resistance), and Ke (back-emf constant), we
self gain-scheduling control with anti-windup feature, along
with a spatial-based repetitive controller, was then designed may have
for this system. In yet another of the author's work [10], 4 =0,
adaptive feedback linearization combined with repetitive bo= KtlJmLmn
control was suggested to linearize the nonlinear system while a,=(JmRm + LmDm)lJmLm
adapting the parameters to minimize the tracking error.
Repetitive control not only played the role of reducing ao = (Dm Rm + KtKe)/JmLm,
periodic disturbances, but also helped with parameter Y(s) is the motor rotational velocity, and U(s) is the motor
adaptation by shrinking the dead zone. input voltage. A possible state space realization of (1) is
In this paper, we propose another method for design ofthe (t) Ax(t) + Bu(t),
aforementioned nonlinear system, i.e., the non-repetitive dt
-

V-I
portion ofthe overall repetitive control system. To begin with, y(t) = Px(t) + d(t),
backstepping design is used to stabilize the system. However,
since this technique is sensitive to parameter uncertainty, it where x(t) =[x1 (t) x2 (t)]T,
cannot be applied directly without modification. Instead,
adaptive backstepping is applied to derive an error equation [A B]=[ai 1 b
and =[I O].
construct a control law and a parameter adjusting law so that
the state of the error equation decays to zero. Note that Note that (2) is in observable canonical form. Instead of using
adaptive control plays a role here differently from that in time t as the independent variable, it is possible to consider
previous literature (i.e., to adapt the parameters of the system an alternate independent variable 0= A(t), i.e., the angular
instead of the repetitive controller). Next, a spatial-based displacement. Since by definition
reduced-order repetitive controller along with a stabilizing
controller is synthesized and augmented with the adaptive i(t) = f c(z)dz + i(O),
backstepping system. The overall adaptive backstepping where co(t) is the angular velocity, the following condition
repetitive control (ABRC) system is thus robust to both
structured uncertainty of system parameters and spatially CO(t) =- >0, Vt>O (3)
periodic disturbances under variable process speed. Without dt
loss of generality and for preliminary study, a second order will guarantee that >t(t) is strictly monotonic such that
LTI model of a typical rotational motion system is used for t = A-'(0) exists. Thus all the variables in the time domain
demonstration and derivation of the control algorithm. can be transformed into their counterparts in the 0-domain,
Simulation is performed to verify the feasibility and
effectiveness of the proposed scheme. i.e.
This paper is organized as follows. Reformulation of an X(O) =X(A-1 (0)), yO) =Y(A-l (0)),
LTI rotational motion system into a nonlinear system in u(O) =u(A-' (0)),d )=d(Al()
angular displacement domain will be presented in Section II. di(0) = (A-2 (0)).
Section III will cover derivation and discussion of the Note that, in practice, (3) can usually be satisfied for most
proposed ABRC scheme. Simulation verification for the rotational motion system where the rotary component rotates
proposed scheme will be presented in section IV. Conclusion only in one direction. Since
and future work are given in section V.

639
 dx(t) _ dO d#() dx(O) X1 =XI1 - 0(),X2 = X2 (7)
dt dt d0 dO Within this new set of coordinates, (6) can be rewritten as
(2) can be rewritten as . X2
XI = -a, - 0+ X1
_
C(O) d (0) = Ax(0) + Bui(0), + t
dO (4) xbo (8)
j\ 0) = TX^(0) + d(0). X2 = -aO + _ u
XI +w
Equation (4) can be viewed as a nonlinear position-invariant
(as opposed to the definition of time-invariant) system with
y = X1 + (0+ d.

the angular displacement 0 as the independent variable. Note Now state x-2 can be treated as an input to stabilize the first
that the concept of transfer function is still valid for linear state equation in (8). To design x2, consider a Lyapunov
position-invariant systems if we define the Laplace transform function V1 = (1/2)x-2 and note that
of a signal g(O) in the angular displacement domain as
G(s)= C (0)e-sOdO. VI = XI Ka, - (+ _
XI + w
This definition will be used later on when we design the linear Thus we can choose x2 = = (x- + )(al + -
k3x-) such
portion of the overall control system.
that Lj=-kx1I V x10 where k,, is a positive
< 0, ,

III. ADAPTIVE BACKSTEPPING REPETITIVE CONTROL constant used to adjust characteristic of convergence.
SYSTEM Next, define Z2 X2 - and (8) can be expressed in terms
In this section, we will present the design of the proposed of xI and z2, i.e.
spatial-based adaptive backstepping repetitive control system.
The overall control configuration is illustrated in Fig 1, which X-1 =
-al,_a)+_+2 + _
can be seen as consisting of inner and outer feedback loops. XI+O) XI+O) tMA
~J)
The inner loops (with adaptive backstepping) are aiming at
stabilizing the system at the specified equilibrium point Z2 = -ao + _~
X +0)
u
u-¢.
adaptively. The outer loop (with repetitive control) is aiming To design ui consider another Lyapunov function
at reduction ofthe disturbances with bounded amplitudes and
,

Vc VI= + (1/2)z2 and note that

periodic components.
A. Backstepping Design Vc= V+Z2±2
As first step, drop the 0 notation and rewrite (4) in the = X1X1 + Z2 2
form
X1K al - +-2 + X1 + jj+ z2K(° 4+ Xb+Co)
= f (x) + g(i)^I
(5)
y= h(x) + d =-k xX12 + X1 2 -z2au-z2z4+z2 ° U
where X1 +o X1 +o
Thus we can design
a+X2
f(x)= T t11'g(X)= h(i)=x1. 1xZ2
zUb= a
Z20-1
-ao
Z2bO XI +
- 2)

XlkX-k 2)
X1+00 *

Suppose that the angular velocity corresponds to the bo 2X + 22

uncontaminated output, i.e., = h(x). In this paper, we will
limit the discussion to the system with relative degree of two
suchthatV=-kx1 X -k_ z2 <0,Vx1, Z2 0, where kZ2 is
(i.e., b1 = 0 ), which has the following description another adjustable positive constant like k . Since
x1 =x1-co and z2 =x2- , have xi
we oas x1 ->0
XI =-al,+ A
and x2-2 01 as Z2 -0 . Furthermore, the output y will
XI
approach + d as x1n.
X2 = -aO + A u (6)
XI B. Adaptive Backstepping [11]
y= + d. As shown previously, typical backstepping design is
Choose the equilibrium state at (co(0),0), where 0(O) is a partially based on cancellation of certain nonlinear terms in
function of 0. Next, introduce new coordinates, i.e. the system so that the derivative of the Lyapunov function
becomes negative. If there is any uncertainty in the system

640
parameter, the cancellation will not be exact. However, I
as
backstepping design combined with online parameter
adaptation, i.e., adaptive backstepping, can be implemented = 2
to overcome this issue. aO -Z2
Consider the system described by (8). Suppose that the a1 =
actual values of the parameters, i.e., ao, a, and bo are |22
unknown. Denote &O, al, and bo as their corresponding ° xI + co'
estimates. Then the parameter (estimation) errors can be and the control input ui as
defined as ao =ao -a0, 51 = a - , and b =bo -bo. To - 4
design x2 and parameter adaptation law, consider a bXI2a +,co 2
(10)
Lyapunov function V1O = (1/2)(x +a, ) and note that x, ± A x . A
o
VK =XlXl + ala bo CKo
+
LX,+,I ~1
k
k222j
we conclude that
=X1((l+ a^,) - a)+ ,) l(a, -

=-kxxlI -kZ 22<0,Vx1,22 (11)

where k is another adjustable positive constant like k

= l-1 -x al - xlal -
xlo)+ _+ X2-
and k .
If we specify
C. Reduced-Order Repetitive Controller
a,A1 -XI,
_

Until now the design has not taken into account the output
X2
= = -+O
X1
(x1a, X1 k- kx ) =(X1 co) (a^ kxX1), disturbance d(O) In this paper, we consider a reduced-order
.

and attenuated repetitive controller whose

then VKo < 0, V x1 .0.
continuous-position (as opposed to the definition of
Next, define a new coordinate Z2 - X2 ¢ and express the - continuous-time) version can be expressed as
k -
system as
R(s) = ils 2 +2
5*
wni g+
2'
Ojn

X1 = -a, _-)+ +_
X1+O) X1+Oa where k is the number of periodic frequencies to be rejected,
con, is determined based on the ih disturbance frequency in
Z2 = -ao + _ °u-¢s. rad/rev, and J, and Si are two damping ratios that satisfy
XI +C
Consider another Lyapunov function O<J, <g <1.
Vco= V1O + (1/2) (22 +5a2 + i0) and calculate its derivative, We can adjust the gain of R(s) at those periodic frequencies
i.e. by varying the values of J, and g .

To 'insert' the repetitive controller into the system, modify

VcO = VIO + Z2Z2 + 0O60 + bObO
the control input uz in (10) as
= X1X1 + a1a1 + Z2Z2 + 515l + bobo Xl+ - 22x - 2+ ) (12)
=X1 !-(£
(+&l)-a
i,+ ~ + X+_+j
+2 a)X- +a
X1+O) IO where U2R denotes the output of the repetitive controller.
Substituting (12) into (8), the second state equation becomes
Z2 (aO + )+ +
X2 = -ao + Ia0 -
XI -k ^

Z2 +
o - )

xi +W bo XI +, 2 2
al (-a, + aO (-aO) + bo(-bo)
b ( URI
--a0 + a^0+ ol- XI-k R

= al (-XI -al )+£O (-Z2 a0 ) + bX:o2O )

Next, we look at the output equation and take the derivative
once, i.e.

X+ +0 y = X1 + a)+ d. (14)
Kz2a0 Z2
By choosing the parameter adaptation laws for a0, a1, and Substituting the first state equation (8)
of and (13),into (14),
we obtain

641
 1 be seen that the repetitive controller will provide extra
sy-a1+_ x
X +0) disturbance reduction at the specified frequencies. The
advantage of adding repetitive control can be further justified
by looking at the simulation results as shown in Fig 3. For
ao + ^ aO+- x1 + k22 + R Xl + sd
s o coZ2 easy comparison, the output errors defined as e = y - co are
If we choose also shown in the bottom figures.

R () = -( x + co) SZ R(s)Qs) V. CONCLUSION AND FUTURE WORK

where R(s) is the reduced-order repetitive controller and This paper presents the design of a new spatial-based
repetitive control system, which can be applied to rotational
C(s) is the stabilizing controller, the overall closed-loop motion systems operating at variable speeds, with uncertain
system can be expressed as parameters, and subject to spatially periodic disturbances.
Preliminary results based on numerical simulation shows
~~~+1 ~~~-2~
+R(s)C(s)~2 +P(. feasibility and effectiveness of the proposed scheme. Note
+ d (~15)
2
Y2 + s+ +R(s)C(s)) that a full-order repetitive controller (instead of the proposed
with reduced-order one) can be used in the design if desired.
P(.
Rigorous stability analysis of the overall system, i.e., the
= -a, + X1
_
+
x
adaptive backstepping repetitive control system subject to
external disturbances, is currently under investigation.
Experimental verification is also in progress.
I L-aO+ ( a+Vd() )+q - -kZ21'
bo ~~XI +co 2
where REFERENCES
[1] C.-L. Chen, G. T.-C. Chiu and Jan P. Allebach, "Banding reduction in
-k 2 URz EP processes using human contrast sensitivity function shaped
Vd ( ) l _ z2 + photoconductor velocity control," J. Imaging Science and Technology,
X1 + c) Z2 vol. 47, no. 3, pp. 209-223, May/June 2003.
Thus we can design R(s)C(s) to reduce the effect of [2] T. Inoue, M. Nakano, and S. Iwai, "High accuracy control of
servomechanism for repeated contouring," in Proceeding of the 10th
disturbance d on the system output y'. Annual Symposium on Incremental Motion Control Systems and
Devices, pp. 258-262, 1981.
[3] S. Hara, Y. Yamamoto, T. Omata, and M. Nakano, "Repetitive control
IV. SIMULATION RESULTS system: A new type servo system for periodic exogenous signals,"
IEEE Transactions on Automatic Control, vol. 33, no. 7, pp. 659-668,
The proposed adaptive backstepping repetitive control July 1988.
scheme is applied to a brush dc motor system. The actual [4] M. Tomizuka, T. C. Tsao, and K. K. Chew, "Analysis and synthesis of
discrete-time repetitive controllers," ASME Journal of Dynamic
system is assumed to be a 2nd order system as described by (1) Systems, Measurements and Control, vol. 111, no. 3, pp. 353-358,
with ao = 2, a1 = 3, bo = 2, and b1 = 0 . For verification September 1989.
[5] Y. Onuki and H. Ishioka, "Compensation for repeatable tracking errors
purpose, the spatially periodic disturbance is assumed to in hard drives using discrete-time repetitive controllers," IEEE/ASME
consist of three sinusoidal components: Transactions on Mechatronics, vol. 6, no 2, pp. 132-136, June 2001.
[6] T. J. Manayathara, T.-C. Tsao, J. Bentsman, and, D. Ross, "Rejection of
d(0) = 0. 1 [sin(1 .6;f0) + sin(2;r0) + sin(3 .4f0)] . unknown periodic load disturbances in continuous steel casting process
using learning repetitive control approach," IEEE Transactions on
Thus, a feasible R(s) can be expressed as Control Systems Technology, vol. 4, no. 3, pp. 259-265, May 1996.
[7] M. Nakano, J.-H. She, Y. Mastuo, and T. Hino, "Elimination of
R(~) +2~co, s+con21
Si.2+2"s n
a2 position-dependent disturbances in constant-speed-rotation control
systems," Control Engineering Practice, vol. 4, pp. 1241-1248, 1996.
[8] C.-L. Chen and G. T.-C. Chiu, "Robust spatially sampled controller
where =0.1, 4 = 0.0000000001 , con = 5.0265 design for banding reduction in electrophotographic process," Proc.
IEEE Int. Conf. Robotics and Automation (ICRA), vol. 2,
n2 =6.2832 andco = 10.6814 . The stabilizing controller pp.2622-2627, 2003.
is specified as C(s) = 30000(s+0.1)/(s+1000) The [9] C.-L. Chen and G. T.-C. Chiu, "Compensating for spatially repetitive
disturbance with linear parameter varying repetitive control," Proc.
convergence parameters are set to k = 6 and k 2=30. IEEE Int. Conf Control Applications (CCA), vol. 1, pp. 736-74 1, 2004.
[10] C.-L. Chen and Y.-S. Yang, "Adaptive Repetitive Control for
Suppose that a task demands the system to initially run at 8 Uncertain Variable-Speed Rotational Motion Systems Subject to
rev/s for 30 revolutions, then speed up to 9 rev/s for 30 Spatially Periodic Disturbances," Proc. American Control Conference,
revolutions, and finally slow down to 7 rev/s. To avoid New York, NY, USA, pp. 564-569, 2007.
[11] Miroslav Krstic and Petar V. Kokotovic, "Control Lyapunov functions
discontinuity when taking derivative, the reference command for adaptive nonlinear stabilization," System & Control Letters 26
is specified to have smooth (instead of instant) transition. Fig (1995) 17-23.
2 shows the frequency response from the disturbance d to
the system output y with or without repetitive control. It can

642
 co d

-URJ

Bode Diagram from Output to Disturbance

~~~ g I
Without RC
m ~~~~~~~~~~Wt RC

C1)a
_

0)

II I
II I

-------------------------
IA--------------------
1

io-3 io-2 io-i 10 101 1 0o2

Frequency (H z)
Fig 2. Frequency responses from the disturbance to the system output.
Without RC With RC
r ^~~~~
9

8 ?
> 7 -
-------1------- -

6 6 ____ (0
co
5 -.
4 20 4 6 0 4
D) 20 40 60 80 0 20 40 60 80

0.5 --- 0.5

i
0 0

-0.5 -0.5
0 20 40 60 80 0 20 40 60 80
angular displacement ( revolution ) angular displacement ( revolution )
Fig 3. Performance comparison for the overall closed-loop system with or without repetitive control.

643
 AI-Aided Kalman Filters
Nir Shlezinger, Senior Member, IEEE, Guy Revach, Senior Member, IEEE, Anubhab Ghosh, Student
Member, IEEE, Saikat Chatterjee, Senior Member, IEEE, Shuo Tang, Student Member, IEEE,
Tales Imbiriba, Member, IEEE, Jindrich Dunik, Senior Member, IEEE, Ondrej Straka, Member, IEEE,
Pau Closas, Senior Member, IEEE, and Yonina C. Eldar, Fellow, IEEE

Abstract— The Kalman filter (KF) and its variants are among navigation computer of the Apollo spaceship [3]. To date,
the most celebrated algorithms in signal processing. These the KF is among the most celebrated algorithms in signal
methods are used for state estimation of dynamic systems by processing and electrical engineering at large, with numerous
relying on mathematical representations in the form of simple
applications including radar, biomedical systems, vehicular
arXiv:2410.12289v1 [cs.LG] 16 Oct 2024

state-space (SS) models, which may be crude and inaccurate

descriptions of the underlying dynamics. Emerging data-centric technology, navigation, wireless communications, etc.
artificial intelligence (AI) techniques tackle these tasks using The KF is a model-based method, namely, it leverages
deep neural networks (DNNs), which are model-agnostic. Recent mathematical parametric representations of the environment.
developments illustrate the possibility of fusing DNNs with classic Specifically, the KF and its variants [4] rely on the ability
Kalman-type filtering, obtaining systems that learn to track
in partially known dynamics. This article provides a tutorial- to describe the underlying dynamics as a state-space (SS)
style overview of design approaches for incorporating AI in model. Such model-based designs have several core advan-
aiding KF-type algorithms. We review both generic and dedicated tages when the dynamics are faithfully captured using a known
DNN architectures suitable for state estimation, and provide a and tractable SS representation: (i) they can achieve optimal
systematic presentation of techniques for fusing AI tools with KFs performance. For example, the KF achieves the minimal mean-
and for leveraging partial SS modeling and data, categorizing
design approaches into task-oriented and SS model-oriented. The squared error (MSE) for linear Gaussian SS models; (ii)
usefulness of each approach in preserving the individual strengths model-based methods operate with controllable and often
of model-based KFs and data-driven DNNs is investigated in reduced complexity. In fact, the KF and its variants are
a qualitative and quantitative study, whose code is publicly implemented on devices such as sensors and mobile systems,
available, illustrating the gains of hybrid model-based/data- where they operate in real-time using limited computational
driven designs. We also discuss existing challenges and future
research directions that arise from fusing AI and Kalman-type and power resources; (iii) decisions made by the KF and
algorithms. its variants are interpretable, in the sense that their internal
features have concrete meaning and their reasoning can be
explained from their observations and underlying SS model;
I. I NTRODUCTION (iv) they are inherently adaptive, as changes in the SS model
The Apollo program, which successfully landed the first are naturally incorporated into the operation; and (v) they
humans on the moon, is still considered one of mankind’s reliably characterize uncertainty in their estimate, providing
greatest achievements. Among the technological innovations the error covariance alongside their estimates [5].
that played a role in the success of the Apollo program is a A key characteristic of KF-type algorithms is their reliance
filtering method based on the extended version of an algorithm on knowledge of the underlying dynamics and specifically,
developed by Rudolf Kalman in the late 1950s [1], which was on the ability to accurately capture these dynamics using a
used to track and estimate the trajectory of the spaceship. known and tractable SS representation. A common approach
This algorithm, known as the Kalman filter (KF), and its is to rely on simplifying assumptions (e.g., linear systems,
extension into the extended KF (EKF) [2], provide an accurate Gaussian mutually and temporally independent process and
and reliable real-time trajectory estimation while still being measurement noises, etc.) that make models understandable
simple to implement and applicable on the hardware-limited and the associated algorithms computationally efficient, and
then use data to estimate the unknown parameters. However,
N. Shlezinger is with the School of ECE, Ben-Gurion University of simple models frequently fail to represent some of the nuances
the Negev, Beer-Sheva, Israel (e-mail: nirshl@bgu.ac.il). G. Revach is with and subtleties of dynamic systems and their associated signals.
the Institute for Signal and Information Processing, D-ITET, ETH Zürich,
Switzerland (e-mail: grevach@ethz.ch). A. Ghosh and S. Chatterjee are Practical applications are thus often required to operate with
with the School of EE and CS, KTH Royal Institute of Technology, partially known SS models. Furthermore, the performance and
Stockholm, Sweden (e-mail: {anubhabg; sach}@kth.se). J. Dunik and O. reliability of KF-type algorithms degrade considerably when
Straka are with the Faculty of Applied Science, University of West Bo-
hemia, Czech Republic (email: {dunikj; straka30}@kky.zcu.cz). T. Imbiriba using a postulated SS model that deviates from the true nature
is with the Dept. of CS, University of Massachusetts Boston, Boston, of the underlying system.
MA, USA (email: tales.imbiriba@umb.edu). S. Tang and P. Closas are The unprecedented success of machine learning (ML),
with the Dept. of ECE, Northeastern University, Boston, MA, USA (email:
{tang.shu; closas}@northeastern.edu). Y. C. Eldar is with the Math and and particularly deep learning, as the enabler technology for
CS Faculty, Weizmann Institute of Science, Rehovot, Israel (e-mail: yon- artificial intelligence (AI) in areas such as computer vision
ina.eldar@weizmann.ac.il). has initiated a general mindset geared towards data. It is now
This work has been partially supported by the Israel Innovation Authority,
and by the National Science Foundation under Awards ECCS-1845833 and quite common practice to replace simple principled models
CCF-2326559. with data-driven pipelines, employing ML architectures based

1
on deep neural networks (DNNs) that are trained with massive our focus to deep learning techniques that are suitable for
volumes of labeled data. DNNs can be trained end-to-end processing time sequences, briefly reviewing both generic
without relying on analytical approximations, and therefore, time-sequence architectures, such as recurrent neural networks
they can operate in scenarios where analytical models are (RNNs) and attention mechanisms, and proceeding to DNN
unknown or highly complex [6]. With the growing popularity architectures that are inspired by SS representations [24] and
of deep learning, recent years have witnessed the design of by KF processing flow [7]. We discuss the gains offered
various DNNs for tasks associated with KF-type algorithms, by such data-driven techniques, while also highlighting their
including, e.g., the application of DNNs for state estimation limitations, in turn indicating the potential of combining AI
[7], [8] and control [9]. techniques with classic SS model-based KF-type methods.
AI systems can learn from data to track dynamic systems The bulk of the article is dedicated to presenting design
without relying on full knowledge of underlying SS models. approaches that combine deep learning techniques with KF-
However, replacing KF-type algorithms with deep architec- type processing based on a partial mathematical representation
tures gives rise to several core challenges. For one, the com- of the dynamics. We categorize existing design approaches,
putational burden of training and utilizing highly parametrized drawing inspiration from the ML paradigms of discrimina-
DNNs, as well as the fact that massive data sets are typically tive (task-oriented) learning and generative (statistical model-
required for their training, often constitute major drawbacks oriented) learning [25], [26]. Accordingly, the first part is
in various signal-processing applications. This limitation is dedicated to AI-augmented KFs via task-oriented learning,
particularly relevant for hardware-constrained devices (such presenting in detail candidate approaches for converting KF-
as mobile systems and sensors) whose ability to utilize highly type filtering into an ML architecture via DNN-augmentation.
parametrized DNNs is typically limited [10]. Furthermore, due These include designs that employ an external DNN, ei-
to the complex and generic structure of DNNs, it is often ther sequentially [27] or in parallel [19], as well as DNNs
challenging to understand how they obtain their predictions, or augmented into a KF-type algorithm, e.g., for Kalman gain
track the rationale leading to their decisions. Moreover, DNNs (KG) computation [16].We then proceed to detail SS model-
typically struggle to adapt to variations in the data distribution oriented AI-aided KFs designs. These can be viewed as a form
and are limited in their capability to provide estimation of of AI-aided system identification, i.e., techniques that utilize
uncertainty. DNNs in the process of identifying SS models, which can
The challenges outlined above gave rise to a growing then be used by model-based KF-type tracking [14], [15],
interest in using AI not to replace KF-type algorithms, but [28]. We discuss several approaches with various structures
to empower them, as a form of hybrid model-based/data- connecting physics-based and data-based model components,
driven design [11]. Various approaches have been proposed for such as incremental, hybrid, or fixed SS structures.
combining KFs and deep learning including training DNNs to To highlight the gains of each approach for designing AI-
map data into features that obey a known SS model and can aided KFs, and capture the interplay between the different
be tracked with a KF [12], [13]; employing deep models for algorithms and conventional model-based or contemporary
identifying SS models to be used for KF-based tracking [14], data-driven methods, we provide a comparative study. We be-
[15] and converting the KF into an ML model that can be gin with a qualitative comparison, pinpointing the conceptual
trained in a supervised [16]–[19] or unsupervised [20], [21] differences between the design approaches in terms such as the
manner. Such AI-empowered KFs were already in various level of domain knowledge required, the type of data needed,
signal processing applications, ranging from brain-machine and flexibility. We also provide a quantitative comparison,
interface, acoustic echo cancellation, financial monitoring, where we evaluate representative methods from the presented
beam tracking in wireless systems, and drone-based monitor- design approaches in a setting involving the tracking of the
ing systems [22], [23]. These recent advances in combining challenging Lorenz attractor. The article concludes with a
AI and KFs, along with their implications on emerging tech- discussion of open challenges and future research directions.
nologies, motivate a systematic presentation of the different Notations: In the following, we use boldface lowercase for
design approaches, as well as their associated signal processing vectors, e.g., x and boldface uppercase letters, e.g., X for
challenges. matrices. For a time sequence xt and time indices t1 ≤ t2 ,
In this article, we provide a tutorial-style overview of the we use the abbreviated form xt1 :t2 for the set of subsequent
design approaches and potential benefits of combining deep variables {xt }tt=t
2
1
. We use N (µ, Σ) for the multivariate
learning tools with KF-type algorithms. We refer to this as Gaussian distribution with mean µ and covariance Σ, while
‘AI-aided Kalman Filters’. While highlighting the potential p(·) is a probability density function. The notation ∥x∥2C
benefits AI-aided KFs, we also mention the main signal denotes the squared ℓ2 norm of x weighted by the matrix
processing challenges that arise from such hybrid designs. C, i.e. ∥x∥2C = x⊤ Cx. The operations (·)⊤ and ∥ · ∥2 are
For this goal, we begin by reviewing the fundamentals of used for transpose and ℓ2 norm, respectively.
KF that are relevant to understanding its fusion with AI. We
briefly describe its core statistical representation – the SS
II. F UNDAMENTALS OF K ALMAN F ILTERING
model – and formulate the mathematical steps for filtering
and smoothing. We then discuss the pros and cons associated In this section, we review some basics of Kalman-type
with KF-type algorithms and pinpoint the main challenging filtering. We commence with reviewing SS models, after which
aspects that motivate the usage of AI tools. We then shift we recall the formulation of the KF and EKF. We conclude

2
this section by highlighting the challenges that motivate its with the recovery of the state variable xt based on a set
augmentation with deep learning. of observations {y τ }, i.e., an open-loop system where the
input ut is either absent or not controlled. Some of the most
A. State-Space Models common state estimation tasks are [4]
SS models are a class of mathematical models that describe • Filtering: estimate xt from y 1:t .
the probabilistic behavior of dynamical systems. They consti- • Smoothing: estimate x1:T from y 1:T for some T > 0.
tute the fundamental framework for formulating a broad range
Additional related tasks are prediction, input recovery, and
of engineering problems in the areas of signal processing,
imputation. State estimation plays a key role in applications
control, and communications, and they are also widely used
that involve tracking, localization, and denoising, ranging from
in environment studies, economics, and many more. The core
target tracking in radar systems to monitoring biomedical
of SS models lies in the representation of the dynamics of
signals.
a system using a latent state variable that evolves over time
while being related to the observations made by the system. The second family of SS model-based tasks are those that
Generic SS Models: Focusing on discrete-time formula- deal with stochastic control (closed-loop) policies. In this
tions of continuous-valued variables, SS models represent the family, the SS framework is used to select how to set the input
interplay between the observations at time instant t, denoted variable ut based on, e.g., past measurements y 1:t . Such tasks
y t , an input signal ut , and a latent state capturing the system are fundamental in robotics, vehicular systems, and aerospace
dynamics xt . In general, SS models consist of: (i) a state engineering. As stochastic control policies often employ state
evolution model of the form estimation schemes followed by state regulators [29], we focus
in this article on state estimation (i.e., the first family of tasks).
xt+1 = f˜t (xt , ut , v t ), (1a)
representing how the state evolves in time; and (ii) an obser-
B. Kalman Filtering
vation model of the form
The representation of dynamic systems via SS models gives
y t = h̃t (xt , wt ), (1b)
rise to some of the most celebrated algorithms in signal
which relates the observations and the current system state. processing, particularly the family of Kalman-type filters.
While the mappings f˜t (·) and h̃t (·) are deterministic, stochas- To describe these, we henceforth focus on state estimation.
ticity is induced by the temporally independent noises v t and Therefore, for convenience, we omit the input signal ut from
wt , and by the distribution of the initial state x0 . the following relations (as it can also be absorbed into the
Gaussian SS Models: The most common special case of the state evolution noise as a known bias term).
above generic model is that of the linear Gaussian SS model, KF: The KF is the minimal MSE estimator for the filtering
used by the celebrated KF. In this model, the state evolution task in linear Gaussian SS models, i.e., estimating xt from
takes the form y 1:t when these are related via (2). In every time step t,
the KF estimates xt using only the previous estimate x̂t−1
xt+1 = F t xt + Gt ut + v t , (2a)
as a sufficient statistic and the new observation y t , thus its
and the observation model is given by complexity does not grow in time.
The KF updates its estimates of the first- and second-order
y t = H t xt + wt , (2b)
statistical moments of the state, which at time t are denoted
where F t , Gt , H t are matrices of appropriate dimensions, and by x̂t|t and Σt|t , respectively. For t = 0, these moments are
v t and wt are temporally and mutually independent Gaussian initialized to those of the initial state, namely x̂0 and Σ̂0 , while
noise signals, with zero-mean and covariances Qt and Rt , for t > 0 the estimates are obtained via a two-step procedure:
respectively. Namely, v t ∼ N (0, Qt ) and wt ∼ N (0, Rt ), 1) Prediction: The first step predicts the first- and second-
while v t is independent wt , and both are independent of v τ order statistical moments of current a priori state and
and wτ for any τ ̸= t. The initial state x0 is independent of observation, based on the previous a posteriori estimate.
the noises and also assumed Gaussian, with known mean x̂0 Specifically, at time t the predicted moments of the state
and covariance Σ̂0 . are computed via
Similar models to (2) are frequently utilized for settings
characterized by non-linear transformations. In the non-linear x̂t|t−1 = F t−1 · x̂t−1|t−1 , (4a)
additive Gaussian SS model (termed henceforth as non-linear Σt|t−1 = F t−1 · Σt−1|t−1 · F⊤ + Qt−1 . (4b)
t−1
Gaussian), (1) takes the form
The predicted moments of the observations are computed
xt+1 = ft (xt , ut ) + v t , (3a) as
y t = ht (xt ) + wt , (3b)
ŷ t|t−1 = H t · x̂t|t−1 , (5a)
where the noise signals v t and wt are as in (2), i.e., temporally S t|t−1 = H t · Σt|t−1 · H⊤ + Rt . (5b)
t
independent with v t ∼ N (0, Qt ) and wt ∼ N (0, Rt ).
Tasks: SS models are mostly associated with two main 2) Update: The predicted a priori moments are updated
families of tasks. The first is state estimation, which deals using the current observation y t into the a posteriori state

3
 moments. The updated moments are computed as and H t in the KF formulations are respectively replaced with
the Jacobian matrices of ft (·) and ht (·), i.e.,
x̂t|t = x̂t|t−1 + K t · ∆y t , (6a)
Σt|t = Σt|t−1 − K t · S t|t−1 · K⊤ F̂ t = ∇xt−1 ft−1 (x̂t−1|t−1 ), (12a)
t . (6b)
Ĥ t = ∇xt ht (x̂t|t−1 ). (12b)
Here, K t is the KG, and it is given by
Alternatively, the propagation of second-order moments can
K t = Σt|t−1 · H ⊤ −1
t · S t|t−1 . (7)
be approximated using the unscented transform, resulting in
The term ∆y t ≜ y t − ŷ t|t−1 is the defined as the inno- the unscented KF (UKF), or using the cubature and Gauss-
vation, representing the difference between the predicted Hermite deterministic or stochastic quadrature rules.
observation and the observed value. The EKF and UKF are approximations of the KF designed
for non-linear Gaussian SS models. When the SS is also non-
The output of the KF is the estimated state, i.e., x̂t ≡ x̂t|t , Gaussian, state estimation algorithms typically aim at recur-
and the error covariance is Σt = Σt|t . Various proofs are sively updating the posterior distribution via the Chapman-
provided in the literature for the MSE optimality of the KF [4]. Kolmogorov relation, without resorting to its Gaussian spe-
A relatively compact way to prove this follows from the cial case utilized by the KF (See box on page 5). Leading
more general Bayes’ rule and Chapman-Kolmogorov equation, algorithms that operate in this manner include the family
recalled in the box entitled From Chapman-Kolmogorov to the of particle filters, that are based on sequential sampling;
KF on Page 5. Gaussian sum filters, based on Gaussian sum representation
Extension to Smoothing: While the KF is formulated for of all densities; and point-mass filters, numerically solving the
the filtering task, it also naturally extends (while preserving its Bayesian relation in a typically rectangular grid.
optimality) for smoothing tasks. A leading approach to realize
a smoother is the Rauch-Tung-Striebel (RTS) algorithm [31], C. Pros and Cons of Model-Based KF-Type Algorithms
which employs two subsequent recursive passes to estimate the
The KF and its variants are a family of widely utilized
state, termed forward and backward passes. The forward pass
and trusted state estimation algorithms [5]. The core of these
is the standard KF, while refines the estimates for each time t
model-based methods is the SS model, i.e., the mathematical
using future observations at time instants τ ∈ {t + 1, . . . , T }.
representation of the system dynamics via closed-form equa-
The backward pass is similar in its structure to the update tions, as those in (1). When the SS model faithfully captures
step in the KF. For each t ∈ {T −1, . . . , 1}, the forward belief the dynamics, these algorithms have several key desirable
is corrected with future estimates via properties:
←
−
x̂t|T = x̂t|t + K t · (x̂t+1|T − x̂t+1|t ), (9a) P1 When the SS model is well described as being linear
←− ←−⊤ with Gaussian noise, KF-based algorithms can approach
Σt|T = Σt|t − K t · (Σt+1|t − Σt+1|T ) · K t . (9b)
optimal state estimation performance, in the sense of
←−
Here, K t is the backward KG, computed based on second- minimizing the MSE.
order statistical moments from the forward pass as P2 Model-based methods are inherently adaptive to known
←− variations in the SS model. For instance, the matrix
K t = Σt|t · F ⊤ −1
t · Σt+1|t . (10) H t can change with time index t, and one only needs
The output of the RTS is the estimated state for every time to substitute the updated matrix in the corresponding
instant t ∈ {1, . . . , T }, with x̂t ≡ x̂t|T , as well as the error equations.
covariance Σt ≡ Σt|T . P3 Their operation is fully interpretable, in the sense that
one can associate the internal features with concrete
Extensions to Non-Linear SS Models: The KF is MSE interpretation as they represent, e.g., statistical moments
optimal for linear and Gaussian SS models. For non-linear of prior and posterior predictions.
settings, existing approaches vary between non-linear Gaussian P4 They provide reliable uncertainty measures via the error
SS models as in (3), and settings that are also non-Gaussian. covariance in (6b) and (9b).
Filtering algorithms for state estimation in non-linear Gaus- P5 KF-type algorithms operate with relatively low complex-
sian SS models are typically designed to preserve the linear ity, that does not grow with time.
operation of the KF update step with respect to measurement. However, the reliance of KF-type algorithms on faithful
Specifically, the prediction of the first-order moments in (4a) mathematical modelling of the underlying dynamics, and their
and (5a), is replaced with natural suitability with simplistic linear Gaussian models, also


x̂t|t−1 = ft x̂t−1|t −1 , (11a) gives rise to several core challenges encountered in various

applications. These challenges can be roughly categorized as
ŷ t|t−1 = ht x̂t|t−1 . (11b) follows:
The key challenge in approximating the operation of the KF C1 The state evolution and observation models employed in
lies in the propagation of the second-order moments. Arguably SS models are often approximations of the true dynamics,
the most common non-linear variant of the KF, known as the whose fidelity can vary considerably between applica-
EKF, is based on local linearizations. Here, the matrices F t tions. For instance, while the temporal evolution of a state

4
 From Chapman-Kolmogorov to the KF
Consider a generic SS model as in (1) without an input signal ut . By Bayes’ rule and the Markovian nature of (1),
it holds that the conditional distribution of xt |y 1:t satisfies
p (y t |xt ) p (xt |{y τ }τ ≤t−1 )
p (xt |{y τ }τ ≤t ) , (8a)
p (y t |{y τ }τ ≤t−1 )
where
Z
p (y t |{y τ }τ ≤t−1 ) = p(y t |xt )p (xt |{y τ }τ ≤t−1 ) dxt , (8b)
Z
p (xt |{y τ }τ ≤t−1 ) = p(xt |xt−1 )p (xt−1 |{y τ }τ ≤t−1 ) dxt−1 . (8c)

Combining (8a)-(8b) with Equation (8c), referred to as the Chapman-Kolmogorov

equation for SS models [30, Ch.
4.2], describes how the posterior distribution p xt−1 y 1:t−1 is recursively updated into p (xt |y 1:t ).
For the special case of a linear Gaussian SS model as
in (1), all the considered variables are jointly Gaussian.

Specifically, if xt−1 |y 1:t−1 ∼ N x̂t−1|t−1 , Σt−1|t−1 , then (8c) implies that
 xt y 1:t−1 ∼  N x̂t|t−1 , Σt|t−1
(computed via (4)), which together with (8b) indicates that y t y 1:t−1 ∼ N ŷ t|t−1 , S t|t−1 (computed via (5)).


Combining these with (8a) reveals that xt |y 1:t ∼ N x̂t|t , Σt|t with moments computed via (6). Accordingly, x̂t
computed by the KF is exactly the conditioned expectation of xt conditioned on y 1:t , i.e., it is MSE optimal.

corresponding to the position and velocity of a vehicle III. C OMBINING AI WITH KF S

can be represented as a linear transformation via mechan- Recent years have witnessed remarkable empirical success
ical relationships, e.g., a constant velocity model [32], of deep learning, being the main enabler framework for AI, in
such modeling of ft (·) is inherently a crude first-order various applications involving processing of time sequences.
approximation. Similarly, the relationship between the Data-driven DNNs were shown to be able to catch the sub-
velocity of a vehicle and its sensed motor currents can be tleties of complex processes and replace the need to explicitly
captured via an observation model of the form (3), the characterize the domain of interest. Therefore, an alternative
exact specification of the mapping ht (·) is likely to be strategy to implement state estimation while coping with C1-
elusive. C4, namely, without requiring explicit and accurate knowledge
C2 The purpose of the noise signals v t and wt is to (i) of the SS representation, is to learn this task from data using
capture the inherent stochasticity in the state evolution deep learning.
and measurements, respectively; and (ii) model the dis-
crepancy between the SS representation and the true A. Time Sequence Filtering with DNNs
system. Their actual distribution is thus often unknown,
complex, and possibly intractable. Non-Gaussianity can A common strategy in ML is to utilize highly parameterized
have a notable effect on performance and reliability, abstract models that are trained from data to find the param-
especially since Kalman-type algorithms seek a linear eterization that minimizes the empirical risk (with regulariza-
filtering operation. tion introduced to prevent overfitting). Their mapping, denoted
C3 Even when the dynamics are faithfully characterized by a Fθ , is dictated by a set of parameters denoted θ. In deep
non-linear Gaussian SS model, Kalman-type algorithms learning, the parametric model Fθ is a DNN, with θ being the
are sub-optimal, with gaps from optimality largely de- network parameters. Such highly-parametrized abstract models
pending on the nature and complexity of the underlying can effectively approximate any Borel measurable mapping,
non-linearity. as follows from the universal approximation theorem [6, Ch.
C4 Despite their relatively low complexity, non-linear vari- 6.4.1].
ants of the KF, such as the EKF and UKF, induce some DNNs can learn various tasks from data without requiring
latency during filtering. This is due to the need to carry any mathematical and statistical representation of the under-
out, e.g., local linearization and matrix inversion, on each lying dynamics and observations model. Accordingly, DNNs
time instant (which the linear KF can do offline based on applied for tasks such as filtering and smoothing do not
knowledge of the statistics). require the formulation of the dynamics as a SS representation.
Despite this invariance, the resulting architecture of the DNN
can still draw some inspiration from SS representation and
KF-type processing. Consequently, we divide our presentation
to conventional DNNs, which only account for the fact that the
These challenges motivate exploring data-driven approaches data being processed is a time sequence, and SS/KF-inspired
for tackling tasks associated with SS models, as detailed in the DNNs, with both families being invariant of the underlying
following section. statistical modeling.

5
 1) Conventional DNNs: Tasks associated with SS models,
particularly filtering and smoothing, involve the processing of
time sequences that exhibit temporal correlations. Accordingly,
DNNs designed to process time sequences can conceptually be
trained to carry out these tasks. These architectures, discussed
next, involve some deviation from the basic DNN form, i.e.,
based on fully connected (FC) layers, which processes a fixed-
size vector rather than a time sequence.
RNNs: A common DNN architecture tailored for time
sequences is based on RNNs. Recurrent parametric models
maintain an internal state vector, denoted st , representing the
memory of the system [6, Ch. 10]. The parametric mapping
is given by:
x̂t = Fθ (y t , st−1 ), (13)
where the internal state also evolves via a learned parametric
mapping
st = Gθ (y t , st−1 ). (14)
The vanilla implementation of an RNN uses a hidden layer to
map y t and the latent st−1 into an updated hidden variable
st . Alternative RNN architectures, such as gated recurrent
units (GRUs) and long short-term memorys (LSTMs), employ
several learned cells to update st . Then, st is used to generate
the instantaneous output x̂t using another layer, as illustrated
in Fig. 1(a). Fig. 1: Illustration of conventional DNN architectures for tasks
Attention: A widely popular DNN architecture, which is the related to filtering and smoothing, including (a) RNNs; (b)
core of the transformer model, is based on attention mecha- Attention; and (c) CNNs.
nisms [33]. Such architectures were shown to be extremely
successful in learning complex tasks in natural language
processing, where the considered signals can be viewed as
time sequences. convolutional neural networks (CNNs) are DNN architectures
An attention mechanism jointly processes a query vector that originate from image processing, and not time sequences.
q and na key-value pairs {ki , ν i }ni=1
a
. In the common case Specifically, CNNs are designed to learn the parameters of
of attention with scaled dot-product scoring, its output is spatial kernels, aiming to exploit the locality and spatial
Pna T


i=1 softmax const · q ki ν i . When applied for tasks such
stationarity of image data [6, Ch. 9]. Nonetheless, CNNs can
as filtering or smoothing, this mechanism is used as a form also be applied for time sequence processing, and particularly
of self-attention. A self-attention head is an ML model that for tasks such as filtering and smoothing. For once, the
applies trained linear layers to map the input into the queries, trainable kernel of CNNs can implement a learned finite
keys, and values. A single head self-attention with parameters impulse response filter (as a form of 1D CNN) and be applied
θ = {W q , W k , W v } applied for smoothing can be written to a time sequence, as illustrated in Fig. 1(c). Alternatively,
as one can apply CNNs to the time sequence by first converting
the time sequence (or an observed window) into the form of
x̂t = Fθ {y τ }Tτ=1


an image via, e.g., short-time Fourier transform, and then use
T
X   a CNN to process this representation.
= softmax (W q y t )T (W k y τ ) W v y τ . (15)
τ =1 2) SS/KF-Inspired DNNs: DNNs applied to process time
This procedure is illustrated in Fig. 1(b). sequences, are invariant of the underlying statistics governing
Attention-based DNNs processing time sequences typically the dynamics, and learn their operation purely from data.
apply multiple mappings as in (15) in parallel, as a form of Nonetheless, one can still design DNNs for processing time
multi-head attention. As opposed to RNNs, attention mech- sequences whose architecture is to some extent inspired by
anisms do not maintain an internal state vector that is se- traditional model-based processing of such signals, particularly
quentially updated. This makes attention-based DNNs more on SS representations and KF processing.
amenable to parallel training compared with RNNs. However, SS-Inspired Architectures: A form of DNN architecture
attention mechanisms as in (15) are invariant of the order of that is inspired by SS representation models the filter (and not
the processed signal samples, and are thus typically combined the dynamics as in (1)), as a deterministic SS model. These
with additional pre-processing termed positional embedding ML models, which we term SSM following [24], parameterize
that embed each sample while accounting for its position. the mapping from y t into the estimate x̂t using a latent state
CNNs: Unlike RNNs and attention mechanisms, vector st−1 . Particularly, the architecture operates using two

6
 Selective State-Space Models (SSMs)
An emerging SSM DNN, which is the core of the Mamba architecture and
its variants [34], is the selective SSM [35]. Selective SSMs are ML models
that process time sequences via (17). However, instead of using fixed linear
mappings as in (17), here the matrices W y , W s , W x are obtained from learned
mappings of the input y t , effectively implementing a time-varying and non-
linear SSM.
Generally speaking, consider the discretization with sampling period ∆ of
continuous-time deterministic SS representations. The resulting state evolution
in (17a) is parameterized by
W y = exp(∆ · W̄ y ) (16a)
W s = (∆ · W̄ y )−1 (exp(∆W̄ y ) − I) · ∆ · W̄ s , (16b)
where W̄ y , W̄ s are the continuous-time state evolution matrices. Selective
SSMs implement SSMs based on this representation, while using a dedicated Fig. 2: Selective SSM illustration.
layer with parameters θ 1 to map y t into W̄ s , ∆, and W x , where the former
two are then substituted in (16) along with the learned W̄ y . The overall parameters are thus θ = {θ 1 , W̄ y }. The
resulting operation is illustrated in Fig. 2.

linear mappings, where first st is updated via The resulting architecture can be combined with additional
learned input and output processing, as proposed in [7].
st = W y y t + W s st−1 . (17a)
Then, the predicted sequence is given by 3) Pros and Cons of DNN-Based State Estimation: The
x̂t = W x st . (17b) DNN architectures detailed so far can be trained to carry out
state estimation tasks, i.e., map the observed time sequence
The parameters of the resulting ML architecture are θ = into the latent state sequence. Specifically, provided data de-
{W y , W s , W x }. scribing the task, e.g., labeled data set comprised of trajectories
The SSM operation in (17) can be viewed as a form of of observations and corresponding states, these DNNs learn
an RNN, with (17a) implementing (14), and (17b) imple- their mapping from data without relying on any statistical
menting (13). While the restriction to linear time-invariant modeling of the dynamics. This form of discriminative learn-
learned operators in (17) often results in limited effectiveness, ing, i.e., leveraging data to learn to carry out a task end-to-
an extension of SSMs, termed Selective SSMs (see box on end [26], excels where model-based methods struggle: it is
page 7), was recently shown to yield efficient and high- not affected by inaccurate modeling (thus coping with C1-
performance architectures that are competitive to costly and C2, and their abstractness allows DNNs to operate reliably
complex transformers [34]. in complex settings (C3). In terms of inference speed (C4),
KF-Inspired Architectures: While SSMs parameterize a while DNNs involve lengthy and complex training, they often
learned filter via a SS representation, one can also design provide rapid inference (forward path), particularly when using
DNNs whose architecture is inspired by filters suitable for relatively compact parameterization. This follows as trained
tracking in SS models, e.g., KF-type algorithms. A represen- DNNs are highly amenable to parallelization and acceleration
tative example of such architectures is the recurrent Kalman of built-in hardware software accelerators, e.g., PyTorch.
network (RKN), proposed in [7].
The RKN is a DNN whose operation imitates the prediction- Nonetheless, replacing KF-type algorithms with DNNs
update steps of the KF and the EKF. However, while the trained end-to-end gives rise to various shortcomings, par-
model-based algorithms realize these computations based on ticularly in losing some of the desired properties of model-
the characterization of the dynamics as a SS model, the based methods. For once, DNNs lack in adaptability (P2), as
RKN parameterizes the predict and update stages as two one cannot substitute time-varying parameters of SS models
interconnected DNNs. The predict DNN, whose parameters into their operation, and thus changes in the SS model may
are θ 1 , replaces (4), and computes the a-priori state prediction necessitate lengthy retraining. Moreover, DNNs are typically
via highly parameterized architectures viewed as black-boxes,

that do not share the interpretability of model-based KF-type
x̂t|t−1 , Σt|t−1 = Fθ1 x̂t−1|t−1 , Σt−1|t−1 . (18a) methods (P3), and are complex to train or even store on
limited devices (P5). Furthermore, DNNs struggle in providing
Similarly, the update DNN, parameterized by θ 2 , produces the
uncertainty (P4), for which there is typically no ”ground truth”
posterior state estimate of (6) via

to learn from, and do not share the theoretical guarantees of
x̂t|t , Σt|t = Gθ2 x̂t|t−1 , Σt|t−1 , y t . (18b) KFs (P1).

7
Fig. 3: Illustrative comparison between model-based Kalman-type filters (a); AI-based filters (b); and AI-augmented KF (c)
divided into task-oriented and SS-oriented designs.

B. AI-Augmented KFs principled KF stages as inductive bias.

The DNN-based approaches discussed so far are highly This categorization, illustrated in Fig. 3, serves for our struc-
data-driven, in the sense that they do not rely on any statistical tured review of existing approaches in the subsequent sections.
characterization of the dynamics. Even SS or KF inspired
architectures, such as the RKN whose operation generally IV. AI-AUGMENTED KF S VIA TASK -O RIENTED L EARNING
follows the high-level stages of the KF, are ignorant of any The first family of AI-aided KFs converts Kalman-type
SS modeling. state estimation into an ML architecture that is trainable
An alternative approach that aims to benefit from the best end-to-end via DNN-augmentation. The key rationale is to
of both worlds is based on hybrid model-based/data-driven leverage deep learning techniques to directly learn the state
designs via model-based deep learning [11], [36]. This family estimation as a form of discriminative learning. A common
of algorithms typically jointly leverage data along with some approach to designing such architectures is based on using an
form of domain knowledge, i.e., partial knowledge of some external DNN, operating alongside a classic state estimator,
components of the underlying SS model (which is often with recent architectures integrating deep learning modules
available to some degree as noted in C1). In the context of state into the internal processing of Kalman-type algorithms.
estimation, such hybrid algorithms operate by augmenting
KF-type algorithms with deep learning modules, rather than A. External DNN Architectures
replacing them with DNNs. Utilizing external DNNs aims to enhance KF-type algo-
As discussed above, the direct application of DNNs for rithms without altering their internal processing. This approach
state estimation reviewed so far is based on task-oriented facilitates design, as one can separate the DNN components
end-to-end discriminative learning. In the same spirit, existing from the classic state estimator. Broadly speaking, the leading
approaches for augmenting the operation of KFs with AI tools approaches to utilizing external DNNs employ them either
can be generally categorized following the ML paradigms of sequentially, i.e., for pre-processing, or in parallel, e.g., as
generative and discriminative learning [25], [26]: learned correction terms.
• SS-oriented hybrid algorithms, that use data to learn the 1) Learned Pre-Processing: A popular approach when
underlying statistical model as DNN-aided system iden- dealing with complex measurement models, e.g., visual or
tification (thus bearing similarity to generative learning). multi-modal observations, builds on the ability of deep learn-
• Task-oriented schemes, that directly learn to carry out ing to extract meaningful features from complex data. Specif-
the state estimation task (as a form of discriminative ically, it uses a DNN pre-processor to map the observations
learning), while leveraging partial state knowledge and into a latent space via a DNN [13], [27], [37].

8
 and uncertainty measures are provided (P4). The excessive
complexity lies in the incorporation of the pre-processing
DNN and thus depends on its parameterization, as well as
on the dimensions of z t . For instance, when z t is of a much
lower dimension compared to y t , the complexity and inference
latency savings of applying a KF to z t compared to using y t
as observations may surpass the added complexity and latency
of the pre-processing DNN.

The adaptivity of KFs (P2) is not necessarily preserved.

Specifically, when the DNN is fully separate from the state
Fig. 4: EKF with DNN pre-processing illustration. estimator (e.g., when training via (21)), then the state evolution
parameters only affect the model-based algorithm, and thus
one can still operate with time-varying ft (·) (assuming its
Architecture: Consider a SS model where the observation variations are known). This is not necessarily the case when
model ht (·) is complex and possibly intractable. In such cases, training via (22), as the latent features are learned to be
one can design a DNN with parameters θ whose output ones most supporting state estimation based on the evolution
is approximated as obeying a linear Gaussian observations model used during training [17]. Moreover, variations in the
model, i.e., observation model typically necessitate re-training of θ.

z t = Fθ (y t ) ≈ Hxt + wt , wt ∼ N (0, R), (19)

Using an external pre-processing DNN, while being simple
with the observation matrix H being fixed a-priori. The latent and straightforward to combine with model-based state esti-
z t , combined with a known state evolution model, represent mation, relies on Gaussianity and known distribution of the
observations of a closed-form SS representation. Accordingly, state evolution. The resulting latent SS model is often non-
the latent signal can be used as input to a model-based KF-type Gaussian, which can impact the tracking accuracy in the latent
state estimator to track xt , as illustrated in Fig. 4. space, thus not handling C2. Moreover, this approach does not
Supervised Training: In a supervised setting, one has cope with complexities in the state evolution, thus not being
access to a labeled data set comprised of multiple pairs of geared towards handling inaccuracies (C1) and dominant non-
state and observation length T trajectories, e.g., linearities (C3) in ft (·).
 (i) (i) N
Ds = {xt , y t }Tt=1 i=1 . (20)
2) Learned Correction Terms: In scenarios where one
Given such labeled data, θ can be trained by encouraging the
has full characterization of the dynamic system as a SS
DNN output to closely match Hxt , i.e., by using a loss of
model, such that KF-type algorithms are applicable, yet the
the form
characterization is not fully accurate, external DNNs can
N T
1 XX (i) (i) further enhance performance by providing correction terms to
LDs (θ) = Fθ (y t ) − Hxt . (21)
N T i=1 t=1 2 internal computations of the model-based method [19]. This
is illustrated using a representative example, based on the
When training via (21), the noise covariance R can be augmented Kalman smoother proposed in [19].
estimated empirically from the validation data.
An alternative training approach finds θ based on its
Architecture: Consider a linear Gaussian SS model, and
downstream state estimation task, leveraging the differentiable

focus on the smoothing task, i.e., recovering a sequence of
operation of KFs [38]. Here, by letting x̂t Fθ (y t ) be the
T state variables x1:T = {xt }Tt=1 from the entire observed
state estimate obtained by a KF/EKF with observations z t =
sequence y 1:T = {y t }Tt=1 . As discussed when presenting the
Fθ (y t ), a candidate loss measure is
RTS algorithm, various algorithms exist for such tasks, one
N T of which involves smoothing by seeking to maximize the log-
1 XX (i)
(i)
LDs (θ) = x̂t Fθ (y t ) − xt . (22) likelihood function via gradient ascent optimization, i.e., by
N T i=1 t=1 2
iterating over
The covariance matrix R can be learned during training (q+1) (q)

(q)

by being incorporated into the trainable parameters of the x1:T = x1:T + γ∇x(q) log p y 1:T , x1:T , (23)
1:T

architecture. where γ > 0 is a step-size, for q = 0, 1, 2, . . . denoting

Discussion: Using an external pre-processing DNN that is the iteration index. The Markovian nature of the SS model
separated from the Kalman-type state estimation algorithm is indicates that the gradients in (23) can be computed via
a design approach geared mostly towards handling complex message passing, such that for the t’th index
observation models. As state estimation is carried out by  
(q)
a model-based algorithm, it preserves most of its favorable ∇x(q) log p y 1:T , x1:T = µ(q) (q) (q)
xt−1 →xt + µxt+1 →xt + µy t →xt ,
t
properties: The state estimation operation is interpretable (P3), (24)

9
where the summands, referred to as messages, are given by B. Integrated DNN Architectures
 
µ(q) −1 (q) (q) Unlike external architectures, which employ DNNs sepa-
xt−1 →xt = −Q xt − F xt−1 , (25a)
  rately from Kalman-type algorithms, integrated architectures
T −1 (q) (q)
µ(q)
xt+1 →xt = F Q xt+1 − F xt , (25b) replace intermediate computations with DNNs. Doing so
  converts a Kalman-type state estimation algorithm into a
T −1 (q)
µ(q)
y t →xt = H R y t − Hxt . (25c) trainable ML model which follows the operation of the classic
state estimation method as an inductive bias. The key design
The iterative procedure in (23), is repeated until convergence,
(q) rationale is to augment computations that depend on missing
and the resulting x1:T is used as the estimate. It is worth domain knowledge with dedicated DNNs. Accordingly, exist-
noting that in (25), the messages are obtained by assuming a ing designs vary based on the absent domain knowledge or,
time-invariant version of the SS model in (2) [19]. alternatively, on the augmented computation.
An external DNN augmentation suggested in [19] aims at 1) Learned Kalman Gain: A key part of Kalman-type
facilitating the operation of such smoother under approximated algorithms is the derivation of the KG K t . In particular, its
SS characterization by learning to map the messages in (25) computation via (7) encapsulates the need to propagate the
(q+1)
into a correction term ϵ1:T , replacing the update rule (23) second-order moments of the state and observations. This, in
with turn, induces the requirement to have full knowledge of the
   
(q+1) (q) (q)
x1:T = x1:T + γ ∇x(q) log p y 1:T , x1:T + ϵ1:T
(q+1)
. underlying stochasticity (C2); leads to some of the core chal-
1:T lenges in dealing with non-linear SS models (C3); and results
(26)
in excessive latency which is associated with propagating these
Particularly, based on the representation of the smoothing op-
moments (C4). Accordingly, a candidate approach for state
eration as messages exchanged over a Markovian factor graph
estimation in partially known SS models, which is the basis
whose nodes are the state variables, [19] proposed a graph
for the KalmanNet algorithm [16] and its variants [17], [39],
neural network (GNN)-RNN architecture with parameters θ.
[40], augments the KG computation with a DNN.
This architecture maps the messages in (25) into the correction
(q+1) Architecture: Consider a dynamic system represented as a
term ϵ1:T , namely,
  SS model in which one has only a (possibly approximated)
(q+1)
ϵ1:T = Fθ y 1:T , {µ(q) , µ(q)
, µ(q)
}T model of the time-invariant state evolution function f (·) and
xt−1 →xt xt+1 →xt y t →xt t=1 .
(27) the observation function h(·). An EKF with a learned KG
employs a DNN with parameters θ to compute the KG for
Supervised Training: The external DNN correction mecha-
each time instant t, denoted K t (θ). Using this learned KG,
nism is trained end-to-end, such that the state predicted by the
an EKF is applied, predicting only the first-order moments via
corrected algorithm best matches the true state. Particularly,
(11), and estimating the state as
since smoothing here is based on an iterative algorithm (gradi-
ent ascent over the likelihood objective), its training is carried x̂t = x̂t|t−1 + K t (θ)(y t − ŷ t|t−1 ). (29)
out via a form of deep unfolding [36], fixing the number of
An illustration of the architecture is depicted in Fig. 5.
iterations to some Q. Then, accounting for the fact that the
As the resulting architecture does not explicitly track the
iterative steps in (26) should provide gradually refined state
second-order moments, that are needed for the KG, and thus
estimates, the loss function accounts for the contribution of
the learned computation must do so implicitly. Accordingly,
the intermediate iterations with a monotonically increasing
the DNN has to
contribution. Particularly, the loss function used for training
θ is 1) Process input features that are informative of the noise
N T Q
signals.
1 XXX q (q) (i)
(i) 2) Possess some internal memory capabilities.
LDs (θ) = x̂t y 1:T ; θ − xt ,
N T i=1 t=1 q=1 Q 2 To meet the first requirement, the input features processed
(28) by the DNN typically include (i) differences in the observa-
(q) (i) (i)
where x̂t (y 1:T ; θ) is the smoothed estimate of xt produced tions, e.g., ∆y t = y t − ŷ t|t−1 ; and (ii) differences in the
by the q’th iteration, i.e., via (26), with parameters θ and y 1:T . estimated state, such as ∆x̂t−1 = x̂t−1 − x̂t−1|t−2 . In order
Discussion: Augmenting a KF-type algorithm via a learned to provide internal memory capabilities, architectures based on
correction term is useful in settings where one can still RNNs [16], [40], or even transformers [41] can be employed.
apply the model-based algorithm quite reliably (up to some For instance, a simple FC-RNN-FC architecture was proposed
possible errors that external DNN corrects). For instance, the in [16], as well as a more involved interconnection of RNNs,
augmented algorithm detailed above assumes the dynamic while [40] used two RNNs – one for tracking Σt|t−1 and one
system can be approximated as a linear Gaussian SS model, for tracking S −1
t|t−1 – that are combined into the KG via (7).
such that model-based smoothing can be applied based on this While the above architecture is formulated for filtering, it
formulation, yet it is not MSE optimal (as is the case without can also be extended to smoothing tasks. Particularly, when
discrepancy in the SS model). Consequently, this approach smoothing via the RTS algorithm [31] employs a KF for a
is suitable for tackling C1, yet it is less valid for handling forward pass, followed by an additional backward pass (9) that
←−
complex dynamics (C3), and only adds excessive latency to uses the backward gain matrix K t given in (10). Consequently,
the model-based algorithm, thus not handling C4. the DNN augmented methodology above extends to smoothing

10
 • Downstream Task: Often, in practice, state estimation
is carried out as part of an overall processing chain,
with some downstream tasks. In various applications,
one can evaluate an estimated state without requiring
a ground truth value. When such evaluation is written
(or approximated) as a function that is differentiable
with respect to the estimated state, it can be used as
a training loss. This approach was shown to enable
unsupervised learning of KalmanNet when integrated in
stochastic control systems, where it is combined with a
linear quadratic regulator [42], as well as in financial pairs
trading, where the state tracked are financial features used
for trading policies [23].
Discussion: The design of DNN-aided Kalman-type algo-
rithms by augmenting the KG computation is particularly
suitable for tackling identified challenges in C1-C4. Unlike
Fig. 5: EKF with learned KG illustration. purely end-to-end DNNs designed for generic processing of
time-sequences, augmenting the KG allows leveraging domain
knowledge of the state evolution and observation models,
by introducing an additional KG DNN that learns to compute as these are utilized in the prediction step. However, as the
←−
K t . Moreover, the single forward-backward pass of the RTS following processing of these predictions utilizes a DNN that
algorithm is not necessarily MSE optimal in non-linear SS is trained based on the accuracy of the overall algorithm, mis-
models, the learned RTS can be applied in multiple iterations, matches, and approximation errors are learned to be corrected,
using the estimates produced at a given forward-backward pass thus fully tackling C1. As the DNN augmentation bypasses the
be used as the input to the following pass [39]. Here, the fact need to track second-order moments, the resulting algorithm
that each pass is converted into a ML architecture allows to does not require knowledge of the distribution of the noises. In
learn a different forward and backward gain DNNs for each fact, the resulting filter is not linear (as the KG depends on the
pass, as a form of deep unfolding [36] observations), allowing to learn non-linear state estimators that
Training: KalmanNet and its variants use DNNs to compute are suitable for non-Gaussian SS models (C2) and non-linear
the KG. While there is no ‘ground-truth‘ KG when deviating dynamics (C3). Finally, as the computation of the KG and the
from dynamics fully captured as linear Gaussian SS models, propagation of the second-order moments induces most of the
the overall augmented state estimation algorithm is trainable latency in Kalman-type algorithms for non-linear SS models,
in a supervised manner. Particularly, given a labeled data set replacing these computations with a compact DNN often leads
as in (20), a candidate loss measure is to more rapid inference (C4) [39].
N T The fact that the resulting state estimator is converted into a
1 XX (i)
(i)
LDs (θ) = x̂t y t ; θ − xt , (30) trainable discriminative ML architecture makes it natural to be
N T i=1 t=1 2
combined with DNN-based pre-processing (as in Fig. 4), e.g.,


where x̂t y t ; θ is the state estimate obtained using the for processing images or high dimensional data. Particularly,
augmented EKF with parameters θ and observations y t . the learned state estimator can be trained jointly with the pre-
While KalmanNet is designed to be trained from labeled processing stage, thus having it learn latent features that are
data via (30), in some settings it can also be trained with- most useful for processing with the learned filter, without
out providing it with ground-truth state labels. This can be requiring one to approximate the distribution of the latent
achieved via the following approaches: observation model noise [17].
Unlike state estimation based on end-to-end DNNs, which
• Observation Prediction: The fact that the DNN-
also tackles C1-C4 (as discussed in the previous section), the
augmented algorithm preserves the interpretable opera- usage of Kalman-type algorithms as an inductive bias allows to
tion of Kalman-type state estimation can be leveraged also preserve some of the desirable properties of model-based
for unsupervised learning, i.e., learning from a data set state estimators. In particular, the interpretable operation is
of the form preserved, in the sense that the internal features are associated
 (i) N
Du = {y t }Tt=1 i=1 . (31) with concrete meaning (P3), which can be exploited for, e.g.,
Specifically, as Kalman-type algorithms with time- unsupervised learning. Moreover, while the error covariance is
invariant SS models internally predict the next observa- explicitly tracked, in some cases it can actually be recovered
from the learned KG (see box entitled ‘Uncertainty Extraction


tion as ŷ t+1|t = h f (x̂t ) in (5), a possible unsupervised
loss is [21] from Learned KG‘ on Page 12), thus providing P4 to some
N T −1
extend. In addition, the fact that only an internal computation
1 XX (i)


(i) is learned allows utilizing relatively compact DNNs, striking a
LDu (θ) = h f x̂t y t ; θ − y t+1 .
N T i=1 t=0 2 balance between the excessive complexity of end-to-end DNNs
(32) and the relatively low one of model-based methods (P5).

11
 Uncertainty Extraction from Learned KG
Kalman-type algorithms use both the prior covariance Σt|t−1 as
well as the KG to compute the posterior error covariance Σt ,
which represents the estimation uncertainty. Particularly, by (6b)
and (7), the EKF computes the error covariance as
Σt = (I − K t Ĥ t )Σt|t−1 . (33a)
Consequently, despite the fact that AI-aided filters with learned
KG do not explicitly track the second-order moments, in some
cases, these can still be recovered via (33a), as shown in [43].
In particular, the architecture provides the learned KG as K t (θ).

−1
⊤
Accordingly, when the matrix H̃ t = Ĥ t Ĥ t exists, then,
combining (5b) and (7), the prior covariance can be recovered
via
Σt|t−1 = (I − K t (θ)Ĥ t )−1 K t (θ)RĤ t H̃ t . (33b)
This indicates that when one accurately computes Ĥ t (via (12)) Fig. 6: EKF with learned KG and covariance
and has knowledge of the observations noise covariance R, then extraction illustration.
the learned KG can be used to recover the error covariance via
(33a) and (33b), as illustrated in Fig. 10. The extracted covariance
can be encouraged to match the empirical estimation error in
training by, e.g., adding an additional loss term that penalizes
uncertainty extraction, or by imposing a Gaussian prior on the
error and minimizing its likelihood, see [43].

The core limitations of designing AI-aided state estimator servation model without any knowledge of the state evolution
by augmenting the KG computations are associated with its model. The state evolution model in (1a) may not be linear or
adaptivity. The design is particularly geared towards time- require v t to be Gaussian noise. A candidate approach in this
invariant state evolution and observation functions. As the KG category is the data-driven nonlinear state estimation (DANSE)
computation is coupled with these models, any deviation, even method that provides a closed-form posterior of the underlying
known ones, is expected to necessitate re-training. This can state using linear state measurements under Gaussian noise
be alleviated under some forms of variations using hypernet- [20]. The noisy measurements follow the linear observation
works [44], i.e., having an additional DNN that updates the KG model similar to (2b), namely,
DNN, while inducing some complexity increase. Moreover,
y t = H t xt + wt , (34)
the architecture is designed for supervised learning. The ability
to train in an unsupervised manner via (32) relies on full where the measurement noise is wt ∼ N (0, R) with known
knowledge of f (·) and h(·), thus not being applicable in the covariance R. The matrix H t is assumed to be full column
same range of settings as its supervised counterpart. The same rank and known ∀t. Note that, p (y t |xt ) = N (H t xt , R).
also holds for uncertainty extraction via (33), which requires
some additional domain knowledge compared to that needed Architecture: The core of the DANSE method relies on the
for merely tracking the state via (29). sequential modeling capability of RNNs [6, Ch. 10]. DANSE
The introduced learned KG algorithms calculate the com- consists of an RNN with parameters θ that parameterizes
plete gain by a DNN. However, if a Kalman-type filter, such a Gaussian prior on xt given y 1:t−1 at a given
time t.
as the unscented Kalman filter or divided-difference filter, Concretely, the prior distribution p xt |y 1:t−1 ; θ is
depends on a user-defined scaling parameter, a DNN can be

p xt |y 1:t−1 ; θ = N x̂t|t−1 (θ) , Σt|t−1 (θ)


used for the parameter prediction (instead of the complete gain
calculation) [45], [46]. As a consequence, such DNN-reasoned s.t. {x̂t|t−1 (θ) , Σt|t−1 (θ)} = Fθ (y t−1 ), (35)
scaling parameter affects all elements of the KG and is able, where Fθ (y t−1 ) refers to the RNN that recursively processes
up to a certain extent, to compensate for model discrepancies the sequence of past observations y 1:t−1 as described in (13),
or linearisation errors. This DNN-augmented filter inherently (14). Also, x̂t|t−1 (θ) and Σt|t−1 (θ) denote the mean vector
provides the estimate error covariance matrix, but under the and the covariance matrix respectively of the RNN-based
assumption of the known SS model (3). Gaussian prior at time t. The covariance matrix Σt|t−1 (θ)
2) Learned State Estimation: Another class of integrated can be designed to be full or diagonal. More accurately, the
DNN architectures includes methods that learn the task of hidden state of the RNN in (35) at time t is non-linearly
state estimation in a data-driven manner using the known ob- transformed using feed-forward networks with appropriate

12
 DANSE Architecture
The DANSE method utilizes
an RNN for recursively pro-
cessing the input sequence
y 1:t−1 [20]. An example of
this sequential processing at
time t is shown in Fig.
7. One can use prominent
RNNs for sequential process-
ing, such as GRUs or LSTMs
. In DANSE, GRU was used
due to simplicity and good
expressive power compared
to vanilla RNNs. Specifically,
for modeling the mean vec-
tor x̂t|t−1 (θ) and diagonal Fig. 7: A detailed schematic diagram of the DANSE method at time t, highlighting
covariance matrix Σt|t−1 (θ) both training-specific and inference-specific blocks [20]. The dash-dotted line
of the parameterized Gaussian represents the gradient flow to Fθ during training, solid lines indicate information
prior in (35), the hidden state flow during training/inference.
of the GRU was nonlinearly
transformed using feed-forward networks with suitable activation functions. In Fig. 7, the ‘Accumulator’ block is
shown for ease of understanding; in practice, the block is embedded in the recursive operation of the RNN.

activation functions to obtain x̂t|t−1 (θ) and Σt|t−1 (θ) [20].

In [20], DANSE uses GRU for implementation purposes owing
to its simplicity and modeling capability.
Using the fact that the measurement system in (34) is linear
and also Gaussian, we have p (y t |xt ) = N (H t xt , R). This
allows the use of the Chapman-Kolmogorov equation (8a) to
obtain p (xt |y 1:t ; θ) in closed-form [20]. One can show that
the posterior distribution p (xt |y 1:t ; θ) is also Gaussian with
p(xt |y 1:t ; θ) = N (x̂t|t (θ), Σt|t (θ)),
x̂t|t (θ) = x̂t|t−1 (θ) + K ′t (θ) ∆y t (θ) , Fig. 8: Simplified schematic of DANSE [20].
Σt|t (θ) = Σt|t−1 (θ)

⊤
− K ′t (θ) S t|t−1 (θ) K ′t (θ) . (36)
is shown in Fig. 8 with a detailed schematic and details
Notably, (36) resembles the Kalman update equations in (6), regarding architectural choices present in the box ‘DANSE
(7) where Architecture’ on Page 13.
K ′t (θ) ≜ Σt|t−1 (θ)H ⊤ −1
t S t|t−1 (θ) ,
Training: The parameters θ in DANSE are learned in an
unsupervised manner using a training dataset consisting of
S t|t−1 (θ) ≜ H t Σt|t−1 (θ) H ⊤
t + R, (i)
only noisy measurement trajectories Du = {y 1:T }N i=1 , where
∆y t (θ) ≜ y t − H t x̂t|t−1 (θ). (37) N is the number of training samples, where every i’th sample
is assumed to have the same trajectory length T . The learning
Note that K ′t (θ) is conceptually similar as the traditional
mechanism is based on maximizing the likelihood of the
Kalman gain term in (7), but computed in a data-driven
dataset D, where one computes the joint likelihood of a full
manner, and that the computation approach is different from
sequence y 1:T . In order to achieve this, one first
calculates
the one illustrated in Fig. 5 due to (35).
the conditional marginal distribution p y t |y 1:t−1 ; θ using the
In the backdrop of KF, a crucial aspect of DANSE is that
Chapman-Kolmogorov equation (8b) as follows
there is no Gaussian propagation of the posterior in (36) to Z
the prior because DANSE does not use any state evolution
p(y t |y 1:t−1 ; θ) = p(y t |xt )p(xt |y 1:t−1 ; θ) dxt
model like (1a). The parameters x̂t|t−1 (θ) , Σt|t−1 (θ) of the
prior are obtained directly from the RNN, which does not = N (H t x̂t|t−1 (θ), H t Σt|t−1 (θ)H ⊤
t + R)
use any explicit state evolution model as in (1a), or require
= N (H t x̂t|t−1 (θ), S t|t−1 (θ)), (38)
any knowledge regarding the same, e.g. first-order Markovian,
Gaussian process noise, etc. A simplified schematic of DANSE where we use the notation for S t|t−1 (θ) in (37). Then for the

13
sequence y 1:T , one can calculate the joint likelihood as The DANSE method is also partly interpretable as the
T posterior update of DANSE as shown in (36) is tractable and
bears similarity to the KF posterior update in (6). This is
Y
p (y 1:T ; θ) = p(y t |y 1:t−1 ; θ). (39)
t=1 ensured by using the linear observation model in (34) and
the choice of the Gaussian prior for xt in (35). This ensures
Thus, using Du , (38) and (39), the optimization problem can
tractable posterior updates, resonating with the feature P3
be formulated as maximization of the logarithm of the joint
regarding the interpretability of conventional model-based ap-
log-likelihood of Du as follows
proaches using first and second-order moments. Furthermore,
T
N Y
Y 
(i) (i)
 the advantage P4 regarding providing uncertainty estimates is
max log p y t |y 1:t−1 ; θ = min LDu (θ) , (40) also present in DANSE as we have both prior and posterior
θ θ
i=1 t=1 covariance estimates in (35), (36). As mentioned earlier, the
where the loss term LDu (θ) is obtained using (38) as follows key difference compared to model-based approaches is the
N X T
( absence of propagating the posterior moments to the prior at
X n 1 
(i)

the next time point.
LDu (θ) = log 2π + log det S t|t−1 (θ)
i=1 t=1
2 2 Lastly, it is worth noting that DANSE cannot immediately
) adapt to underlying changes in the SS model and would require
1 (i)
+ ∥y t − H t x̂t|t−1 (θ)∥2 (i) −1 . (41) re-training. This also applies to changes in the observation
2 S t|t−1 (θ)
model as it requires complete knowledge of the same. Hence
(i) (i) DANSE does not have the advantage P2, which is inherent in
In the above {x̂t|t−1 (θ), Σt|t−1 (θ)} denote the parameters of
(i)
model-based approaches and AI-aided approaches that utilize
the Gaussian prior (35) obtained using the i’th sample y 1:t−1 the knowledge of the state evolution model or additional
as input at time t. The parameters θ are thus learned by hypernetworks as mentioned earlier in section IV-B1.
minimizing LDu (θ) with respect to θ in (40). Practically, this
is achieved by using mini-batch stochastic gradient descent
as explained in [20, Sec. II-C]. V. AI-AUGMENTED KF S VIA SS-O RIENTED L EARNING
The second family of AI-aided KFs uses data to learn,
Discussion: The DANSE approach can be used to address refine, or augment the underlying statistical model using deep
the challenges associated with model-based KF-type algo- learning tools. Then, the data-augmented SS can be directly
rithms explained in C1-C4. DANSE can be used to partly used in Kalman-type state estimation. Instead of learning the
address C1 as it does not require knowledge of the state state estimation or the specific parameters in the Kalman filter,
evolution model (1a) or the involved process noise in (1a). e.g., KG, the key rationale of SS-oriented learning is to exploit
However, it requires knowledge of the linear observation the data to obtain a more accurate model. At the same time,
model shown in (34), where H t should be full-rank and noise SS-oriented learning preserves explainability of the state and
covariance R should be known a priori. The parameterization brings associated benefits of the statistical estimation1 such
of the Gaussian prior in (35) ensures that DANSE can capture as the inherent calculation of the covariance matrix of the
long-term temporal dependency in the state model so that estimate error [15].
one is not constrained to state estimation for Markovian SS Architecture: The SS-oriented AI-augmented KFs focus
models. This is also mentioned by the authors in [20] where on data-augmented modeling of the state equation, while the
they introduce DANSE in a ‘Model-free process’ setting. In measurement (or observation) model is assumed to be known.
[20], the authors define a ‘Model-free process’ as a process The motivation behind this formulation is twofold: (i) the
where the knowledge of the state evolution model is absent. fact that the sensors can often be well modeled on the basis
This also partly tackles C2 since there is no requirement on of first principles but a state dynamics model is typically
the distribution of v t in (1a) to be Gaussian. At the same time, approximate and widely depends on a user decision (for
it is required that wt is Gaussian with a fixed covariance R example, object kinematic can be modeled by nearly constant
as shown in (34). velocity/acceleration model or a Singer model) [15]; (ii) the
The use of recurrent neural networks and the parameteri- need to estimate state dynamics that carry physical meaning.
zation of the Gaussian prior also ensures that one can learn The concept of data-augmented modeling thus resides in the
a nonlinear state estimation method for nonlinear SS models definition of four models of state dynamics;
similar to that in the case of KalmanNet. This in turn helps to • True model (TM), or a data generator, is a complex
tackle the challenge described in C3. The training of DANSE, and context-dependent model that cannot be expressed
as explained in the previous paragraph, is unsupervised and in a finite-dimensional form. The TM is thus more of a
offline as it requires access to a dataset Du consisting of noisy theoretical construct.
measurements. Once the training is completed, the inference • Physics-based model (PBM) is defined by (3a) and can be
step in DANSE is causal and separated from the offline understood as the “best achievable” model of the given
training step. This ensures that one has a rapid inference at complexity found from the first principles.
test time. This helps address C4, as DANSE doesn’t require
any linearization of the state evolution model as in EKF or 1 Such augmented model can directly be used also for design a fault-
sampling sigma points or particles in UKF or PF, respectively. detection or control algorithm.

14
 • Data-driven model (DDM) is solely identified from data. where v APBM
t is a state noise, dt represents additional data3 , the
The use of ML in system identification has been studied function g(·) is aware of the PBM part, and the vector θtAPBM
for several decades [47], [48] and the DDM can be written is designed to minimize the discrepancy between outputs of
as (43) and the TM. The APBM compensates the PBM structure
and parameter mismodelling using information extracted from
xt+1 = g(xt ; θ DNN ) + v DDM
t , (42)
available data. Consequently, the APBM preserves the physical
where v DDM
t is a state noise, g(·) is a vector-valued meaning of the state components and exploits actual system
function, possibly a DNN, which is parametrized by behavior dependencies, which were ignored in the PBM de-
the vector θ DNN designed to minimize the discrepancy sign. An example of this modeling versatility was shown in
between outputs of (42) and the TM. [58], where APBMs were used to filter a high-order Markov
• The current trend is to consider models that are not purely process without the need for order selection. One important
trained from data, as in DDMs, but that make use of characteristic of the APBM formulation (43) is the flexibility
the available information that PBMs offer. These hybrid to cope with the non-stationarity of the model over time or
models are discussed in more detail in the remainder of space (θ APBMt−τ ̸= θ t
APBM
), which requires adaptive estimation
the section. strategies [59].
The approaches to data-augmented modeling of the state The general APBM structure (43) can be simplified into
equation can be classified in terms of the SS components that an additive form with explicitly controlled contribution of
the DNN characterizes or is embedded in. In the general SS the data-based component (see box entitled ‘APBM with
structures (see Fig. 9a), the whole model of state dynamics is Controlled Additive Structure’ on Page 17). Bounding the
identified from data and replaced by an DNN2 (42). In [49], FC contribution of the data-driven component of the model is an
DNNs were used to represent the function ft (·), and in [50], essential feature of the APBM, which prevents the data-based
the authors use predictive partial autoencoders to find the state component from overruling the PBM component contribution
and its mapping to the measurement. In [51], FC DNNs trained and, thus, preserving APBM explainability.
by the expectation-maximization algorithm were used to rep- The APBM state dynamics model (43) together with the
resent both functions while [52] utilized LSTM DNNs for this observation model (any of (1b), (2b), (3b)) can directly be
purpose. A comparison of the performance of several general used for joint state xt and parameter θ APBM t estimation by a
SS structures was presented in [14], considering LSTM, GRU, regular state estimator such as the EKF or the UKF [30].
and variational autoencoders (VAEs). The authors assumed Training APBMs: Training APBMs can be performed
Gaussian likelihood and combined the RNN and VAE into under different paradigms depending on data availability,
a deep state space model. Stochastic RNNs were introduced architecture, and assumptions regarding the stationarity of
for system identification in [53] to account for stochastic the system’s dynamics. Although the learning strategy can
properties of the SS model. A similar approach is followed be supervised when Ds is available, we will focus on the
physics-informed neural network (PINN) [54] modeling the more common problem in the Bayesian filtering literature,
state equation by a DNN which training is constrained by the where only noisy observations, Du , are accessible, setting up
physics describing the PBM (see Fig. 9b). an unsupervised learning scenario. In this context, parameter
Another approach to model state equation is to consider a estimation can be achieved by (i) obtaining and maximizing
fixed structure such as a linear parameter-varying model (2a) the marginal posterior p(θ|y 1:T ), often using the energy
and use the DNNs to represent the matrices [55] (see Fig. 9c). function (φT (θ) = − log p(y 1:T |θ) − log p(θ)) [30]; (ii) the
Such an approach has also been adopted in [38], where the joint posterior p(xt , θ|y 1:T ) [59]; or (iii) obtaining a point
covariance matrices Q and R were provided by a DNN. estimate θ̂ through deterministic optimization strategies often
A similar idea was elaborated in [56] where an incremental aiming at maximizing the variational lower bound of the log-
SS structure was proposed to separate ft (·) into linear and likelihood p(y 1:T ; θ) [60].
nonlinear parts and use a DNN to represent the nonlinear In [15], [59], the authors opted for a state-augmentation
part (see Fig. 9d). In [57], a hybrid model (see Fig. 9e) was approach aiming at obtaining the joint posterior distribution
proposed, which combines the PBM for some state elements p(xt , θ t |y 1:T ) through Bayesian filtering recursion. For such,
and the DNN for the rest. It leverages the fact that the dynamic system states are augmented with the APBM parameters:
     x
of some states is precisely known (e.g., the position derivative xt+1 g (ft (xt ), xt , dt ; θ APBM ) v
is velocity). = t
+ tθ (44)
θ APBM
t+1 θ APBM
t vt
The data-augmented modeling represented by the data-
where a near-constant state transition process is introduced for
augmented physics-based model (APBM) combines the
θ APBM , allowing one to cast the APBM learning as a filtering
physics- and data-driven components into a single model,
problem. v θt is a “small” noise and is introduced to avoid
which reads
numerical issues [30]. Note that such noise can also allow
xt+1 = g (ft (xt ), xt , dt ; θ APBM
t ) + v APBM
t , (43) θ APBM
t to drift over time and eventually evolve if the model is
time-varying.
2 A structure similar to the general one can be considered for the continuous-
time models, where the DNN represents the function in the differential state 3 These denote data related to the system available to the user but not
equation, and then the Euler discretization is used to obtain a discrete-time used in the PBM as additional inputs to avoid overly complex models (e.g.,
model used for the estimation. ionospheric models, weather forecasts).

15
 Physics xt xt+1
xt+1 = f (xt ; λ)

λ
xt DNN xt+1 xt PINN xt+1
DNN
xt+1 = g(xt ; θ DNN ) xt+1 = g(xt ; θ DNN )

(a) General SS structure (b) General physics-informed SS structure (c) Fixed SS structure

xt xLt+1 xt+1
xLt+1 = F xt +
+ DNN
xt
(1)
xt+1 = g (1) (xt ; θ DNN ) xt+1 g (f t (xt , xt , dt ; θDNN )
xt xt+1
DNN xNL
t+1 (2) PBM
xt+1 = f (2) (xt ; λ) dt
xNL
t+1 = g(xNL
t ;θ
DNN
) f t (xt ; λ)

(d) Incremental SS structure (e) Hybrid physics-based SS structure (f) APBM SS structure

Fig. 9: Structures for SS-oriented learning.

Another important component regarding the APBM learning keeping the APBM parameters fixed. In the online setting, the
process is the need for a control mechanism to prevent the training approach becomes a conventional filtering problem
DNN component from overpowering the PBM. The prevention that continuously adapts both states and model parameters over
can be achieved by imposing appropriate constraints over time, starting from some initial condition x̂0 , θ̂ APBM
0 . Again,
θ APBM . A Bayesian filtering based approach is to introduce the fast convergence of the Bayesian filter allows for quick
pseudo-observation equations into the observation model [61].
reaction of the data-driven component over time, correcting
Let θ̄ be a vector such that g ft (xt ), xt , dt ; θ APBM
t = θ̄ = the PBM to adapt to new conditions and improving the state
ft (xt ), then the observation model in (3b) can be augmented estimation performance. This feature is extremely important
as: when dealing with non-stationary dynamics that continuously
  
yt h (x )
  
wt change over time. Both strategies can be combined, where the
= t APBMt + (45) offline training solution is used as the initial condition for the
θ̄ θt wθt
online procedure, which, in turn, keeps updating both states
where the bottom equation acts as a constraint, forcing θ APBM
t to and parameters during the test.
be in the vicinity of θ̄; and the distribution of wθt governs the
trade-off between regularization and data fit. In the case where Discussion: The SS-oriented AI-augmented KF design re-
the noise terms in the SS model defined in Equations (44) lies on augmenting the physics-based component, namely
and (45) are Gaussian, cov{ω t } = R and cov{ω θt } = ηI, the “deterministic” part of the state equation with a data
the Bayesian filtering solution can be cast as a sequence of component. As seen in the box entitled ‘APBM with Controlled
minimization problems for every time step t: Additive Structure’, a solution might lead to an additive APBM
state equation (47). The data component is designed to extract
(x̂t , θ̂ APBM
t ) = arg min ∥y t − ht (xt )∥2R−1 + η∥θ̄ − θ t ∥2 a time-correlated component of the discrepancy between the
(xt ,θ t )
  pure PBM (3a) and the TM, that is usually embraced by the
2 overbounded state noise v t . As a consequence, the APBM
+ ∥xt − g ft (xt−1 ), xt−1 , dt−1 ; θ̂ APBM
t−1 ∥[P̂ x ]−1
t|t−1
state noise v APBM
t has different statistical properties from the
+ ∥θ t − θ̂ t−1 ∥2[P̂ θ ]−1
APBM
(46) PBM v t and for optimal performance of the KF, the noise
t|t−1
properties have to be identified. In [64], the noise properties
where η ≥ 0 controls the strictness of the regularization added identification of the APBM was discussed and illustrated using
by the pseudo-observation equation in (45). correlation and maximum likelihood methods. Utilizing the
identified state noise covariance matrix in the KF led to
Offline vs Online Training of APBMs: One interesting
significant improvement of the estimate consistency.
aspect of leveraging the Bayesian filtering approach for joint
parameter and state estimation lies in its equivalence to a Following the terminology used in system identification
second-order Newton method [62], leading to fast convergence [65], the PBM augmentation with DNN belongs into block-
and making it suitable for both online and offline training. oriented “slate-gray” models. The idea of block-oriented
The training methodology seamlessly allows for either offline, models is “to build up structures from simple building blocks”
online, or both (offline-online) training of the system dynam- [65], which allows physical insight and data-oriented flexible
ics. In the offline scenario, the filtering with the augmented complement. Note that besides the DNN, the PBM can be
model can be performed over the multiple sequences (and augmented with the nonlinear autoregressive moving-average
multiple epochs) in Du if system states, x0 , are properly model as in [66], where the data component is used for drift
initialized for every data sequence in Du [63]. Once training compensation. The concept of the APBM can also be found in
is completed, a standard Bayesian filtering approach can be the area of deterministic models with a completely measured
used to update only the system states, xt , over time while state. In this area, the “deterministic” version of the APBM

16
 APBM with Controlled Additive Structure
The general APBM in (43) can be particularized as
an additive augmentation of the PBM [15]
xt+1 = ϕ0 ft (xt ) + ϕ1 g(xt ; θ DNN ) + v APBM
t , (47)
where the APBM parameters θ APBM = [ϕ0 , ϕ1 , θ DNN ]
may be estimated offline or online, with supervised
or unsupervised training strategies. To ensure, that
the data-based component does not overrule the
PBM component, we can use constrained state
estimation algorithms where the parameters of the
DNN are encouraged to be close to a nominal
value θ̄ that keeps the DNN contribution limited.
State estimation can be interpreted as regularized
optimization problems, such as
(x̂1:t , θ) = arg min LDu (y 1:t , ŷ 1:t ) + ηR(x1:t , θ)
(x1:t ,θ)
Fig. 10: Illustrative APBM with additive structure.
where LDu and R are cost and regularization functionals, and η ∈ R+ is a regularization parameter governing the
trade-off between model fit and regularization. In the context of Kalman filtering, the minimized cost function is
the MSE of the state given observations and dynamics (47) model and the control of the DNN can be effectively
controlled through the regularization term, for which several implementation options are possible [15].

was successfully used for characterization of the unmodelled 1) Domain Knowledge: The presented methodologies sub-
acceleration caused, e.g., by the quadrotor drag [67]. stantially vary in the level of knowledge and characterization
of the underlying SS model required, corresponding to chal-
lenges C1 and C2. The extreme cases are those of fully model-
VI. C OMPARATIVE S TUDY
based filters (such as the standard KF and its variants) that
The previous sections presented a set of design approaches require full and accurate knowledge of the SS model, and those
and concrete algorithms fusing Kalman-type filtering and AI of end-to-end DNNs, that are full model-agnostic (yet do not
techniques. While all these methods tackle the same task of incorporate available characterization when such is provided).
state estimation in dynamic systems, they notably vary in Among the methodologies representing AI-augmented KFs,
their strengths, requirements, and implications. To highlight the usage of an external DNN to classic Kalman-type tracking
the interplay between the approaches mentioned above and typically requires the same level of domain knowledge needed
provide an understanding of how practitioners should prioritize to apply its classic counterpart. When the DNN is applied
one approach over the others, we next provide a comparative in parallel as a learned correction term, then the complete
study. We divide this study into two parts: we first present SS model should be known, while applying a learned pre-
a qualitative comparison, which pinpoints the conceptual processing module can compensate for unknown and in-
differences between the approaches in light of the identified tractable observation modeling. Settings in which one has
desired properties P1-P5 and challenges C1-C4. We then detail full knowledge of the observations model but does not know
a quantitative study for a representative scenario of tracking the state evolution model are the focus of methods based on
in challenging dynamics to capture the regimes in which each learned state estimation and the SS-oriented DDM and PINN.
approach is expected to be preferable. Knowledge (though possibly an approximated one, as in C1) of
the functions f (·) and h(·) is required by techniques that learn
the KG, while some modeling of these functions is needed
A. Qualitative Comparison by SS-oriented approaches based on parameter learning and
Here, we discuss the relationship and individual gains of the APBM.
different approaches over each other in terms of key concep- 2) Interpretability: In interpretability of a state estimation
tual figures of merit that are not quantifiable in the same sense algorithm, we refer to the ability to explain the operation
as estimation performance on a given test-bed is. Based on the of each computation and associate its internal features with
identified desired properties P1-P5 and challenges C1-C4 of concrete meaning, as in P3. This high level of interpretability
conventional Kalman-type algorithms, we focus on qualitative is naturally provided by purely model-based KF-type algo-
comparison in terms of domain knowledge, interpretability, rithms, as well as by SS-oriented AI techniques that learn the
uncertainty extraction, adaptability, target family of SS models, parameters of a pre-determined parametric SS model [55], sim-
and learning framework. The comparison detailed below is ilarly to conventional system identification. On the contrary,
summarized in Table I. algorithms employing end-to-end DNNs for state estimation

17
 Approach SS Knowledge Interpretability Uncertainty Adaptability SS Model Learning

End-to-end Generic Fully model-agnostic Black-box Only outputs state Not adaptive Can be non-linear Supervised learning from
DNN and non-Gaussian large data sets
KF-inspired Fully model-agnostic Connection of black- Estimates both state Not adaptive Can be non-linear Supervised learning from
(e.g., RKN box DNNs and uncertainty and non-Gaussian large data sets
[7])

External Learned pre- Requires state evolution Input transformation Tracks both state and Adapts to known State evolution Preferably supervised
DNN process is black-box, while uncertainty variations only in should be simple for end-to-end learning,
tracking based on the the state evolution (preferably linear) though DNN can also
latent representation and Gaussian be trained as a feature
is fully interpretable extractor, possibly
unsupervised
Learned Requires (estimated) SS Highly interpretable Tracks both state and Designed for a spe- Should be Gaussian Supervised learning from
correction representation uncertainty cific SS model with simple non- moderate data sets
(e.g., [19]) linearities

Integrated Learned Requires (estimates) of Processing follows Can be extracted in Requires re- Can be non-linear Preferably supervised for
DNN KG (e.g., h(·) and f (·) the same pipeline some scenarios [43] training [21] or and non-Gaussian end-to-end learning, can
KalmanNet as standard EKF, hypernetworks [44] also be trained unsuper-
[16]) with KG computation for adaptivity vised with additional do-
being black-box main knowledge [21]
Learned State Requires SS observation Posterior update is Tracks both state and Requires re-training State evolution Unsupervised learning
Estimation model and does not re- interpretable and uncertainty for adaptivity model can be non-
(e.g., DANSE quire SS state evolution resembles that of SS- linear, observation
[20]) model oriented KF, while model should be
state prediction is linear, Gaussian
black-box
SS- DDM (e.g., Fully model-agnostic The state evolution is Tracks both state and Not adaptive Can be non-linear Supervised learning
Oriented fully data- (DNN) or some black-box, while the uncertainty and non-Gaussian
DNN driven model knowledge of the filter operation is in-
or PINN [68]) model required (PINN). terpretable
The observation model
is assumed to be known.
Parameter Requires models of h(·) Fully interpretable Tracks both state and Not adaptive Can be non-linear Unsupervised learning
learning (e.g., and f (·) uncertainty and non-Gaussian
[55])
APBM (e.g., Requires models of h(·) Highly interpretable Tracks both state and Fully adaptive Can be non-linear Unsupervised learning
[15]) and f (·) uncertainty and non-Gaussian

Model- Kalman-type Requires (accurate) SS Fully interpretable Tracks both state and Fully adaptive Should be Gaussian Fully model-based
Based filters (e.g., representation uncertainty with mild non-
EKF) linearities

TABLE I: Qualitative comparison between the considered approaches.

are essentially black-box methods. which implement conventional filtering on top of a learned
The level of interpretability varies when considering algo- state evolution model, as well as task-oriented architecture
rithms that augment classic algorithms with DNNs, such that that augments the prior state prediction (as in [69]) or utilize
some of the computations are based on principled statisti- DNNs external to model-based filters provide uncertainty in
cal models, while some are based on black-box data-driven the update computation.
pipelines. For instance, the internal features of task-oriented Among AI-aided algorithms that do not fully preserve the
designs with integrated DNNs are exactly those of standard posterior covariance propagation of classic KFs, the ability to
Kalman-type algorithms, while some of the internal computa- provide uncertainty varies between the approaches. Generic
tions – such as the computation of the KG in KalmanNet or the end-to-end DNNs that track only the state do not offer such
propagation of the prior state moments in DANSE – are carried measures, while KF-inspired architectures are designed to
out by black-box DNNs. A higher level of interpretability is output error covariances via their update DNNs. AI-aided
provided when the DNN is applied in parallel to a fully model- KFs with learned KG (such as KalmanNet) are specifically
based and interpretable algorithm for learned correction, as designed to bypass the need to propagate second-order mo-
in [19]. ments such as the posterior covariance. Still, as noted in the
3) Uncertainty: The ability to provide faithful uncertainty box entitled ‘Uncertainty Extraction from Learned KG‘ on
measures (as in P4) arises in classic Kalman-type algorithms Page 12, in some settings, one can still extract uncertainty
from their inherent tracking of the error (posterior) covariance measures from their internal KG features.
matrix. As the posterior update is an integral aspect of the 4) Adaptability: As noted in P2, classic KF-type algorithms
update step of conventional Kalman filtering, any algorithm are adaptable to temporal variations in the statistical model
that fully implements this computation provides uncertainty governing the underlying dynamic system. Assuming that
measures along with its state estimate. This property is exhib- one can identify and characterize the variations, the updated
ited by fully model-based Kalman-type filters, as well as by SS model parameters are simply substituted into the filter
some AI-aided KFs. Specifically, both SS-oriented designs, equations. End-to-end DNNs, in which there is no explicit

18
dependence on the SS model parameters, and these are implic- vised in some settings. Methods such as DANSE, SS-oriented
itly embedded in the learned weights, adapting the SS models parameter learning, and APBM are specifically designed to be
that deviate from those observed in training necessitates re- trained based solely on observations, i.e., unsupervised.
training, i.e., they are not adaptive.
Task-oriented AI-aided KFs are trained in a manner that
B. Quantitative Comparison
is entangled with the operation of the augmented Kalman-
type algorithm. Accordingly, while algorithms with integrated To illustrate the performance of the considered state estima-
and external DNNs typically use the SS model parameters in tion algorithms, we provide a dedicated experimental study4 .
their overall processing, their DNN modules are fixed, and are We consider the task of tracking the nonlinear movement of a
suitable for the SS models observed in training. Accordingly, free particle in three-dimensional space (m = 3) from noisy
adaptations require re-training, where some level of variations position observations. The Lorenz Attractor, a chaotic solution
can still be coped with using hypernetworks [44]. An exception to the Lorenz system of ordinary differential equations, defines
is the usage of external DNNs for feature extraction, which the continuous-time state evolution of the particle’s trajectory.
is typically designed to map complex observations into a To enhance the challenge of this tracking case, we introduce
simplified observation model and thus are not affected by additional uncertainty due to a sampling-time mismatch in
variations in the state evolution model. Among SS-oriented the observation process. While the underlying ground truth
approaches, adaptivity is provided by the design of APBM, synthetic trajectory is
generated using a high-resolution time
while alternatives based on, e.g., DDM or PINN architectures interval ∆τ = 10−5 , the tracking filter can access only
1
need retraining when the state evolution statistics change. noisy observations decimated at a rate of 2000 , resulting in
5) SS Model Type: A key distinct property among the a decimated process with ∆t = 0.02.
presented methodologies lies in the family of SS models for To ensure a fair evaluation, we restrict any populated evo-
which each algorithm is suitable. As noted in C3, fully model- lution model of the tracking filter, if it exists, to discrete time
based KF-type algorithms are most suitable for Gaussian SS with ∆t ≥ 0.02. Further details about the Lorenz Attractor
models with simple and preferably not highly non-linear state evolution model can be found in [39]. The averaged MSE
evolution and observation models. On the opposite edge of values and their standard deviation values for filtering 10
the spectrum are end-to-end model-agnostic DNNs, that can be sequences with a length of T = 3000 time steps are reported
applied in complex and intractable dynamic systems, provided in Table II. There, we compare the MSE achieved by using
with sufficient data corresponding to that task. the noisy observations as state estimate (Noise), that that of
Among task-oriented AI-augmented KFs, architectures em- filtering via the model-based EKF and particle filter (PF); the
ploying external DNNs as learned correction terms are suitable DNN-integrated KalmanNet and DANSE; and two forms of
for similar SS models as classic KFs, being geared towards the SS-oriented APBM, employing offline and online learning.
settings where the latter is applicable. External DNNs that In the comparative performance reported in Table II, the
pre-process observations facilitate coping with complex ob- model-based filters, i.e., the EKF and PF manage to improve
servation models, while integrated DNNs for learned state upon using noisy observations, but suffer from an error floor
estimation overcome complex state evolution models, with due to their sampling mismatch. All considered AI-aided
each approach requiring the remainder of the SS model to filters manage to improve upon this error floor. Specifically,
be simple (preferably linear), fully known, and Gaussian. KalmanNet, which is trained in a supervised manner, achieves
Augmenting KFs with learned KG notably enhances the the best performance, improving by approximately 5 dB in
family of applicable SS models, not requiring Gaussianity MSE compared to the model-based algorithms. Among the
and modeling of any of the noise terms, and learning to cope AI-aided filters that are trained from unlabeled data, DANSE
with non-linear and possibly approximated state evolution and achieves the best performance. APBM, which is designed to
observation functions. The same holds for the reviewed SS- boost adaptivity, achieves MSE within a small gap of DANSE
oriented methods that can all cope with non-linear and non- when trained offline. These quantitative results complement
Gaussian dynamics. the conceptual comparison provided in Table I, in revealing
6) Learning Framework: The learning framework is spe- the interplay between the reviewed methods for combining
cific to algorithms that incorporate AI, and encapsulates the deep learning with classic KF-type filtering.
amount of data and the reliance on its labeling. Accordingly,
fully model-based algorithms that rely on given mathematical VII. F UTURE R ESEARCH D IRECTIONS
modeling of the SS representation do not involve learning
The tutorial-style presentation of designs, algorithms, and
in their formulation, while black-box DNNs typically require
experiments of AI-aided KFs indicate potential gains of proper
learning from large volumes of labeled data sets.
fusion of model-based tracking algorithms and data-driven
Designs combining partial domain knowledge with deep
deep learning techniques. These, in turn, give rise to several
learning techniques typically result in a dominant induc-
core research directions that can be explored to further unveil
tive bias that allows training with limited data, compared
the potential, strengths, and prospective use cases of such de-
to black-box end-to-end DNNs. Methods based SS-oriented
signs. Exploring these directions is expected to further advance
DDM/PINN, external DNN, and learned integrated KG DNN,
typically require this data to be labeled, though for learned 4 The source code for all simulated algorithms and the hyperparameters used
pre-processing and for KalmanNet one can also train unsuper- can be found online at https://github.com/ShlezingerLab/AI Aided KFs

19
 TABLE II: MSE [dB] - Lorenz Attractor with Sampling Mismatch.
Noise EKF PF KalmanNet DANSE APBM (offline) APBM (online)
-0.024 -6.316 -5.333 -11.106 -10.115 -9.457 -7.452
± 0.049 ± 0.135 ± 0.136 ± 0.224 ± 0.163 ± 0.231 ± 0.548

the development and understanding of algorithmic tools to [4] J. Durbin and S. J. Koopman, Time Series Analysis by State Space
jointly leverage classic SS model-based KF-type algorithms Methods. Oxford University Press, 05 2012, vol. 38.
[5] S. Gannot and A. Yeredor, “The Kalman filter,” Springer Handbook of
with emerging deep learning techniques, and preserving the Speech Processing, pp. 135–160, 2008.
individual strengthens of each approach that has a bearing on [6] I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press,
the potential of alleviating some of the core challenges shared 2016.
[7] P. Becker, H. Pandya, G. Gebhardt, C. Zhao, C. J. Taylor, and
by a broad range technologies tracking dynamic systems G. Neumann, “Recurrent Kalman networks: Factorized inference in
ans state estimation. We thus conclude this article with a high-dimensional deep feature spaces,” in International Conference on
discussion of some of the open challenges that can serve as Machine Learning, 2019, pp. 544–552.
[8] R. Krishnan, U. Shalit, and D. Sontag, “Structured inference networks
future research directions. for nonlinear state space models,” in Proceedings of the AAAI Confer-
1) Time-varying SS models and adaptation: A practical state ence on Artificial Intelligence, vol. 31, no. 1, 2017.
estimation method might need to cater for a scenario [9] S. Kuutti, R. Bowden, Y. Jin, P. Barber, and S. Fallah, “A survey of
deep learning applications to autonomous vehicle control,” IEEE Trans.
where the observation model (1b) is mismatched between Intell. Transp. Syst., vol. 22, no. 2, pp. 712–733, 2020.
training and testing stages, and/or the observation or [10] J. Chen and X. Ran, “Deep learning with edge computing: A review,”
state evolution functions are time-varying without a clear Proc. IEEE, vol. 107, no. 8, pp. 1655–1674, 2019.
[11] N. Shlezinger, J. Whang, Y. C. Eldar, and A. G. Dimakis, “Model-based
pattern of time-varying nature. Typically AI-aided KFs deep learning,” Proc. IEEE, vol. 111, no. 5, pp. 465–499, 2023.
are tied to a fixed observation model, typically not time- [12] A. Klushyn, R. Kurle, M. Soelch, B. Cseke, and P. van der Smagt,
varying, and remains same between the the training stage “Latent matters: Learning deep state-space models,” Advances in Neural
Information Processing Systems, vol. 34, pp. 10 234–10 245, 2021.
and inference stage. Therefore, the future research ques- [13] H. Coskun, F. Achilles, R. DiPietro, N. Navab, and F. Tombari, “Long
tion is how to design AI-aided state estimation methods short-term memory Kalman filters: Recurrent neural estimators for pose
that can adapt to time-varying SS models, mainly for regularization,” in Proceedings of the IEEE International Conference on
Computer Vision, 2017, pp. 5524–5532.
time-varying observation models. [14] D. Gedon, N. Wahlström, T. B. Schön, and L. Ljung, “Deep state
2) Non-Markovian SS models: Throughout this article and space models for nonlinear system identification,” IFAC-PapersOnLine,
being consistent with usual practice, the prevalent state vol. 54, no. 7, pp. 481–486, 2021.
[15] T. Imbiriba, O. Straka, J. Dunı́k, and P. Closas, “Augmented physics-
evolution model is Markovian in nature (see (1a)). Nat- based machine learning for navigation and tracking,” IEEE Trans.
urally, it is a quest to design methods that can exploit Aerosp. Electron. Syst., vol. 60, no. 3, pp. 2692–2704, 2024.
short and long-term memories in state evolution, which [16] G. Revach, N. Shlezinger, X. Ni, A. L. Escoriza, R. J. van Sloun, and
Y. C. Eldar, “KalmanNet: Neural network aided Kalman filtering for
means non-Markovian state evolution. partially known dynamics,” IEEE Trans. Signal Process., vol. 70, pp.
3) Non-Gaussian SS models: Most of the directions dis- 1532–1547, 2022.
cussed in the article focused on Gaussian noises in SS [17] I. Buchnik, G. Revach, D. Steger, R. J. van Sloun, T. Routtenberg,
and N. Shlezinger, “Latent-KalmanNet: Learned Kalman filtering for
models. Extensions to non-Gaussian noises encountered tracking from high-dimensional signals,” IEEE Trans. Signal Process.,
in real problems will require more complex algorithms vol. 72, pp. 352–367, 2023.
with high computational complexity, which is challeng- [18] K. Pratik, R. A. Amjad, A. Behboodi, J. B. Soriaga, and M. Welling,
“Neural augmentation of Kalman filter with hypernetwork for channel
ing, especially when learning the large-scale DNN pa- tracking,” in IEEE Global Communications Conference (GLOBECOM),
rameters. 2021.
4) Distributed AI-aided KFs: While there exists considerable [19] V. G. Satorras, Z. Akata, and M. Welling, “Combining generative and
discriminative models for hybrid inference,” in Advances in Neural
research on distributed KFs, such as the work of [70] Information Processing Systems, 2019, pp. 13 802–13 812.
for sensor networks, there is little attention currently [20] A. Ghosh, A. Honoré, and S. Chatterjee, “DANSE: Data-driven non-
to design distributed AI-aided KFs. Design of them for linear state estimation of model-free process in unsupervised learning
setup,” IEEE Trans. Signal Process., vol. 72, pp. 1824–1838, 2024.
federated learning and edge computing can be a new [21] G. Revach, N. Shlezinger, T. Locher, X. Ni, R. J. G. van Sloun, and Y. C.
research direction. Eldar, “Unsupervised learned Kalman filtering,” in European Signal
5) Robust training: Outliers appearing in the SS models, Processing Conference (EUSIPCO), 2022, pp. 1571–1575.
[22] S. N. Aspragkathos, G. C. Karras, and K. J. Kyriakopoulos, “Event-
mainly in the observation model (1b), may substantially triggered image moments predictive control for tracking evolving fea-
affect the training process. The approaches should address tures using UAVs,” IEEE Robot. Autom. Lett., vol. 9, no. 2, pp. 1019–
this issue by providing robust training. 1026, 2024.
[23] A. Milstein, G. Revach, H. Deng, H. Morgenstern, and N. Shlezinger,
R EFERENCES “Neural augmented Kalman filtering with Bollinger bands for pairs
trading,” IEEE Trans. Signal Process., vol. 72, pp. 1974–1988, 2024.
[1] R. E. Kalman, “A new approach to linear filtering and prediction [24] A. Gu, I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. Ré,
problems,” Journal of Basic Engineering, vol. 82, no. 1, pp. 35–45, “Combining recurrent, convolutional, and continuous-time models with
1960. linear state space layers,” Advances in neural information processing
[2] S. F. Schmidt, “The Kalman filter-its recognition and development for systems, vol. 34, pp. 572–585, 2021.
aerospace applications,” Journal of Guidance and Control, vol. 4, no. 1, [25] T. Jebara, Machine learning: discriminative and generative. Springer
pp. 4–7, 1981. Science & Business Media, 2012, vol. 755.
[3] M. S. Grewal and A. P. Andrews, “Applications of Kalman filtering in
aerospace 1960 to the present [historical perspectives],” IEEE Control
Syst. Mag., vol. 30, no. 3, pp. 69–78, 2010.

20
[26] N. Shlezinger and T. Routtenberg, “Discriminative and generative learn- [49] J. Suykens, B. Moor, and J. Vandewalle, “Nonlinear system identification
ing for linear estimation of random signals [lecture notes],” IEEE Signal using neural state space models, applicable to robust control design,”
Process. Mag., vol. 40, no. 6, pp. 75–82, 2023. International Journal of Control, vol. 62, no. 1, p. 129–152, 1995.
[27] L. Zhou, Z. Luo, T. Shen, J. Zhang, M. Zhen, Y. Yao, T. Fang, [50] D. Masti and A. Bemporad, “Learning nonlinear state–space models
and L. Quan, “KFNet: Learning temporal camera relocalization using using autoencoders,” Automatica, vol. 129, pp. 0005–1098,, 2021.
Kalman filtering,” in Proceedings of the IEEE/CVF Conference on [51] A. Gorji and M. Menhaj, “Identification of nonlinear state space models
Computer Vision and Pattern Recognition, 2020, pp. 4919–4928. using an MLP network trained by the EM algorithm,” in 2008 IEEE
[28] S. Jouaber, S. Bonnabel, S. Velasco-Forero, and M. Pilte, “NNAKF: International Joint Conference on Neural Networks, 2008, p. 53–60.
A neural network adapted Kalman filter for target tracking,” in IEEE [52] Y. Wang, “A new concept using LSTM neural networks for dynamic
International Conference on Acoustics, Speech and Signal Processing system identification,” in 2017 American Control Conference (ACC,
(ICASSP), 2021, pp. 4075–4079. 2017, pp. 5324–5329,.
[29] A. Bensoussan, Stochastic control of partially observable systems. [53] M. Fraccaro, S. Sønderby, U. Paquet, and O. Winther, “Sequential
Cambridge University Press, 1992. neural models with stochastic layers,” in Advances in neural information
[30] S. Särkkä and L. Svensson, Bayesian filtering and smoothing. Cam- processing systems, 2016, p. 29.
bridge University Press, 2023, vol. 17. [54] F. Arnold and R. King, “State–space modeling for control based
[31] H. E. Rauch, F. Tung, and C. T. Striebel, “Maximum likelihood estimates on physics-informed neural networks,” Engineering Applications of
of linear dynamic systems,” AIAA Journal, vol. 3, no. 8, pp. 1445–1450, Artificial Intelligence, vol. 101, p. 104195, 2021. [Online]. Available:
1965. https://www.sciencedirect.com/science/article/pii/S0952197621000427
[32] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan, Estimation with applica- [55] Y. Bao, J. M. Velni, A. Basina, and M. Shahbakhti, “Identification
tions to tracking and navigation: theory algorithms and software. John of state-space linear parameter-varying models using artificial neu-
Wiley & Sons, 2004. ral networks,” IFAC-PapersOnLine,Volume, vol. 53, pp. 5286–5291,
[33] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, 2405–8963,, 2020.
Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances [56] M. Schoukens, “Improved initialization of state-space artificial neural
in Neural Information Processing Systems, 2017, pp. 5998–6008. networks,” arXiv preprint arXiv:2103.14516, 2021.
[34] A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with [57] M. Forgione and D. Piga, “Model structures and fitting criteria for
selective state spaces,” arXiv preprint arXiv:2312.00752, 2023. system identification with neural networks,” in IEEE International Con-
[35] A. Gu, K. Goel, and C. Ré, “Efficiently modeling long sequences ference Application of Information and Communication Technologies,
with structured state spaces,” International Conference on Learning 2020.
Representations (ICLR), 2022. [58] S. Tang, T. Imbiriba, J. Dunı́k, O. Straka, and P. Closas, “Augmented
[36] N. Shlezinger, Y. C. Eldar, and S. P. Boyd, “Model-based deep learning: Physics-Based Models for High-Order Markov Filtering,” Sensors,
On the intersection of deep learning and optimization,” IEEE Access, vol. 24, no. 18, 2024. [Online]. Available: https://www.mdpi.com/
vol. 10, pp. 115 384–115 398, 2022. 1424-8220/24/18/6132
[37] A. H. Li, P. Wu, and M. Kennedy, “Replay overshooting: Learning [59] T. Imbiriba, A. Demirkaya, J. Dunı́k, O. Straka, D. Erdoğmuş, and
stochastic latent dynamics with the extended Kalman filter,” in IEEE P. Closas, “Hybrid neural network augmented physics-based models for
International Conference on Robotics and Automation (ICRA), 2021. nonlinear filtering,” in International Conference on Information Fusion
[38] L. Xu and R. Niu, “EKFNet: Learning system noise statistics from (FUSION). IEEE, 2022.
measurement data,” in IEEE International Conference on Acoustics, [60] R. Krishnan, U. Shalit, and D. Sontag, “Deep Kalman filters,” arXiv
Speech and Signal Processing (ICASSP), 2021, pp. 4560–4564. preprint arXiv:1511.05121., 2015.
[39] G. Revach, X. Ni, N. Shlezinger, R. J. van Sloun, and Y. C. Eldar, [61] D. Simon, “Kalman filtering with state constraints: a survey of linear
“RTSNet: Learning to Smooth in Partially Known State-Space Models,” and nonlinear algorithms,” IET Control Theory & Applications, vol. 4,
IEEE Trans. Signal Process., vol. 71, pp. 4441–4456, 2023. no. 8, pp. 1303–1318, 2010.
[40] G. Choi, J. Park, N. Shlezinger, Y. C. Eldar, and N. Lee, “Split- [62] J. Humpherys, P. Redd, and J. West, “A fresh look at the kalman filter,”
KalmanNet: A robust model-based deep learning approach for state SIAM review, vol. 54, no. 4, pp. 801–823, 2012.
estimation,” IEEE Trans. Veh. Technol., vol. 72, no. 9, pp. 12 326–12 331, [63] A. Bemporad, “Recurrent neural network training with convex loss
2023. and regularization functions by extended kalman filtering,” IEEE Trans.
[41] J. Wang, X. Geng, and J. Xu, “Nonlinear Kalman filtering based on Autom. Control, vol. 68, no. 9, pp. 5661–5668, 2023.
self-attention mechanism and lattice trajectory piecewise linear approx- [64] J. Dunik, O. Straka, O. Kost, S. Tang, T. Imbiriba, and P. Closas, “Noise
imation,” arXiv preprint arXiv:2404.03915, 2024. identification for data-augmented physics-based state-space models,” in
[42] A. N. Putri, C. Machbub, D. Mahayana, and E. Hidayat, “Data driven Accepted for IEEE Workshop on Signal Processing Systems (SiPS 2024),
linear quadratic Gaussian control design,” IEEE Access, vol. 11, pp. Cambridge, MA, USA, 2024.
24 227–24 237, 2023. [65] L. Ljung, “Perspectives on system identification,” IFAC Proceedings
[43] Y. Dahan, G. Revach, J. Dunik, and N. Shlezinger, “Uncertainty Volumes, vol. 41, no. 2, pp. 7172–7184, 2008, 17th IFAC World
quantification in deep learning based Kalman filters,” arXiv preprint Congress.
arXiv:2309.03058, 2023. [66] D. Li, J. Zhou, and Y. Liu, “Recurrent-neural-network-based unscented
[44] X. Ni, G. Revach, and N. Shlezinger, “Adaptive KalmanNet: Data-driven Kalman filter for estimating and compensating the random drift of
Kalman filter with fast adaptation,” in IEEE International Conference MEMS gyroscopes in real time,” Mechanical Systems and Signal Pro-
on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 5970– cessing, vol. 147, p. 107057, 2021.
5974. [67] G. Torrente, E. Kaufmann, P. Föhn, and D. Scaramuzza, “Data-driven
[45] A. Sakai and Y. Kuroda, “Discriminative parameter training of unscented MPC for quadrotors,” IEEE Robot. Autom. Lett., vol. 6, no. 2, pp. 3769–
Kalman filter,” IFAC Proceedings Volumes, vol. 43, no. 18, pp. 677–682, 3776, 2021.
2010, 5th IFAC Symposium on Mechatronic Systems. [68] M. Raissia, P. Perdikarisb, and G. Karniadakis, “Physics-informed neural
[46] Z. Fan, D. Shen, Y. Bao, K. Pham, E. Blasch, and G. Chen, “RNN- networks: A deep learning framework for solving forward and inverse
UKF: Enhancing hyperparameter auto-tuning in unscented kalman filters problems involving nonlinear partial differential equations,” Journal of
through recurrent neural networks,” in 27th International Conference on Computational physics, vol. 378, pp. 686–707, 2019.
Information Fusion, Venice, Italy, 2024. [69] A. Ghosh, J. Hong, D. Yin, and K. Ramchandran, “Robust fed-
[47] A. Chiuso and G. Pillonetto, “System identification: A machine learning erated learning in a heterogeneous environment,” arXiv preprint
perspective,” Annual Review of Control, Robotics, and Autonomous arXiv:1906.06629, 2019.
Systems, vol. 2, no. 1, pp. 281–304, 2019. [70] R. Olfati-Saber, “Distributed kalman filtering for sensor networks,” in
[48] L. Ljung, C. Andersson, K. Tiels, and T. B. Schön, “Deep learning and 2007 46th IEEE Conference on Decision and Control, 2007, pp. 5492–
system identification,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 1175– 5498.
1181, 2020, 21st IFAC World Congress.

21
2017 American Control Conference
Sheraton Seattle Hotel
May 24–26, 2017, Seattle, USA

Repetitive Control Design and Implementation for Periodic

Motion Tracking in Aerial Robots
Xiang He, Dejun Guo and Kam K. Leang†
Design, Automation, Robotics and Control (DARC) Lab
Univ. of Utah Robotics Center, Dept. of Mechanical Engineering, Salt Lake City, Utah 84112

Abstract— This paper describes the design and implemen- fact, a UAV equipped with sensing and imaging technologies
tation of a repetitive controller (RC) for tracking periodic may be required to track a prescribed trajectory in the field,
trajectories in aerial robots, such as unmanned aerial vehicles such as following a flight pattern over and over again while
(UAVs). The RC is ideally suited for situations where the robot is
required to execute a defined flight pattern, over and over again collecting data or searching for survivors following a natural
in a periodic fashion. Applications include precision agriculture, disaster. In these situations, the aerial robot is required to
and search and rescue, where the desired trajectory is often track a periodic trajectory in time, and do so with high
periodic in time and known a priori. Compared to iterative precision, as well as speed.
learning control (ILC), RC is a feedback-based approach which To this end, the contribution of this paper is the design
can be plugged into an existing feedback control loop to
improve the tracking of period trajectories. Herein, a plug- and implementation of a discrete-time repetitive controller
in type RC is applied to a micro aerial robot, and a systematic (RC) for tracking periodic trajectories for aerial robots. It
approach is presented for designing the control system. The is pointed out that the RC approach has received limited
plug-in nature of the design also allows users to enable/disable attention for motion control of aerial robots [12], but the
the controller on the fly. The proposed RC is also compared control structure is ideally suited for tracking periodic trajec-
to existing approaches, including no-reset ILC, which have
recently been applied to control aerial robots. The effectiveness tories [13], [14]. Recently, a related controller known as iter-
of the RC is demonstrated on a custom-designed experimental ative learning control (ILC), an iterative feedforward-based
micro aerial robot. Tracking of circle, square, and figure-eight controller, has been developed for aerial robots [15], [16],
trajectories are shown in both simulations and experiments. In [17]. The ILC and RC controllers are isomorphic [18], but
these cases, the RC reduces the average tracking error by over they differ in many respects, including implementation, sta-
58% compared to a well-tuned PID controller, and simulations
that compare to no-reset ILC show that the performance of RC bility conditions, and computational complexity. In general,
is less sensitive to changes in the reference trajectory. the RC is a time-domain, feedback-based approach which
does not require its initial conditions to be reset, whereas
I. I NTRODUCTION the ILC is an iteration-domain, feedforward technique and
The capabilities of aerial robots such as unmanned aerial often requires initial condition reset for convergence [19].
vehicles (UAVs) have recently improved significantly and However, a no-reset ILC was proposed in [20], then recently
thus their use in civil and commercial applications has grown studied for aerial robots [17]. It is shown that the proposed
immensely over the past few decades [1]. Applications of RC with its feedback structure is observed to be less sensitive
UAV technology include mapping [2], search and rescue [3], to changes in the desired trajectory, making it better suited
precision farming, space exploration [4], traffic manage- for tracking aggressive trajectories, compared to ILC. Exper-
ment [5], environmental monitoring [6], [7], and even for iment of the RC is applied to a custom-designed micro aerial
entertainment [8]. In fact, many small multi-rotor UAVs robot and the results show over 58% reduction in the tracking
can access indoor locations that may be hard to reach or error on average compared to a well-tuned PID controller.
unsafe for humans, as well as navigate through complex This paper is organized as follows. Section II presents
urban environments. Such UAVs are ideally suited for search prior related work. Section III introduces the RC design,
and rescue, law enforcement, and/or emergency response to followed by simulation and comparison between RC and
enhance situational awareness [9]. ILC in Section IV. The description of the experimental
Despite recent advances in design, sensing, and mo- system and modeling are presented in Section V. Section VI
tion control, precision autonomous navigation and trajectory describes the controller implementation and experimental
tracking remains a challenge, especially in environments results. Finally, concluding remarks and acknowledgments
subject to variable terrain and operating conditions such are found in Section VII and VIII, respectively.
as wind and other environmental disturbances [10], [11].
With the growing interest in using UAVs for many civilian II. P RIOR W ORK
and commercial applications, advanced control systems are Control of UAVs and aerial robots in general is an active
needed to enable safe and reliable autonomous operation. In field of research [1], [11], [21], [22]. A wide variety of
controllers have been developed, for example, linear and
† Corresponding author (E-mail: kam.k.leang@utah.edu). nonlinear controllers [23], [24] for precision motion control

978-1-5090-5992-8/$31.00 ©2017 AACC 5101

 (a)
and regulation (stabilizing at hover conditions). Specific PID
Outer loop Quadcopter system
examples include traditional proportional-integral-derivative r e u ua y
C0(z) G
(PID) control [23], model predictive control [25], linear + – +
+

quadratic regulation (LQR), dynamic feedforward, fuzzy kP(z) Phase-compensation

logic and genetic algorithms [21], [26], [27], and iterative w
learning control [15], [16], [17]. In particular, for tracking Γ(z)
of periodic trajectories the iterative feedforward-based ILC Repetitive control (plug-in style)
Repetitive controller, Γ(z)
was employed [15], [16], [17]. In this case, the robot would (b)
execute the same task over and over again, and the input e
+ z–N+m Q(z) w
would be updated, in a feedforward fashion, from one itera- +
Time delay Low-pass filter

tion to the next. Thus, performance relied on convergence

in the iteration domain, from one operating trial to the Γ(z)
next. One of the challenges of ILC is the need to reset the
(c)
initial conditions, and thus the robot needs to return to the Quadcopter system, G
starting position before executing another trial [28], [29]. For Fz
feedback-based tracking of periodic trajectories, repetitive rref Quadcopter
PID Control PID Control dynamics
control, a design based on the Internal Model Principle [30], δ p, ref e w
v f F

was developed [31], [32]. Traditional feedback-based con- δ q, ref + –

C1(z) +
–
C2(z) M D y

Mixer
trollers such as PID often require careful tuning and the q, p, r

residual tracking error persists from one operating cycle to δp , δq

the next. On the other hand, in feedback-based RC the error
reduces as the number of operating cycles increases [33]. u a={ δ p, ref , δ q, ref , r ref , Fz }

Specifically, the time-domain RC utilizes a signal generator

within the feedback loop to create a controller that provides Fig. 1. (a) Plug-in repetitive control scheme; (b) repetitive controller; (c)
infinite gain at the fundamental frequency of the reference the quadcopter system, consisting of two inner loop closed-loop controllers
for attitude stabilization of the quadcopter.
trajectory and its harmonics. One of the advantages of RC is
it requires minimal system information and only knowledge
about the period of the reference trajectory. Additionally, time. One of the main assumptions of RC is the period of
resetting of the initial conditions is not required and the the reference trajectory, Tp , is known a priori.
robot can operate continuously in a period fashion. Rather A block diagram of the overall discrete-time closed-loop
can showing convergence of a sequence of inputs like in control system for the quadcopter is shown in Fig. 1(a). In
ILC, the performance of RC hinges on showing stability of particular, the transfer function for the RC, Γ(z), shown in
the RC closed-loop feedback system. Fig. 1(b) is given by
It is emphasized that compared to traditional feedback con- w(z) Q(z)z−N +m
trol systems (such as PID), “learning”-type controllers have Γ (z) = = , (1)
e(z) 1 − Q(z)z−N +m
the ability to reduce the tracking error from one operating
where Q(z) is a unity-gain low-pass filter
period to the next, and the performance of such controllers
is generally better than traditional feedback controllers for a
Q(z) = , (2)
tracking periodic trajectories [13]. It is pointed out that the z+b
RC method has been widely studied for positioning systems, with |a| + |b| = 1. It is pointed out that the low-pass filter
disk drives, and machine tools [13], [14], but limited work Q(z) provides robustness and helps to stabilize the closed-
has been done on developing and implementing RC for loop system when the RC is added. The design of the cutoff
aerial robots [12]. The closest work to RC for aerial robots frequency ωQ of the low-pass filter to ensure stability are
as pointed out is ILC, which has been well studied [15], discussed below.
[16], [17]. Recent work on frequency-based ILC suggests In Fig. 1(a), it is first assumed that the closed-loop
similarities with RC [17]. Herein, a simulation comparison is dynamics of the quadcopter which includes the control loops
made between RC and ILC, where it is observed that the RC for attitude stabilization is represented by G, which can be
provides improved performance for tracking of aggressive nonlinear. The block diagram of the inner loop closed-loop
trajectories compared to ILC due, in part, to the feedback controllers for attitude stabilization is shown in Fig. 1(c).
structure of RC. The outer-loop PID controller C0 (z) is designed to track a
given reference trajectory r in three-dimensional space. The
III. R EPETITIVE C ONTROL D ESIGN phase-compensator P (z) is incorporated into the loop gain
Repetitive control tracks or rejects arbitrary periodic sig- to compensate for the overall phase lag of the closed-loop
nals, where the period of the repeating trajectory is fixed system, which can lead to instabilities. The design of P (z)
at Tp . To achieve this, a signal generator with period Tp will be discussed below.
is created using a pure delay z −N , where N = Tp /Ts is To analyze the basic stability properties of the RC, it is
the number of points per period Tp ; and Ts is the sampling first assumed that the quadcopter system can be modeled

5102
by the linear transfer function G(z). This assumption is IV. C OMPARISON TO ILC
reasonable at the hover state. By inspection of Fig. 1(a), the In this section, the proposed RC is compared to learning-
transfer function relating the reference trajectory r(z) and type controllers such as the reset ILC, no-reset ILC, and
the tracking error e(z) is given by FFT-based ILC [20], [17].
e(z) 1 − H(z) A. Iterative Learning Control and Comparison to RC
=   , (3) The requirement of standard ILC for convergence is initial
r(z) 1 − H(z) + (kP (z) − 1)H(z) + 1 Ĝ(z)
conditions need to be reset at the beginning of each run, thus
where H(z) = Q(z)z (−N +m) and Ĝ(z) = C0 (z)G(z). this process limits the ability to accomplish a continuous
Let S(z) = 1/(1 + Ĝ(z)) be the sensitivity function of task. The no-reset ILC [20] and the FFT-based ILC, however,
the feedback system without the repetitive controller, then due to the structure of the ILC method, require the controllers
by inspection the sensitivity function for the closed-loop to finish calculation of the additional control input in the first
repetitive controller shown in Fig. 1(a) is given by step of the repeating process. For no-reset ILC, the system
  is described by
e(z) 1 − H(z) S(z)
Src (z) = = . (4)
r(z)

1 − H(z) 1 − kP (z)Ĝ(z)S(z) yk = Hs uk + Ho xk , (13)
xk+1 = Hu uk + Hx xk , k ≥ 0, (14)
To determine the conditions for stability of the closed-loop
system, and to find the acceptable range of the RC gain k where xk , yk and uk denotes the state, the output trajectory
and the required phase-lead P (z), it is first assumed that and control input at the k-th iteration. Hs , Ho , Hu and Hx
1 − H(z) is bounded input - bounded output stable. Then, are matrices that describe the system. The control law is
by the small gain theorem, the system is stable when [34] designed to be
 
H(z) 1 − kP (z)Ĝ(z)S(z) < 1. (5) uk+1 = Lp uk + Le ek , (15)

Replacing z = ejωTs , where Lp = I N and Le are the learning gains. The no-
reset ILC requires the learning gain to be N numbers of
N × m matrix, according to [20], where N is the number
 
H(z) 1 − kP (z)Ĝ(z)S(z) =
of samples in one period and m is the number of states. As
H(ejωTs ) 1 − kejθ(ω) Ĝ(ejωTs )S(ejωTs ) < 1,
 
(6) in this case, N = 200 and m = 2. The matrix Te needs
to be calculated within 20 ms and due to the computational
for all ω ∈ (0, Tπs ), where the phase lead θ(ω) comes from complexity, this no-reset ILC is not being used widely. The
selecting other ILC approach, which also has no restriction on resetting
P (z) = z n , (7) the initial condition, is the FFT-based ILC [16]. This method
implements the learning control law in the frequency domain
where n is an integer, for ω ∈ (0, π/Ts ). It is pointed out and it can be expressed in the RC type structure. However,
that P (z) contributes a phase lead given by a linearized model of the plant is needed for the controller
θ(ω) = nTs ω. (8) design process. Convergence of the FFT-based ILC, with its
control law is given by
Now let T (z) represent the complimentary sensitivity
ek+1 = ek − G(z)(kG−1 (z)ek ), (16)
function of the closed-loop feedback system without RC,
T (z) = Ĝ(z)S(z). Based on the structure of the low-pass is guaranteed based on the assumption that a linearized
filter, the following relationship is obtained from Eq. (6): model G(z) is known. But if the system cannot be modeled
1 perfectly, then the overall modeling error is expressed as
1 − kejθ(ω) T (ejωTs ) < 1 ≤ , (9)
|Q(ejωTs )| Ga (z) = G(z)(1 + γ(z)). (17)
for ω ∈ (0, π/Ts ). Replacing the complimentary sensitive The stability conditions of the ILC are given by
function with T (ejωTs ) = A(ω)ejθT (ω) , where A(ω) >
0 and θT (ω) are the magnitude and phase of T (ejωTs ), 0 < |1 − (1 + γ(z)k)| < 1, (18)
respectively, Eq. (9) becomes π π
− <θγ (ω) < . (19)
2 2
1 − kA(ω)ej[θT (ω)+θ(ω)] < 1. (10) Combining Eq. (17) with Eqs. (11) and (12), the stability
Hence, condition for the proposed RC method is given by,

2 cos[θT (ω) + θ(ω)] 2 cos[θT (ω) + θ(ω) + θγ (ω)]

0<k< and (11) 0<k< , (20)
A(ω) A(ω)(1 + ||γ||)
π π
−π/2 < [θT (ω) + θ(ω)] < π/2. (12) − < [θT (ω) + θ(ω) + θγ (ω)] < . (21)
2 2
Equations (11) and (12) provide the design parameters for Both the magnitude condition in Eq. (18) and Eq. (20) can
the RC to ensure stability of the closed-loop system. be satisfied by choosing a conservative gain k. However, the

5103
Fig. 2. Simulation result comparing RC and FFT-based ILC: (a) FFT-based ILC for circle trajectory tracking, (b) RC for circle trajectory tracking,
(c) FFT-based ILC for square trajectory tracking, and (d) RC for square trajectory tracking.

proposed RC has the ability to compensate for unmodeled investigated: circular, square, and figure-eight trajectories.
phase lag θ(ω), while the ILC method restricts the phase Below, the experimental system is described, followed by
of the unmodeled dynamics. Thus, the proposed RC can modeling the dynamics and using the full nonlinear model
be viewed as being more robust for tracking of aggressive to simulate and fine-tune the control system prior to imple-
trajectories. mentation. It is pointed out that experiments in the motion-
capture environment were performed as a means to illustrate
B. Simulation Study
To study the effects of ILC compared to RC, simulations the application of the approach and to demonstrate feasibility.
were performed using the model presented in Section V. Outdoor experiments can be conducted using other means
The quadcopter is commanded to take off and hover at a of localization such as GPS and other on-board sensing,
desired height, then the FFT-based ILC or RC take control for but in this work flight tests were done in an indoor flight
trajectory tracking shortly after hover is achieved. A circle volume with motion capture hardware for convenience and
and a square trajectory are used in the simulation study. to illustrate the effectiveness of the approach.
The gains for the two type of controllers for the two A. Experimental Setup
desired trajectories were fixed. The performance of ILC The experimental micro aerial robot is shown in Fig. 3.
and RC for tracking the circular trajectory is roughly the The custom-designed platform consists of a 1.6-mm thick
same, where the RMS error was reduced 8% at steady-state. carbon fiber frame, four 6400 kv 3 g micro brushless motors
Figure 2(a) and (b) compare the tracking results between (OverSky MR-03-H, model CR14) driven by 3 A electronic
the two controllers, where it is shown that the RC results speed controllers (Oversky MX-3A), and 75-mm propellers.
in less oscillatory behavior at steady-state. For the square A Crazyflie platform (without motors) served as the flight
trajectory, the ILC showed increased oscillation compared to controller and it was mounted to the carbon fiber frame with
the RC, and the ILC’s response over time slowly increased, the micro brushless motors and ESCs as shown in Fig. 3. The
showing signs of being unstable. In fact, for the FFT-based Crazyflie platform was used because of available hardware
ILC, instability results when the conditions in Eq. (18) and and firmware designed to work with the Robot Operating
Eq. (19) are not satisfied. It is noted that comparing Fig. 2(c) System (ROS). Three small 3-mm diameter infrared (IR)
and (d), the RC was not affected by the change in the desired markers were attached to the frame as shown for localization
trajectory, from circle trajectory to the square trajectory, in a motion-capture environment.
whereas the ILC’s performance degraded in this case. The complete experimental environment is shown in
V. C USTOM - DESIGNED M ICRO A ERIAL ROBOT Fig. 4, where flight experiments are conducted in a netted
Experiments were performed to evaluate the RC control flight volume (approximately 2.4 m × 2.4 m × 2.4 m) in a
approach for tracking periodic trajectories on a custom- indoor laboratory setting. Four Optitrack Prime 17W cameras
designed micro aerial robot. Three types of trajectories were are used for motion capture. A computer runs the Motive

5104
 B. System Modeling
The dynamics of the micro quadcopter was obtained
using first principles. The full nonlinear model was used to
simulate the response of the control system, as well as to
tune the parameters of the RC and three PID controllers as
shown in Fig. 1(a). Figure 5 shows the coordinate system
for the inertial reference frame and the body frame on the
quadcopter, where Fz represents the total thrust with respect
to the body frame and the moments generated by the motors
are denoted by Mx , My , and Mz .
From first principles, the equations of motions for the
linear and rotational motions expressed in the body frame
are X
mv̇b + ωb × mvb = Fb , (22)
X
J ω̇b + ωb × Jωb = Mb . (23)

Then, in the inertial frame the equations of motion are

Fz
ẍ = (sin φ sin ψ + cos φ sin θ cos ψ) , (24)
m
Fz
ÿ = (− sin φ cos ψ + cos φ sin θ sin ψ) , (25)
m
Fz
z̈ = g + cos φ cos θ , (26)
 m
Iy − Iz Ir Mx
ṗ = qr − qΩr + , (27)
Ix Ix Ix
 
Fig. 3. The (a) top and (b) bottom views of the experimental micro Iz − Ix Ir My
quadcopter platform. The platform weighs 55 g and the center-to-center
q̇ = pr + pΩr + , (28)
Iy Iy Iy
distance between the motors is 89 mm as shown. A Crazyflie 2.0 flight  
controller is mounted to the underside of the platform. Customized firmware Ix − Iy Mz
allows the flight controller to drive the ESCs and micro brushless motors.
ṙ = pq + , (29)
Iz Iz
Three infrared (IR) markers are attached to the carbon fiber frame as shown
for localization in the motion capture environment. where the total angular velocity of the motors are Ωr =
−Ω1 + Ω2 − Ω3 + Ω4 ; m is the mass; g is gravity; p, q, and
optical motion capture software and sends the position and r are the roll, pitch, and yaw rates, respectively; and φ, θ,
attitude information of the quadcopter to a computer that and ψ are the roll, pitch, and yaw angles, respectively.
runs ROS. The top surfaces of the quadcopter’s propellers The velocity relationships between the inertial and body
were lightly sanded to minimize IR glare that would affect frames are given by
the performance of the motion capture system. The control        
system is implemented in ROS at a loop rate of 50 Hz and ẋ u φ̇ p
control signals are sent wirelessly to the quadcopter.  ẏ  = RbI  v  and  θ̇  = TbI  q  , (30)
ż I w b ψ̇ r b
I

where RbI is the standard rotation matrix from the body to

the inertial frame,
 
1 sin φ tan θ cos φ tan θ
TbI =  0 cos φ − sin φ  . (31)
0 sin φ/ cos θ cos φ/ cos θ
Finally, the relationship between the motor angular velocities
Ωi and the resulting thrust and moments are given by
    2 
Fz −cf −cf −cf −cf Ω1
 Mx   −dcf −dcf dc f dcf
  Ω22 
 My  =  dcf
     , (32)
−dcf −dcf dcf   Ω23 
Fig. 4. The experimental environment, consisting of a netted flight volume Mz cm −cm cm −cm Ω24
(approximately 2.4 m × 2.4 m × 2.4 m) with four Opti-track Prime 17W
cameras operating at 120 Hz for motion capture. A computer runs the where cf and cm are constants determined empirically and
Motive optical motion capture software and sends the position and attitude d is the perpendicular distance from the center of each rotor
information of the quadcopter to a computer that runs ROS. The control
system is implemented in ROS at a loop rate of 120 Hz and control signals to the body frame axis.
are sent wirelessly to the quadcopter at a rate of 50 Hz.
5105
 800
n=40

600 n=35

n=30
400
n=25

200 n=20

2
n=15
0
n=10

T
-200 n=5

n=0
-400
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Frequency (Hz)
Fig. 5. Coordinate system for the inertial reference frame and the body
frame on the quadcopter, where Fz represents the total thrust with respect Fig. 6. Closed-loop system phase compensation for improved tracking
to the body frame and the moments generated by the motors are denoted performance. Phase lead of n = 25 provided largest control bandwidth.
by Mx , My and Mz .

100 Pure RC

Magnitude (dB)
Herein, the aerodynamic drag on the quadcopter as well
as the ground and ceiling effect are neglected. Experiments 50 With Q(z) With Q(z) + z -N+m

were performed to determine the parameters of the model and 0

they are: Ix = 7.85 × 10−5 N-m-s2 , Iy = 7.88 × 10−5 N- -50

m-s2 , Iz = 1.05 × 10−4 N-m-s2 , m = 4.42 × 10−2 kg, 1 1.5 2 2.5

Frequency (rad/s)
3 3.5 4 4.5 5 5.5 6

Ir = 1.01 × 10−6 N-m-s2 , cf = 2.02 × 10−11 N-s2 , cm =

Fig. 7. Frequency response of RC for tuning phase lead m.
1.01 × 10−12 N-s2 and d = 0.12 m.
VI. E XPERIMENTAL R ESULTS AND D ISCUSSION Next, after the phase compensator was designed, the low-
pass filter Q(z) cut off frequency was set to 2 Hz. Using
A. Inner Loop Controller Tuning: Attitude Stabilization the designed Q(z), the final step was to determine the
Using the model presented above, the two inner control
optimum value for m by adjust this value such that the
loops involving PID controllers C1 (z) and C2 (z) were
frequency response of the RC produced the largest gain at
tuned in simulation and then compared experimentally. Each
the harmonics of the reference trajectory. Figure 7 show the
controller has the form
    frequency response of RC for tuning phase lead m, where
z z−1 m = 5 yielded the maximum RC gain at the harmonics of
Ci (z) = Kp,i + Ki,i + Kd,i , (33)
z−1 z the reference trajectory.
where the Ziegler-Nichols method [35] was used to tune C. Tracking Results
the parameters of each controller. The PID controller C1 (z) Using the RC parameters determined from simulation and
was implemented at a loop rate of 250 Hz and for C2 (z) at experiments as described above, the RC and PID controllers
500 Hz. were implemented on the micro aerial robot for tracking
B. Outer-loop PID and RC Design: Trajectory Tracking of three periodic reference trajectories: circular, square, and
The outer-loop PID controller C0 (z) was designed to track figure-eight trajectories.
a periodic reference trajectory r with period 0.25 s. The For the circular trajectory, the period for completing one
chosen periodic reference trajectory was a circle of radius pass was 4 seconds (0.25 Hz). The actual circular path
0.6 m, at a height of approximately 0.6 m. The PID gains traversed by the robot are shown in Fig. 8, for both the
were tuned using the similar procedures as for C1 (z) and PID and the PID with RC. The measured tracking error
C2 (z). The optimum gains for the x and y axes are: kp = 30, as a function of time are shown in Fig. 9(a), over a time
ki = 0, and kd = 15, for the z axis: kp = 9000, ki = 0, and period of 65 seconds. It can q be seen that the magnitude of
kd = 2000, and for the yaw: kp = 3.5, ki = 2, and kd = 0. the tracking error emax = e2x + e2y for the PID controller
For the repetitive controller, the outer-loop sampling fre- is significantly larger than that for the PID with RC at steady
quency was set at 50 Hz. This value was limited by the rate state. In fact, as time progresses, the tracking error for the
of the wireless link operated to send command signals to PID with RC system decreases. For the PID controller, the
control the robot. For the RC, N = 200 points. The lead magnitude of maximum error is 0.41 cm, where as for the
parameter m, low-pass filter Q(z), and phase compensator PID with RC is 0.09 cm, a 78% reduction. The reason for the
P (z) in Fig. 1(a) were determined both in simulation and error magnitude decrease in PID is because the quadcopter
experiments. First, the phase-lead compensator P (z) = z n start with zero velocity and the desired trajectory is not
was determined by experimentally measuring the phase lag smooth at start so it will experience a larger tracking error
θ in Eq. (12) of the closed-loop system without the RC in until it falls into steady state. Figure 9(b) and (c) show
place. Figure 6 shows the experimentally determined closed- the tracking response at steady for the x and y directions,
loop phase response versus frequency for various values of respectively.
phase lead n. It can be seen that the value of n = 25 provided The reason the PID tracking error magnitude in both xI
the largest bandwidth, at 2 Hz, where the phase response and yI directions reach zero is because the tracking error in
crosses the ±90 deg threshold. the circular trajectory is caused by both the phase lag and

5106
 Desired trajectory
Desired trajectory (a)
(a) 80

z (cm)
60 1

z (cm)
0.5
40
50 0
Actual response 40 60
20 Actual response 40
y (cm) 0 20
0 0
40 60 y (cm) -20 x (cm)
-50 0 20 -20
-40 -20 -40
-60 x (cm ) -40

(b) 80 Desired trajectory (b) Desired trajectory

z (cm)

60 1

z (cm)
40 0.5
50 Actual response 0
40
y (cm) 0 20 Actual response 60
40
40 60 0 20
-50 0 20 y (cm) -20 0
-40 -20 -20 x (cm)
-60 x (cm) -40 -40

Fig. 8. Experimental tracking results at steady-state for circular trajectory: Fig. 10. Experimental tracking results at steady-state for figure-eight
(a) PID and (b) PID with RC. trajectory: (a) PID performance and (b) PID with RC performance.

(a) overshoot. In xI or yI axis, the quadcopter will catch up

RC magnitude error
40 PID magnitude error
with the desired trajectory and then overshoot and lag again.
30 Next, a more aggressive figure-eight desired trajectory was
|e | (cm)

20 imposed on the control system and the results are shown

10 in Fig. 10. In this case, for the PID controller alone, the
0 magnitude of the tracking error is 0.31 cm and for the
0 10 20 30 40 50 60 PID with RC, the outcomes is 0.16 cm, resulting in a 48%
Time (s)
(b) reduction.
40 Finally, the controllers were used to track an aggressive
20 square-shaped desired trajectory. The results are shown in
x (cm)

0 Fig. 11. For the PID controller alone, the magnitude of the
-20 Desired tracking error is 0.25 cm and for the PID with RC the
Repetitive control
-40 PID control outcome is 0.13 cm, resulting in a 48% reduction.
48 49 50 51 52 53 54 Based on the experimental results, it is evident that by
(c) Time (s)
adding RC, the aerial robot was able to track the periodic
30 RC magnitude error
PID magnitude error reference trajectory with higher precision. Compared to PID
| ex | (cm)

20 only, the RC causes the tracking error to decrease as the

number of repetitions increased. These results suggest the
10
benefit of adding RC to a standard PID feedback system to
0 improve performance. Thus, when a robot is required to track
48 49 50 51 52 53 54
(d)
Time (s) a periodic trajectory, the plug-in RC can be switched on to
40 provide enhanced performance.
y (cm)

0 VII. C ONCLUSIONS
Desired This paper presented the design and implementation of a
Repetitive control
-40 PID control plug-in repetitive control system to track periodic reference
48 49 50 51 52 53 54 trajectories for a micro aerial robot. A discrete-time repetitive
Time (s)
(e) controller which factored in the phase lag of the closed-loop
RC magnitude error
30 PID magnitude error system was used to track three periodic reference trajectories.
| ey | (cm)

20 The RC design was described in detail and experimental

results showed that the RC reduced the overall tracking error
10
by over 58% on average for three test cases compared to the
0
48 49 50 51 52 53 54 case of only using PID control. Simulations that compare to
Time (s)
no-reset FFT-based ILC showed that the performance of RC
Fig. 9. Experimental tracking error for circular trajectory: (a) comparison is less sensitive to changes in the reference trajectory.
of magnitude of tracking error for PID and RC augmented PID, (b) tracking
response for x-axis at steady state, (c) magnitude of tracking error in x- VIII. ACKNOWLEDGMENTS
axis, (d) tracking response for y-axis at steady state and (e) magnitude of This material is based upon work supported, in part, by
tracking error in y-axis.
the National Science Foundation, Partnership for Innovation

5107
 Desired trajectory [11] V. Kumar and N. Michael, “Opportunities and challenges with au-
(a)
tonomous micro aerial vehicles,” Int. J. of Robotics Research, vol. 31,
no. 11, pp. 1279 – 1291, 2012.
[12] T. Flanzer, “Robust trajectory optimization and control of a dynamic
1
soaring UAV,” Thesis, Stanford University, 2012.
z (cm)
[13] U. Aridogan, Y. Shan, and K. K. Leang, “Design and analysis
0.5 of discrete-time repetitive control for scanning probe microscopes,”
ASME J. Dyn. Syst. Meas. and Cont., vol. 131, no. 6, p. 061103 (12
0 40 pages), 2009.
40
20
Actual response 20 [14] C. J. Li and S. Y. Li, “To improve workpiece roundness in precision
0 0 diamond turning by in situ measurement and repetitive control,”
y (cm) x (cm) Mechatronics, vol. 6, no. 5, pp. 523 – 535, 1996.
-20 -20
-40 -40 [15] M. Hehn and R. D’Andrea, “An iterative learning scheme for high
performance, periodic quadrocopter trajectories,” in European Control
(b) Desired trajectory Conference, 2013, Conference Proceedings, pp. 1799 – 1804.
[16] ——, “A frequency domain iterative feed-forward learning scheme
for high-performance periodic quadrocopter maneuvers,” in IEEE/RSJ
1 International Conference on Intelligent Robots and Systems, 2013,
z (cm)

Conference Proceedings, pp. 2445 – 2451.

0.5 [17] ——, “A frequency domain iterative learning algorithm for
high-performance, periodic quadrocopter maneuvers,” Mechatronics,
0 40 vol. 24, no. 8, pp. 954 – 965, 2014.
40 Actual response [18] Y. Wang, F. Gao, and F. J. Doyle, “Survey on iterative learning control,
20
20 repetitive control, and run-to-run control,” Journal of Process Control,
0 0 vol. 19, no. 10, pp. 1589–1600, 2009.
y (cm) x (cm)
-20 -20 [19] D. A. Bristow, M. Tharayil, and A. G. Alleyne, “A survey of iterative
-40 -40 learning control,” IEEE Cont. Syst. Mag., vol. 26, no. 3, pp. 96 – 114,
2006.
Fig. 11. Experimental tracking results at steady-state for square trajectory: [20] L. Sison and E. Chong, “No-reset iterative learning control,” in 35th
(a) PID performance and (b) PID with RC performance. IEEE Conference on Decision and Control, no. Ilc, 1996.
[21] A. Zulu and S. John, “A review of control algorithms for autonomous
quadrotors,” Open Journal of Applied Sciences, vol. 4, pp. 547 – 556,
Program, Grant No. 1430328. Any opinions, findings, and 2014.
conclusions or recommendations expressed in this material [22] S. Grzonka, G. Grisetti, and W. Burgard, “A fully autonomous indoor
quadrotor,” IEEE Trans. Robotics, vol. 28, no. 1, pp. 90 – 100, 2012.
are those of the author(s) and do not necessarily reflect the [23] B. Leong, T. Meng, S. M. Low, and M. P. Ooi, “Low-cost
views of the National Science Foundation. microcontroller-based hover control design of a quadcopter,” Procedia
Engineering, vol. 41, pp. 458 – 464, 2012.
R EFERENCES [24] Y. M. Al-Younes, M. A. Al-Jarrah, and A. A. Jhemi, “Linear vs.
nonlinear control techniques for a quadrotor vehicle,” in Proceeding of
[1] R. Mahony, V. Kumar, and P. Corke, “Multirotor aerial vehicles: the 7th International Symposium on Mechatronics and its Applications
Modeling, estimation, and control of quadrotor,” IEEE Robotics & (ISMA10), 2010, Conference Proceedings.
Automation Magazine, vol. 19, no. 3, pp. 20 – 32, 2012. [25] A. Bemporad, C. A. Pascucci, and C. Rocchi, “Hierarchical and hybrid
[2] F. Nex and F. Remondino, “UAV for 3D mapping applications: a model predictive control of quadcopter air vehicles,” Analysis and
review,” Applied Geomatics, vol. 6, no. 1, pp. 1 – 15, 2013. Design of Hybrid Systems, vol. 3, no. 2, 2009.
[3] S. Waharte and N. Trigoni, “Supporting search and rescue operations [26] M. Bangura and R. Mahony, “Nonlinear dynamic modeling for high
with uavs,” in International Conference on Emerging Security Tech- performance control of a quadrotor,” in Proceedings of Australasian
nologies (EST). IEEE, 2010, pp. 142–147. Conference on Robotics and Automation, 2012, Conference Proceed-
[4] G. A. Landis, “Robots and humans: synergy in planetary exploration,” ings.
Acta Astronautica, vol. 55, no. 12, pp. 985 – 990, 2004. [27] R. W. Beard, “Quadrotor dynamics and control,” Brigham Young
[5] P. S. Lin, L. Hagen, K. Valavanis, and H. Zhou, “Vision of unmanned University,” Report, 2008.
aerial vehicle (UAV) based traffic management for incidents and emer- [28] S. Arimoto, S. Kawamura, and F. Miyazaki, “Bettering operation of
gencies,” in 12th World Congress on Intelligent Transport Systems, robots by learning,” Journal of Robotic Systems, vol. 1, no. 2, pp.
2005, Conference Proceedings. 123–140, 1984.
[6] D. Hausamann, W. Zirnig, G. Schreier, and P. Strobl, “Monitoring [29] K. L. Moore, M. Dahleh, and S. P. Bhattacharyya, “Iterative learning
of gas pipelines a civil UAV application,” Aircraft Engineering and control: a survey and new results,” Journal of Robotic Systems, vol. 9,
Aerospace Technology, vol. 77, no. 5, pp. 352 – 360, 2005. no. 5, pp. 563–594, 1992.
[7] H. S. Trammell, A. R. Perry, S. Kumar, P. V. Czipott, B. R. Whitecot- [30] B. A. Francis and W. M. Wonham, “The internal model principle of
ton, T. J. McManus, and D. O. Walsh, “Using unmanned aerial vehicle- control theory,” Automatica, vol. 12, no. 5, pp. 457 – 465, 1976.
borne magnetic sensors to detect and locate improvised explosive [31] T. Inoue, M. Nakano, and S. Iwai, “High accuracy control of a proton
devices and unexploded ordnance,” in Proc. SPIE Sensors, and Com- synchrotron magnet power supply,” in Proc. 8th World Congr. IFAC,
mand, Control, Communications, and Intelligence (C3I) Technologies vol. 20, 1982, Conference Proceedings, pp. 216 – 221.
for Homeland Security and Homeland Defense IV, vol. 5778, 2005, [32] S. Hara, Y. Yamamoto, T. Omata, and M. Nakano, “Repetitive control
Conference Proceedings. system: a new type servo system for periodic exogenous signals,” IEEE
[8] H. Yoshimoto, K. Jo, and K. Hori, “Toward entertainment blimps for Trans. Autom. Cont., vol. 33, no. 7, pp. 659 – 668, 1988.
everyone by everyone,” in Proceedings of the seventh ACM conference [33] K. K. Chew and M. Tomizuka, “Digital control of repetitive errors in
on creativity and cognition, 2009, Conference Proceedings, pp. 445 – disk drive systems,” IEEE Cont. Syst. Mag., vol. 10, no. 1, pp. 16 –
446. 20, 1990.
[9] M. A. Goodrich, B. S. Morse, D. Gerhardt, J. L. Cooper, M. Quigley, [34] K. Zhou and J. C. Doyle, Essentials of robust control. Prentice-Hall,
J. A. Adams, and C. Humphrey, “Supporting wilderness search and Inc., 1998.
rescue using a cameraequipped mini UAV,” Journal of Field Robotics, [35] G. F. Franklin, J. D. Powell, and A. Emami-Naeini, Feedback control
vol. 25, no. 1-2, pp. 89 – 110, 2008. of dynamic systems, 5th ed. Addison-Wesley Reading, 2006.
[10] S. Adams and C. Friedland, “A survey of unmanned aerial vehicle
(UAV) usage for imagery collection in disaster research and man-
agement,” in The 9th International Workshop on Remote Sensing for
Disaster Response, 2011, Conference Proceedings.

5108
 An Overview of
Systems-Theoretic
Guarantees in Data-Driven
Model Predictive Control
Julian Berberich and Frank Allgöwer
arXiv:2406.04130v1 [eess.SY] 6 Jun 2024

Institute for Systems Theory and Automatic Control, University of Stuttgart,

70569 Stuttgart, Germany; email: julian.berberich@ist.uni-stuttgart.de,
frank.allgower@ist.uni-stuttgart.de

Annual Review of Control, Robotics, and Keywords

Autonomous Systems yyy. xxx:1–25
Data-driven control, model predictive control, robustness, stability,
https://doi.org/10.1146/((please add
article doi))
adaptive control

Abstract

The development of control methods based on data has seen a surge

of interest in recent years. When applying data-driven controllers in
real-world applications, providing theoretical guarantees for the closed-
loop system is of crucial importance to ensure reliable operation. In
this review, we provide an overview of data-driven model predictive
control (MPC) methods for controlling unknown systems with guaran-
tees on systems-theoretic properties such as stability, robustness, and
constraint satisfaction. The considered approaches rely on the Funda-
mental Lemma from behavioral theory in order to predict input-output
trajectories directly from data. We cover various setups, ranging from
linear systems and noise-free data to more realistic formulations with
noise and nonlinearities, and we provide an overview of different tech-
niques to ensure guarantees for the closed-loop system. Moreover, we
discuss avenues for future research that may further improve the theo-
retical understanding and practical applicability of data-driven MPC.

1
 1. INTRODUCTION
Determining models that are both accurate and tractable, as commonly required for con-
troller design, is challenging. On the other hand, the increasing availability of data provides
unprecedented opportunities for enhancing controllers with additional information. System
identification provides one possibility for extracting information from data by estimating a
model of the underlying dynamical system (1), which can then be employed for control. Ob-
taining theoretical guarantees for this indirect data-driven control strategy requires rigorous
bounds on the estimation error, whose derivation is an active field of research (2).
As an alternative to indirect approaches, designing controllers directly from data with-
out intermediate system identification has received increasing attention in the recent litera-
ture (3). The Fundamental Lemma by Willems et al. (4) provides a unifying framework for
direct data-driven control. It states that one persistently exciting trajectory of an unknown
linear time-invariant (LTI) system can be used to parametrize all other trajectories. This
allows to address various analysis and controller design problems, that are traditionally
solved using model knowledge, based purely on data, see (5) for a recent survey. One of
the most promising applications of the Fundamental Lemma is the design of data-driven
model predictive control (MPC) schemes. MPC is a powerful modern control technique
which can handle constraints, performance criteria, and nonlinear dynamics (6). It relies
on a receding-horizon principle: At each time step, an open-loop optimal control problem
is solved, the first component of the optimal input sequence is applied to the plant, and
the process is repeated at the next time step using a new measurement. While classi-
cal MPC schemes assume precise model knowledge, they can be enhanced via data using
learning-based (7) and adaptive (8, 9, 10) approaches.
This review provides an overview of recent research on direct data-driven MPC based on
the Fundamental Lemma with a focus on systems-theoretic guarantees for the closed-loop
system. It was first shown in (11, 12) that the Fundamental Lemma can be used to set
up data-driven MPC schemes, which compute control inputs directly from data without in-
termediate system identification. Since then, data-driven MPC has been successfully used
in a large number of applications, including a quadcopter (13), a four-tank system (14),
synchronous motor drives (15), a quadrupedal robot (16), a soft robot (17), green house
automation (18), and traffic control (19, 20, 21, 22). Further, various successful applications
have been demonstrated in the domain of energy systems, including grid-connected power
converters (23, 24), power system oscillation damping (25), building control (26, 27, 28, 29),
battery charging (30), a combined-cycle power plant (31), energy networks (32, 33, 34), and
a fuel cell system (35). These remarkable empirical demonstrations have motivated nu-
merous theoretical contributions with the goal of understanding and improving data-driven
MPC. When using data-driven MPC in real-world applications, especially for complex and
safety-critical systems, it is desirable to provide a priori guarantees on reliable closed-loop
operation. Mathematically, the goal is to ensure systems-theoretic properties such as sta-
bility, robustness, and constraint satisfaction for the controlled system.
In this review, we provide an overview of recent progress on systems-theoretic guaran-
tees in data-driven MPC. We discuss appropriate modifications that allow to prove rigorous
guarantees for the closed-loop system, covering different setups including linear and non-
linear systems as well as noise-free and noisy data. While we mainly focus on theoretical
aspects of data-driven MPC, in particular closed-loop guarantees, we refer to (5, 36, 37) for
further details, e.g., on implementations.
This review is structured as follows. In Section 2, we introduce preliminaries on the

2 Berberich & Allgöwer

Fundamental Lemma and on model-based MPC. Next, in Section 3, we provide an overview
of data-driven MPC for linear systems, discussing stability guarantees with noise-free data,
robustifications and associated guarantees for noisy data, as well as more advanced MPC
formulations. In Section 4, we turn our attention to nonlinear data-driven MPC approaches.
These are either based on global linearity, exploiting the structure of specific system classes
and knowledge of basis functions, or on local linearity, pursuing an adaptive formulation
with online data updates. Finally, Section 5 concludes the review with a discussion and
avenues for future research.
Notation: We denote the set of nonnegative integers by I≥0 and integers in the interval
−1
[a, b] by I[a,b] . For a sequence {uk }N
k=0 , we define the stacked column vector u[k1 ,k2 ] :=
h i⊤
u⊤ ⊤ ⊤
k1 uk1 +1 . . . uk2 and we abbreviate u := u[0,N −1] for the vector containing the full
sequence. Further, we define the Hankel matrix
 
u0 u1 . . . uN −L
 u
 1 u2 . . . uN −L+1 

HL (u) :=  . .  ∈ RmL×(N −L+1) .
 . . .
.. .. .. 
 . 
uL−1 uL . . . uN −1

For
√ a vector x, we denote the p-norm by ∥x∥p and we define the weighted norm ∥x∥P =
x⊤ Px for some matrix P = P⊤ .

2. Preliminaries
In this section, we introduce the Fundamental Lemma (Section 2.1) as well as model-based
MPC (Section 2.2).

2.1. The Fundamental Lemma

In the following, we consider a discrete-time linear time-invariant (LTI) system

xk+1 = Axk + Buk , yk = Cxk + Duk 1.

with state xk ∈ Rn , input uk ∈ Rm , output yk ∈ Rp , and time k ∈ I≥0 . We make the

standing assumption that (A, B) is controllable and (A, C) is observable. The matrices
N −1
A, B, C, D, are unknown but, instead, an input-output data trajectory {udk , ykd }k=0 is
available. We require the corresponding input signal to be sufficiently rich in the following
sense (4).
−1
Definition 2.1. The signal {uk }N k=0 with uk ∈ R
m
is persistently exciting (PE) of order
L if HL (u) has full row rank, i.e., rank(HL (u)) = mL.

The following result from (4), commonly referred to as the Fundamental Lemma, forms
the theoretical basis of data-driven MPC.
−1
Theorem 2.1. Suppose {udk }N L−1
k=0 is PE of order L + n. Then, {ūk , ȳk }k=0 is a trajectory
N −L+1
of the system in Equation 1 if and only if there exists α ∈ R such that
" # " #
HL (ud ) ū
α= . 2.
HL (yd ) ȳ

www.annualreviews.org • 3
 The Fundamental Lemma (Theorem 2.1) provides a parametrization of all possible
input-output trajectories of the system in Equation 1 based on only one input-output tra-
−1
jectory {udk , ykd }N
k=0 . More precisely, the image of the Hankel matrices in Equation 2 is
equal to the set of all system trajectories (recall the notation ū = ū[0,L−1] and ȳ = ȳ[0,L−1] ).
While the result was originally formulated and proven in behavioral systems theory (4), al-
ternative proofs were provided more recently in the classical state-space framework (38, 39).
Further, the result has found increasing usage in the recent literature for developing direct
data-driven analysis and control methods. In the remainder of the review, we focus on data-
driven MPC based on the Fundamental Lemma with closed-loop guarantees. Beyond MPC,
the result can be used, e.g., for data-driven simulation (40), dissipativity analysis (41, 42),
and state-feedback design (43, 44, 45), see (5, 46, 47, 48) for recent overview articles.

2.2. Model predictive control

Suppose we want to steer the system 1 to a steady-state xs while satisfying constraints on
the input uk ∈ U and the state xk ∈ X for all k ∈ I≥0 with sets U ⊆ Rm , X ⊆ Rn . In MPC,
this is achieved by solving, at time t, the following optimal control problem.
L−1
X
min ∥x̄k (t) − xs ∥2Qx + ∥ūk (t) − us ∥2R 3a.
ū(t),x̄(t)
k=0

s.t. x̄k+1 (t) = Ax̄k (t) + Būk (t), k ∈ I≥0 , x̄0 (t) = xt , 3b.
ūk (t) ∈ U, x̄k (t) ∈ X, k ∈ I[0,L−1] . 3c.
Here, ū(t) and x̄(t) denote the predicted input and state trajectory at time t, e.g., x̄k (t) is
the k-th step of the state trajectory predicted at time t. According to the constraint 3b,
{ūk (t), x̄k (t)}L−1
k=0 is a trajectory of the system 1 which is initialized at the current state xt .
In the cost 3a, the distance of this trajectory to the setpoint is penalized over the prediction
horizon L with positive definite cost matrices Qx , R. Further, Equation 3c enforces the
constraints on the predicted trajectory. We denote the optimal solution of the optimization
problem 3 by ū∗ (t), x̄∗ (t). It can be used to synthesize a feedback controller in a receding-
horizon fashion as commonly done in MPC: At time t, the first component ū∗0 (t) of the
optimal input trajectory is applied to the system and a new optimal input is computed at
the next time step based on a new state measurement, see Algorithm 2.1.
When applying Algorithm 2.1 without further modifications, the closed-loop system
need not be stable, see (49) for an experimental example. Guarantees on closed-loop sta-
bility and constraint satisfaction can be ensured either via a sufficiently long prediction
horizon L (50) or by adding terminal ingredients to the optimization problem 3, e.g., a
terminal cost and a terminal region constraint on the final state x̄L (t) (6).
Both the design and the implementation of standard MPC schemes as in Algorithm 2.1
require model knowledge. For example, for solving the optimization problem 3, the matrices
(A, B, C, D) need to be available. In this review, we cover an alternative MPC approach
which can control unknown systems only from input-output data while maintaining systems-
theoretic guarantees.

3. DATA-DRIVEN MPC FOR LINEAR SYSTEMS

In this section, we review data-driven MPC schemes for LTI systems with a focus on systems-
theoretic guarantees for the closed loop. We consider data-driven MPC for stabilization

4 Berberich & Allgöwer

Algorithm 2.1. Model predictive control
Offline: Choose prediction horizon L, positive definite cost matrices Qx , R, constraint sets
U, X, setpoint (us , xs ).
Online:

1) At time t, solve the optimization problem 3.

2) Apply the first optimal input component ut = ū∗0 (t).
3) Set t = t + 1 and go back to 1).

tasks in the ideal case of noise-free data (Section 3.1) and in the more realistic scenario of
noisy data (Section 3.2). In Section 3.3, we discuss more advanced MPC approaches, e.g.,
addressing control objectives beyond stabilization.

3.1. Data-driven MPC with noise-free data

The Fundamental Lemma can be used to set up data-driven MPC schemes by replacing
the commonly used state-space model (compare Section 2.2) via data-dependent Hankel
matrices. Let us consider the control goal of tracking an input-output setpoint (us , ys ) while
satisfying constraints uk ∈ U and yk ∈ Y for all k ∈ I≥0 with sets U ⊆ Rm , Y ⊆ Rp . The
−1
Fundamental Lemma allows to achieve these two objectives based only on data {udk , ykd }N k=0
by solving, at time t, the following optimal control problem
L−1
X
min ∥ūk (t) − us ∥2R + ∥ȳk (t) − ys ∥2Q 4a.
α(t),ū(t),ȳ(t)
k=0
" # " #
ū(t) HL+n (ud )
s.t. = α(t), 4b.
ȳ(t) HL+n (yd )
" # " #
ū[−n,−1] (t) u[t−n,t−1]
= , 4c.
ȳ[−n,−1] (t) y[t−n,t−1]
ūk (t) ∈ U, ȳk (t) ∈ Y, k ∈ I[0,L−1] . 4d.

The input-output trajectory predicted at time t is denoted by ū(t) = ū[−n,L−1] (t) ∈ UL+n ,
ȳ(t) = ȳ[−n,L−1] (t) ∈ YL+n . The constraint in Equation 4b is based on the Fundamental
Lemma and ensures that {ūk (t), ȳk (t)}L−1
k=−n is a trajectory of the LTI system. The cost
penalizes the difference of the predicted trajectory w.r.t. the setpoint (us , ys ) for user-
specified positive definite matrices Q, R. Note that the predicted trajectory is of length
L + n. In the constraint 4c, the first n components {ūk (t), ȳk (t)}−1
k=−n are used to initialize
the prediction based on the most recent n input-output measurements {uk , yk }t−1 k=t−n . This
initialization over n time steps is required when using an input-output prediction model to
ensure that the internal states of the prediction and the plant coincide, i.e., to implicitly
enforce a constraint analogous to the initial condition in Equation 3b. Instead of the
system order n, any upper bound can be used (to be precise, an upper bound on the lag
is sufficient). Further, Equation 4d contains the input-output constraints. Note that, for
convex polytopic constraint sets U, Y, the optimization problem 4 is a convex quadratic
program. We denote the optimal solution of the optimization problem 4 by α∗ (t), ū∗ (t),
ȳ∗ (t). It can be used to set up a data-driven MPC scheme in a receding-horizon fashion
analogous to the model-based case in Section 2.2, see Algorithm 3.1.

www.annualreviews.org • 5
 Algorithm 3.1. Data-driven MPC
Offline: Choose upper bound on system order n, prediction horizon L, positive definite
−1
cost matrices Q, R, constraint sets U, Y, setpoint (us , ys ), and generate data {udk , ykd }N
k=0 .
Online:

1) At time t, solve the optimization problem 4.

2) Apply the first optimal input component ut = ū∗0 (t).
3) Set t = t + 1 and go back to 1).

Algorithm 3.1 allows to control unknown LTI systems based only on input-output data
and without explicit model knowledge. The main difference to model-based MPC as in
Section 2.2 is the usage of data-dependent Hankel matrices for prediction, instead of a
state-space model. If the data are PE, the optimization problem 4 returns the same optimal
input as using a state-space model. Since the Fundamental Lemma directly parametrizes
input-output trajectories, Algorithm 3.1 inherently is an output-feedback MPC scheme. In
contrast, model-based output-feedback MPC schemes typically involve an observer whose
estimation error needs to be accounted for.
Analogous to the model-based case, data-driven MPC may lead to an unstable closed
loop and can even destabilize an open-loop stable system if L is too short (51). Closed-loop
stability of data-driven MPC can be ensured by modifying the optimization problem 4.
The arguably simplest approach is to add terminal equality constraints, i.e., to restrict the
optimal solution to be equal to the setpoint (us , ys ) over n steps at the end of the prediction
horizon. Mathematically, the following constraint is added to the optimization problem
" # " #
ūk (t) us
= , k ∈ I[L−n,L−1] . 5.
ȳk (t) ys

This implicitly ensures that the internal state corresponding to the input-output trajectory
ū(t), ȳ(t) is equal to the steady-state xs corresponding to (us , ys ) at time L. With this
modification, it can be shown under mild assumptions (e.g., PE) that, if the optimization
problem is feasible at initial time t = 0, then it is recursively feasible for all t ∈ I≥0
and Algorithm 3.1 exponentially stabilizes (us , ys ) in closed loop (51). While terminal
equality constraints are simple to implement and to study theoretically, they have significant
drawbacks in terms of robustness and the size of the region of attraction. In particular,
the latter only includes points from which the optimization problem is initially feasible, i.e.,
from which the setpoint can be reached within L time steps while satisfying the constraints.
As an alternative, one can design more general terminal ingredients, i.e., a terminal
cost function and a terminal region constraint for an extended state vector consisting of
sequential input-output values (52). To be precise, an alternative,
" non-minimal
# state of
u[t−n,t−1]
the LTI system in Equation 1 can be obtained as ξt = , compare (53, 54).
y[t−n,t−1]
Adapting data-driven output-feedback design methods from (55), it is possible to design
a terminal cost function Vf (ξ¯L (t)) = ∥ξ¯L (t)∥2P as well as a corresponding terminal region
constraint ξ¯L (t) ∈ Ξf for a sublevel set Ξf of Vf . Replacing the terminal equality constraint
in Equation 5 by these two components leads to closed-loop stability guarantees with a
significantly larger region of attraction (52). However, this approach faces an important
limitation: The design of the terminal ingredients from (52) is only applicable when the

6 Berberich & Allgöwer

state-space realization with state ξt is controllable, which holds, e.g., if the system is single-
output and the order n is known exactly (52, Lemma 13). Addressing this limitation based
on alternative data-driven output-feedback design approaches (56, 57) is an open research
problem. Beyond the above-described terminal ingredients, it can be shown that data-driven
MPC without any terminal ingredients also provides desirable closed-loop guarantees if the
prediction horizon is sufficiently long (58). Finally, recursive feasibility and stability can be
guaranteed based on dissipativity (59, 60)
All approaches discussed so far require that the setpoint (us , ys ) is a feasible equilib-
rium for the unknown system, i.e., there exists xs ∈ Rn such that xs = Axs + Bus , ys =
Cxs + Dus . This condition is non-trivial to verify, especially when no model is available,
and therefore it can be a critical limitation for practical applications of data-driven MPC. A
powerful way to resolve this problem is to optimize over the equilibrium (us , ys ) online and
penalize its distance to the actual target setpoint in the cost. Such an artificial equilibrium
has been proposed in model-based MPC (61) and it leads to strong theoretical results as
well as useful practical methods. In particular, data-driven MPC with artificial equilib-
rium guarantees exponential stability of the unknown optimal reachable equilibrium for a
given setpoint and cost function (62). This includes, e.g., the case that only an output set-
point without corresponding input component is available. Additional advantages include
theoretical guarantees for online setpoint changes, improved robustness, and a significantly
larger region of attraction compared to a standard MPC scheme with terminal equality con-
straints. Further, online optimization over an artificial equilibrium plays a crucial role when
using data-driven MPC to control unknown nonlinear systems, see Section 4 for details.
To summarize, if the data are noise-free, then the Fundamental Lemma allows to predict
future trajectories exactly such that, under appropriate assumptions (long prediction hori-
zon) or modifications (terminal ingredients), closed-loop stability under data-driven MPC
can be guaranteed. The main technical challenge in the analysis is that the cost function pe-
nalizes input-output values due to the employed input-output prediction model. This only
implies a positive semidefinite cost in the state, in contrast to model-based MPC which com-
monly assumes a positive definite cost, compare the optimization problem 3. Therefore, the
analysis of data-driven MPC requires additional detectability arguments (63).
In the sidebar Subspace Predictive Control, we introduce an alternative data-driven
MPC approach via an indirect perspective, i.e., using an identified model for MPC.

3.2. Data-driven MPC with noisy data

Real-world data are rarely noise-free. When data-driven MPC as discussed in Section 3.1 is
applied in the presence of noise, there is no guarantee that the closed-loop system behaves
desirably. In the following, we describe modifications of data-driven MPC that allow for
comparable theoretical guarantees also in the case of noisy data.
Suppose that both the offline data used for prediction via the Fundamental Lemma as
well as the online data used to specify initial conditions at time t are affected by output
measurement noise. To be precise, instead of the exact output values, we have access to

ỹkd = ykd + εdk , k ∈ I[0,N −1] , ỹt = yt + εt , t ∈ I≥0 ,

compare Figure 1. Further, the noise is bounded by some ε̄ > 0, i.e., ∥εdk ∥∞ ≤ ε̄ for
k ∈ I[0,N −1] and ∥εt ∥∞ ≤ ε̄ for t ∈ I≥0 . The following optimization problem provides the

www.annualreviews.org • 7
SUBSPACE PREDICTIVE CONTROL
Subspace predictive control (SPC) involves a two-step procedure consisting of 1) identification of a multi-
step predictor from data and 2) using this predictor as a model for MPC (64, 65). To introduce SPC, we
partition the Hankel matrices used for prediction in Equation 4b as follows
   
" # Hn (ud[0,N −L−1] ) Up
 H (ud
HL+n (ud )  L [n,N −1] )   Uf 
  
=  =:   . 6.
HL+n (yd ) d
Hn (y[0,N −L−1] ) Yp 
d
HL (y[n,N −1] ) Yf

Here, the data matrices Up , Yp correspond to the first n components of the input-output predictions, i.e.,
to the initial conditions in Equation 4c (hence, the index ’p’ for past). On the other hand, the data matrices
Uf , Yf correspond to future predictions (hence, the index ’f’). In SPC, the data are used to identify
a multi-step predictor which predicts future output values based on a future input trajectory and initial
conditions. To be precise, the predictor M is determined via least-squares estimation and takes the form
 †
Up
M = Yf  Uf  , where A† denotes the Moore-Penrose inverse of a matrix A. Based on this predictor, SPC
 
Yp
solves the following optimal control problem at time t.
 
L−1
X ū[−n,−1] (t)
min ∥ūk (t) − us ∥2R + ∥ȳk (t) − ys ∥2Q s.t. ȳ[0,L−1] (t) = M  ū[0,L−1] (t)  . 7.
 
ū(t),ȳ(t)
k=0 ȳ[−n,−1] (t)

Thus, in contrast to direct data-driven MPC, SPC involves an a priori estimation step and, hence, it is
an indirect data-driven MPC method. In Section 5, we discuss differences between direct and indirect
data-driven MPC in more detail.

basis for controlling unknown linear systems based on noisy input-output data.
L−1
X
min ∥ūk (t) − us ∥2R + ∥ȳk (t) − ys ∥2Q + λα ∥α(t)∥22 + λσ ∥σ(t)∥22 8a.
α(t),σ(t),ū(t),ȳ(t)
k=0
" # " #
ū(t) HL+n (ud )
s.t. = α(t), 8b.
ȳ(t) + σ(t) HL+n (ỹd )
" # " #
ū[−n,−1] (t) u[t−n,t−1]
= , 8c.
ȳ[−n,−1] (t) ỹ[t−n,t−1]
ūk (t) ∈ U, k ∈ I[0,L−1] . 8d.

In contrast to data-driven MPC for noise-free data as in Section 3.1, the exact output
values are replaced by their noisy versions. Further, the scheme contains an additional op-
timization variable σ(t) ∈ Rp(L+n) , the slack variable, which ensures that the constraint 8b
remains feasible despite the noise. In order to reduce the prediction error, the slack vari-
able is penalized in the cost with regularization parameter λσ > 0. Moreover, the norm

8 Berberich & Allgöwer

 ε
u y ỹ
System

Figure 1
Setup considered for robust data-driven MPC in Section 3.2. The figure displays a generic system
generating the data which are affected by output measurement noise ỹk = yk + εk .

of the variable α(t) is penalized with regularization parameter λα > 0 (see Section 3.2.2
for details). Finally, we note that the optimization problem 8 does not contain output
constraints, which would require a robust constraint tightening due to the noise, see (66)
for details. The optimization problem 8 can be used to set up a robust data-driven MPC
scheme in a standard receding-horizon fashion, see Algorithm 3.2.

Algorithm 3.2. Robust data-driven MPC

Offline: Choose upper bound on system order n, prediction horizon L, positive definite
cost matrices Q, R, regularization parameters λα , λσ > 0, constraint set U, setpoint (us , ys ),
−1
and generate data {udk , ỹkd }N
k=0 .
Online:

1) At time t, solve the optimization problem 8.

2) Apply the first optimal input component ut = ū∗0 (t).
3) Set t = t + 1 and go back to 1).

In the following, we discuss several key issues in robust data-driven MPC: closed-loop
guarantees for bounded noise (Section 3.2.1), the role of the regularization of α(t) (Sec-
tion 3.2.2), and closed-loop guarantees for stochastic noise (Section 3.2.3).

3.2.1. Closed-loop guarantees for bounded noise. Similar to Section 3.1, Algorithm 3.2
does not guarantee closed-loop stability unless the prediction horizon is sufficiently long
or terminal ingredients are added. It was shown in (51) that stability and robustness can
be ensured by adding terminal equality constraints as in Equation 5 to the optimization
problem 8. In this case, due to a technical controllability argument, proving theoretical
guarantees requires the application in an n-step fashion (67). This means that, in each
MPC iteration, the first n optimal input components ū∗[0,n−1] (t) are applied to the system
before repeating the optimization. With these modifications, robust data-driven MPC with
terminal equality constraints guarantees that the internal state xt converges exponentially
to a neighborhood of the steady-state xs corresponding to (us , ys ), where the size of the
neighborhood depends on the noise level. Mathematically, there exist 0 < c < 1 as well as
a continuous and strictly increasing function β with β(0) = 0 such that

∥xt − xs ∥2 ≤ ct ∥x0 − xs ∥2 + β(ε̄), 9.

compare (51, Theorem 3). For ε̄ = 0, the closed loop converges to the desired setpoint.
These theoretical guarantees require analogous assumptions as in the noise-free case (e.g.,
PE data, n is an upper bound on the system order, compare Section 3.1) and additionally
that the noise level ε̄ is sufficiently small and that the regularization parameters scale as

www.annualreviews.org • 9
 λα ∼ ε̄, λσ ∼ 1ε̄ . The latter conditions ensure that, in the noise-free case ε̄ = 0, robust data-
driven MPC (Algorithm 3.2) reduces to the nominal scheme (Algorithm 3.1) since λα = 0
and λσ → ∞ enforces σ(t) = 0. Beyond this stability result, the theoretical analysis reveals
direct connections between the data quality and the closed-loop performance. In particular,
the asymptotic tracking error decreases and the region of attraction increases if the minimum
singular value of the input Hankel matrix HL+2n (u) increases, which corresponds to a
quantitative PE condition (39). The possiblity to establish such end-to-end insights is a
valuable feature of direct data-driven control methods, which is often harder to achieve
in indirect approaches. While the original paper (51) required an additional non-convex
constraint ∥σ(t)∥∞ ≤ ε̄(1 + ∥α(t)∥1 ) to bound the slack variable, it was later shown in (58)
that this constraint can be dropped when λσ ∼ 1ε̄ , see also (68, Theorem 4.1).
As in the noise-free case, the occurrence of terminal equality constraints poses practical
limitations, e.g., a small region of attraction, and alternative means of guaranteeing closed-
loop stability are available. Roughly speaking, any model-based MPC scheme which admits
inherent robustness properties against disturbances can be transformed into a data-driven
MPC scheme with closed-loop guarantees in the presence of noise (69). Beyond this general
principle, the literature contains various concrete results on closed-loop guarantees in robust
data-driven MPC. For example, in the noisy data case, terminal ingredients can be designed
based on robust data-driven output-feedback design (55). By combining continuity of data-
driven MPC w.r.t. noise (70) and inherent robustness properties (69, 71), one can show that
the resulting data-driven MPC scheme admits closed-loop guarantees also in the presence of
noise, see (68, Section 4.5.2) for a discussion. Likewise, data-driven MPC without terminal
ingredients practically exponentially stabilizes the closed-loop system, assuming that the
prediction horizon L is sufficiently long (58). Notably, in both cases (terminal ingredients
and L sufficiently long), stability and robustness can be proven even for a 1-step MPC
scheme as in Algorithm 3.2. Alternatively, one can optimize over an artifical equilibrium
online which provides powerful practical features (no knowledge required if (us , ys ) is a
feasible equilibrium, improved robustness, increased region of attraction, handling online
setpoint changes) and also admits closed-loop guarantees in the presence of noise (70).
Due to the inaccurate predictions of the Fundamental Lemma with noisy data, handling
output constraints is non-trivial. By using a bound on the prediction error from (51), a
constraint tightening can be constructed which guarantees closed-loop output constraint
satisfaction (66). The conservatism of this tightening can be reduced, e.g., via a pre-
stabilizing feedback, and analogous results can be derived when the system is affected by
bounded disturbances (72).
In summary, the Fundamental Lemma can be used to design robust data-driven MPC
schemes for unknown LTI system based on noisy data. With appropriate modifications,
these schemes admit rigorous closed-loop guarantees on recursive feasibility, constraint sat-
isfaction, and practical exponential stability.

3.2.2. Regularization in data-driven MPC. While robust data-driven MPC (Algorithm 3.2)
differs from the nominal approach (Algorithm 3.1) in multiple aspects, the most crucial
modification is the regularization of α(t) in the cost. Indeed, satisfactory practical results
can often be obtained even without the slack variable σ(t). On the contrary, the absence
of the regularization of α(t) can pose difficulties already for tiny noise levels or even in
the complete absence of noise due to numerical inaccuracies. Intuitively, this regularization
can be understood as reducing the complexity of the employed prediction model based

10 Berberich & Allgöwer

on the Fundamental Lemma, analogous to regularization methods in linear regression. In
particular, due to the multiplication of α(t) with the noisy output Hankel matrix, values
α(t) with small norm are desired to reduce the prediction error due to the noise.
Since the introduction of regularization as a heuristic in (12), numerous recent works
have illuminated the role of regularization in data-driven MPC. First, we note that regular-
ization plays a crucial role in the closed-loop analysis of data-driven MPC, where it allows to
control the size of α(t), which influences the prediction error (51, 58, 66, 69, 70, 72). More-
over, a variety of works have focused on the effect of regularization in data-driven finite-
horizon optimal control, i.e., applying the full input sequence computed via the optimization
problem 8 in open loop. When considering stochastic noise, the regularized optimization
problem is equivalent to a min-max optimal control problem subject to distributional uncer-
tainty, i.e., regularization serves as a robustification against noise (73). The regularization
can also be derived through the lens of maximum likelihood estimation (74, 75) or by min-
imizing the expected cost rather than the nominal cost (76). Further, (77) has proposed
γ-data-driven predictive control (γ-DDPC), which decomposes the data-driven optimization
problem 8 into two separate steps and avoids the tuning of the regularization parameter λα .
The γ-DDPC scheme also allows to construct regularization terms which systematically re-
duce the prediction error and can be tuned without running additional experiments (78, 79).
This framework has been generalized in (80) via the Final Control Error, which can be used
to construct a data-driven MPC scheme that minimizes the actual prediction error due to
the noise and can be applied without tuning regularization parameters. Moreover, addi-
tional causality information can be used to improve performance in γ-DDPC (81). Notably,
equivalence between the classical regularized problem as in Algorithm 3.2 and γ-DDPC can
be established for suitable choices of the regularization parameters (82).
While the above approaches mainly focus on LTI systems with stochastic noise, regu-
larization can also be motivated via robustness against noise with deterministic description.
In particular, in (83), a data-driven optimal control problem with regularization is derived
as a tractable upper bound on a min-max problem against bounded uncertainty, where
different uncertainty structures lead to different regularization terms.
Regularization can also be studied through a bi-level optimization perspective, which
connects direct and indirect data-driven control: In (84), regularization terms are derived
which serve as a convex relaxation of low-rank approximation (with an ℓ1 -norm regular-
ization) or sequential least-squares estimation and model-based optimal control (with a
regularization ∥Πα(t)∥p for a suitable weighting matrix Π and some p-norm). The connec-
tion between different regularization approaches derived from bi-level optimization has been
further investigated in (85). Closely connected to these results, (86) studies regularization
via implicit predictors, which characterize the predictions caused by the constraints of the
data-driven optimization problem. Finally, regularization was used in (87, 88) to establish
equivalence between direct and indirect data-driven MPC.
To conclude, regularizing the parameter α(t) can be motivated through various reasons
which are all connected to enhancing robustness. More sophisticated regularized data-driven
MPC schemes, e.g., based on maximum likelihood estimation or γ-DDPC, can further
improve theoretical properties and admit practical benefits, e.g., not requiring to tune
additional regularization parameters. Finally, we note that the majority of the above works
focuses on the impact of regularization on the open-loop behavior, i.e., applying the full
optimal input sequence computed via the optimization problem 8. On the other hand,
studying the impact of regularization in closed loop, beyond the available robust stability

www.annualreviews.org • 11
 results (51, 58, 66, 69, 70, 72), and designing new regularization strategies via a closed-loop
perspective provides a promising future research direction.

3.2.3. Closed-loop guarantees for stochastic noise. While Section 3.2.1 has focused on
bounded noise, we now turn our attention to data-driven MPC for systems affected by
stochastic disturbances. One recent stream of works has employed Polynomial Chaos Ex-
pansions (PCE) for designing stochastic data-driven MPC schemes with closed-loop guar-
antees. Stochastic system trajectories are random variables which can be expressed (ap-
proximately) in a polynomial basis – the PCE basis. By relating the images of Hankel
matrices filled by three different quantities (random variables, PCE coefficients, and sam-
ples), a stochastic version of the Fundamental Lemma can be derived (89), see (90) for a
recent tutorial and overview of behavioral data-driven control for stochastic systems.
The stochastic Fundamental Lemma can be used to construct a data-driven MPC
scheme for unknown stochastic systems subject to chance constraints. In (91), it is shown
that this scheme admits closed-loop guarantees on recursive feasibility and practical stabil-
ity as well as closed-loop performance bound. Here, closed-loop guarantees are ensured via
terminal ingredients, i.e., a terminal cost function and a terminal region constraint, which
can be constructed based only on data. Subsequent works have extended the result in (91),
which considers state measurements and a binary selection of the initial condition as either
the measured state or a backup in case of infeasibility (the prediction from the previous
time step). In particular, (92) provides an MPC scheme with analogous guarantees for
input-output measurements and (93) optimizes the initial condition, interpolating between
the measured state and the backup. Since stochastic data-driven MPC based on PCE ad-
mits an increased computational cost, a tailored multiple shooting approach for complexity
reduction was derived in (94).
As an alternative to the PCE approach, closed-loop guarantees of data-driven MPC
with stochastic noise can be ensured by resorting to techniques from tube-based robust
MPC, i.e., computing a constraint tightening offline such that closed-loop guarantees can
be established (95). All previously mentioned approaches for stochastic data-driven MPC
require access to past disturbance realizations and, therefore, they derive data-driven es-
timation procedures. As an alternative, the recent work (96) constructs an explicit data-
driven parametrization of consistent disturbance realizations from which new samples can
be drawn, allowing to set up a sampling-based stochastic data-driven MPC scheme which
guarantees closed-loop properties via a robust first-step constraint.

3.3. Data-driven MPC for more advanced control objectives

Sections 3.1 and 3.2 focused on data-driven MPC schemes which are designed to (robustly)
stabilize an unknown LTI system. In the following, we discuss data-driven MPC approaches
with closed-loop guarantees that address more advanced setups and control objectives.
The Fundamental Lemma can be used to design distributed data-driven MPC schemes
for controlling multi-agent systems with coupled dynamics, where the combination of un-
known dynamics and local communication poses a key challenge. In (97, 98), distributed
data-driven MPC schemes are proposed which rely on state measurements and iterative dis-
tributed optimization and which admit closed-loop stability guarantees. Alternatively, (99)
developed a non-iterative scheme which requires a minimum amount of communication and
can cope with input-output measurements, but possibly admits increased conservatism.

12 Berberich & Allgöwer

 Further data-driven MPC formulations in the literature include explicit MPC (100),
which relies on an explicit solution of the underlying quadratic program via the Karush-
Kuhn-Tucker conditions, and economic MPC (101), which addresses stage cost functions
that are not positive (semi-)definite and can also handle unknown linear cost functions. In
the context of networked and cyber-physical control systems, data-driven MPC schemes
have been proposed that address resilience against denial-of-service attacks (102) as well
as self-triggered (103) and event-triggered (104) MPC formulations. Further, various con-
tributions have been made on safe data-driven control via the Fundamental Lemma. This
includes the design of a data-driven safety filter which equips any controller with safety
guarantees by ensuring that unsafe regions are avoided (105, 106), the combination of a
data-driven MPC scheme with a funnel controller (107), and enhancing data-driven MPC
with control barrier certificates (108). The Fundamental Lemma as well as data-driven
state-feedback design approaches (109, 55) have also inspired alternative data-driven MPC
schemes relying on linear matrix inequalities (110, 111). These approaches consider an
infinite-horizon cost and a state-feedback parametrization of the input, and they may be
less conservative than data-driven MPC when guaranteeing constraint satisfaction in the
presence of noise, see (111) for a numerical comparison. Further formulations include data-
driven MPC for linear descriptor systems (112) as well as data-driven control for iterative
tasks (113). Finally, while all the above approaches use the Fundamental Lemma for data-
driven control, the dual problem of observer design has also been studied: In (114), a data-
driven moving horizon estimation scheme is developed which proves practical exponential
stability of the estimator in case of noisy data.
In conclusion, the literature contains a variety of data-driven MPC schemes for different
problem setups, control objectives, and system classes. While many of these approaches
are inspired by analoguous developments in model-based MPC, the Fundamental Lemma
allows to set up MPC schemes for controlling unknown systems based only on measured
data, which brings unique challenges for the theoretical analysis, in particular in the presence
of uncertainty (e.g., noisy data or disturbances).

4. DATA-DRIVEN MPC FOR NONLINEAR SYSTEMS

While the theory of data-driven MPC is well-developed for linear systems, real-world appli-
cations to complex and safety-critical systems require more sophisticated techniques that
can also cope with nonlinear systems. In this section, we discuss data-driven MPC schemes
for nonlinear systems with systems-theoretic guarantees. As in Section 3, the covered
schemes use the Fundamental Lemma to predict future trajectories in a direct data-driven
fashion. However, the Fundamental Lemma crucially involves linearity, e.g., already the
statement is based purely on linear concepts such as matrices, vector spaces, and rank
conditions. Therefore, care has to be taken when using it for nonlinear systems.
In the literature, two complementary approaches for nonlinear data-driven MPC based
on the Fundamental Lemma have been developed, which we discuss in Sections 4.1 and 4.2,
respectively. The first class of methods rely on nonlinear versions of the Fundamental
Lemma, which can be developed in a tailored fashion for specific system classes by assuming
some structural knowledge, e.g., basis functions, and exploiting global linearity in higher-
dimensional coordinates. The second class follows an adaptive approach and updates the
data used in the Fundamental Lemma online, exploiting that nonlinear systems are locally
well-approximated by linear dynamics.

www.annualreviews.org • 13
 4.1. Exploiting global linearity via a nonlinear Fundamental Lemma
The recent literature contains various extensions of the Fundamental Lemma beyond LTI
systems. Tailored formulations have been derived for Hammerstein and Wiener sys-
tems (115), second-order Volterra systems (116), linear parameter-varying (LPV) sys-
tems (117), affine systems (70, 118), nonlinear autoregressive exogenous systems (119),
bilinear systems (120), linear time-periodic systems (121), and feedback linearizable sys-
tems (122). Further, nonlinear variations of the Fundamental Lemma have been developed
by resorting to multi-step predictors which are linear in basis functions (123), the Koopman
operator (124), and kernel methods (125, 126).
Each of these nonlinear versions of the Fundamental Lemma allows to design a data-
driven MPC scheme for systems in the associated system class. In the following, we show
how this can be done for Hammerstein systems via the corresponding Fundamental Lemma
from (115). Analogous MPC schemes can be designed for other nonlinear system classes
when using the corresponding version of the Fundamental Lemma, cf. above. A Hammer-
stein system is a series interconnection of a static nonlinearity and an LTI system, i.e.,

xk+1 = Axk + Bψ(uk ), yk = Cxk + Dψ(uk ) 10.

for k ∈ I≥0 , compare Figure 2. Here, ψ : Rm → Rnψ is a static nonlinear function which is
assumed to be composed of known basis functions with unknown coefficients, i.e., ψ(u) =
Pq m
i=1 ai ψi (u) for known functions ψi : R → Rnψ and unknown scalars ai ∈ R.

u ψ(u) y
Static nonlinearity LTI System

Figure 2
Block diagram illustration of a Hammerstein system, which consists of a series interconnection of
a static nonlinearity and an LTI system, compare Equation 10.

−1
For a given data trajectory {udk , ykd }N
k=0 and under suitable PE conditions, an arbitrary
L−1
input-output sequence {ūk , ȳk }k=0 is a trajectory of the system in Equation 10 if and only
if there exists α ∈ RN −L+1 such that
" # " #
HL (vd ) v̄
α= 11.
HL (yd ) ȳ

with auxiliary input sequences

   
ψ1 (udk ) ψ1 (ūk )
 .   . 
vkd = 
 ..  , k ∈ I[0,N −1] ,

 ..  , k ∈ I[0,L−1] ,
v̄k =   12.
ψq (udk ) ψq (ūk )

compare (115, Proposition 5). This result exploits the linearity of the system which maps
the auxiliary input v, whose components involve the actual input u and the known basis
functions ψi , to the output y. Since Equation 11 allows to accurately predict input-output
trajectories of the given Hammerstein system, it can be used to set up a data-driven MPC
scheme precisely as in the linear noise-free case considered in Section 3.1. Again, closed-
loop guarantees can be established via appropriate modifications of the basic optimization

14 Berberich & Allgöwer

problem, e.g., terminal equality constraints, terminal cost and terminal region constraints,
or a sufficiently long prediction horizon. The main limitation of this approach, beyond
its restriction to Hammerstein systems and knowledge of the ψi ’s, is that the resulting
optimization problem to be solved at every time step is typically non-convex due to the
nonlinear coupling of the decision variables ū and v̄ in Equation 12.
An analogous data-driven MPC scheme is proposed in (127) for feedback linearizable
systems based on the corresponding nonlinear Fundamental Lemma (122). The scheme
relies on terminal equality constraints and is shown in (128) to admit closed-loop guarantees
on practical exponential stability, even in the presence of noisy data or inaccurate basis
function approximation. Further, the works (129, 130) address data-driven MPC for LPV
systems based on the LPV Fundamental Lemma (117). LPV systems provide a promising
system class for data-driven control due to their ability to represent nonlinear systems while
maintaining a linear structure. In particular, the MPC scheme from (130) is based on solving
convex optimization problems and it includes terminal ingredients which can be designed
from data via LPV state-feedback design. Further, the MPC scheme admits closed-loop
guarantees on recursive feasibility, constraint satisfaction, and exponential stability.
To summarize, nonlinear versions of the Fundamental Lemma allow to design data-
driven MPC schemes with closed-loop guarantees, assuming that the underlying system
indeed belongs to the associated system class. In this case, the predictions are exact and the
main challenges for the theoretical analysis are analogous to the linear case, e.g., handling
positive semidefinite cost functions, designing terminal ingredients based on data, or coping
with uncertainty due to noise or inexact basis function representations (127). The benefits
of this theoretically sound treatment are met by possible challenges due to the assumptions
on the system class. While each of the above approaches is tailored to a specific system
class, it is non-trivial to verify whether a given system indeed belongs to the assumed class.

4.2. Exploiting local linearity via online data updates

As an alternative to nonlinear versions of the Fundamental Lemma, one can resort to a
linear Fundamental Lemma and update the data used for prediction in the Hankel matrices
online. Since (smooth) nonlinear systems are locally well-approximated by linear dynamics,
the resulting predictions are guaranteed to be accurate under suitable conditions. In the
following, we discuss this idea in more detail and introduce the data-driven MPC approach
from (70), which provides closed-loop guarantees when controlling unknown nonlinear sys-
tems based only on input-output data.
The control goal is to steer the output to a given setpoint ys while satisfying input
constraints uk ∈ U (guaranteeing output constraint satisfaction in this setup is an open
challenge). The considered system is assumed to be input-affine

xk+1 = f (xk ) + Buk , yk = h(xk ) + Duk 13.

with state xk ∈ Rn , input uk ∈ Rm , output yk ∈ Rp , and time k ∈ I≥0 . The key idea is
to use online data updates in order to parametrize trajectories of the affine linearization at
the current state, i.e., of

xk+1 = Axt xk + Buk + ext , yk = Cxt xk + Duk + rxt , 14.

www.annualreviews.org • 15
 with Jacobians and remainder terms
∂f ∂h
Axt = , Cxt = , ext = f (xt ) − Axt xt , rxt = h(xt ) − Cxt xt . 15.
∂x xt ∂x xt

Before introducing the data-driven MPC scheme, we take a slight detour to model-based
MPC. The work (131) proposes an MPC scheme to control nonlinear systems based on the
linearized dynamics in Equation 14, i.e., an MPC scheme based on precise model knowledge.
The approach includes a terminal equality constraint w.r.t. an artificial equilibrium xs (t)
for the linearized dynamics, i.e., a vector xs (t) satisfying

xs (t) = Axt xs (t) + Bus (t) + ext , ys (t) = Cxt xs (t) + Dus (t) + rxt 16.

for some equilibrium input us (t) and output ys (t), compare (61). The distance of the arti-
ficial equilibrium xs (t) to the actual setpoint is penalized in the cost. If the corresponding
cost weight is sufficiently small, then xs (t) is encouraged to remain close to the initial state
xt . If xt is, in turn, close to the steady-state manifold of the nonlinear system, then the full
predicted trajectory x̄(t) remains in a small region around xt , i.e., in a region where the
linearized dynamics in Equation 14 provide an accurate prediction. Based on this idea, it
can be shown that the MPC scheme based on the linearized dynamics indeed exponentially
stabilizes the setpoint in closed loop. Notably, this guarantee does not only hold locally
around the setpoint, but the region of attraction is a neighborhood of the entire steady-
state manifold, which can be significantly larger. Roughly speaking, e.g., in an autonomous
driving application, successful tracking of a position can be guaranteed when starting from
any initial position, under the restriction that the car does not drive too fast, i.e., it remains
close to the steady-state manifold.
Let us now return to a data-driven MPC setup, where the quantities f , h, B, D in Equa-
tion 13 are unknown. If, at time t, a PE input-output trajectory of the dynamics linearized
at xt was available, then the Fundamental Lemma for affine systems (70, 118) would yield
equivalent predictions to a state-space model. Thus, the results in (131) on model-based
MPC could be used to guarantee closed-loop exponential stability for the corresponding
data-driven MPC scheme. However, instead of data of the linearized dynamics, in practice
one only has access to measurements from the nonlinear system. The key insight to bridge
this gap lies in using, at time t, the most recent input-output measurements {uk , yk }t−1 k=t−N
for prediction, i.e., the predicted trajectory ū(t), ȳ(t) is parametrized via
" # " #
ū(t) HL+n (u[t−N,t−1] ) X
= α(t), αi (t) = 1. 17.
ȳ(t) HL+n (y[t−N,t−1] ) i

The first equation is analogous to the linear optimization problem 4 with the main differ-
ence of using online trajectories for prediction in the Hankel matrix rather than an offline
trajectory. The second equation ensures that the vector α(t) sums up to 1, which is required
due to the affine Fundamental Lemma (70, 118) since the linearized dynamics 14 are affine.
By using an MPC scheme with artificial equilibrium as in (131), it can be enforced
that the system does not change too rapidly in closed loop and, therefore, the trajectory
{uk , yk }t−1
k=t−N of the nonlinear system is close to a trajectory from the dynamics linearized
at xt . More precisely, if a certain cost matrix in the MPC optimization problem is chosen
small enough, then the closed-loop trajectory moves slowly such that Equation 17 provides
locally accurate predictions for the nonlinear system. Combining these ideas in a rigorous

16 Berberich & Allgöwer

fashion, it can be shown that the setpoint is practically exponentially stable for the closed-
loop system, i.e., the trajectory converges to a neighborhood around the setpoint (70,
Theorem 2). The size of this neighborhood depends on the distance between data samples
generated initially at time t = 0, . . . , N − 1, e.g., through random exploration. As in the
model-based MPC scheme from (131), the guaranteed region of attraction of the data-driven
MPC scheme is a neighborhood of the steady-state manifold of the underlying system.
There are several notable differences of this approach to those based on nonlinear ver-
sions of the Fundamental Lemma explained in Section 4.1. On the theoretical side, data-
driven MPC with online data updates provides guarantees for system classes which are
complementary to those considered in Section 4.1, relying on local arguments and there-
fore requiring entirely different and partially less restrictive assumptions: not restricted
to systems which can be represented linearly in suitable known coordinates; input-affine
dynamics (can be enforced via an input transformation); smooth dynamics; properties that
are also needed in model-based MPC, e.g., controllability, observability. On the practical
side, data-driven MPC with online data updates only requires input-output measurements
for its implementation but no choice of basis functions for nonlinear components as in most
approaches from Section 4.1. Further, since the approach relies on the affine Fundamental
Lemma, the resulting optimization problems are strictly convex quadratic programs, in con-
trast to data-driven MPC based on nonlinear basis functions. Further, data-driven MPC
with online data updates has been successfully applied in real-world experiments, including
a four-tank system (14) and a complex soft robot (17). A related data-driven MPC scheme
which also uses online data updates but does not admit closed-loop guarantees was applied
to a building control experiment in (26).
On the other hand, updating the data online brings additional challenges. In partic-
ular, the data collected from the nonlinear system in closed loop need to carry sufficient
information, i.e., they need to be PE. In practice, this can be ensured, e.g., by adding an
excitation signal to the input applied to the system, by stopping the data updates once
the closed-loop system has (approximately) converged, or by incentivizing PE inputs via
an additional PE cost (10). Addressing the problem rigorously and providing a theoretical
solution is an interesting direction for future research.

5. DISCUSSION
When applying data-driven control to real-world systems, especially in complex and safety-
critical applications, it is desirable to provide rigorous systems-theoretic guarantees for
the closed-loop operation. In this review, we discussed direct data-driven MPC methods
based on the Fundamental Lemma which do provide such guarantees, e.g., on stability, ro-
bustness, and constraint satisfaction. The presented schemes can control unknown systems
based only on input-output data in various scenarios including linear and nonlinear systems
as well as noise-free and noisy data. As in the model-based case, closed-loop stability of
data-driven MPC is not necessarily guaranteed but needs to be ensured via suitable terminal
ingredients or a sufficiently long prediction horizon. When controlling linear systems based
on noise-free data, techniques from model-based MPC can be borrowed, but the technical
analysis needs to additionally cope with input-output cost functions. In the more realistic
scenario of noisy data (bounded or stochastic), desirable guarantees can be provided under
appropriate modifications of the MPC scheme, e.g., adding a regularization. For nonlinear
systems, we discussed two alternatives based on either global or local linearity. The former

www.annualreviews.org • 17
 approach exploits that systems from specific classes can be represented in linear coordinates
using knowledge of suitable basis functions, allowing to formulate nonlinear versions of the
Fundamental Lemma that can be used for MPC. Alternatively, nonlinear systems can be
controlled by updating the data used for prediction via the Fundamental Lemma online,
exploiting local linearity of (smooth) nonlinear dynamics. In guaranteeing closed-loop prop-
erties for the presented approaches, the key challenge is to combine established concepts
from model-based MPC with unique challenges when using the Fundamental Lemma for
prediction, in particular inaccuracies due to noise or nonlinearities.
A natural question in the context of data-driven MPC is the connection between direct
approaches based on the Fundamental Lemma and indirect schemes such as SPC (132).
For linear systems and noise-free data, basic formulations of SPC and data-driven MPC are
equivalent since both employ exact predictions (133, 134). This equivalence carries over to
more advanced formulations, e.g., including regularization (compare Section 3.2.2), and the
synergy between both approaches has led to the development of new data-driven control
methods (77, 60, 135). In the presence of inaccuracies, e.g., noisy data or nonlinearities,
the predictions in direct and indirect data-driven MPC are no longer equivalent and study-
ing performance gaps becomes relevant. The literature contains various empirical studies,
see (5, 36, 37) and the references therein, but also theoretical results indicating that the
open-loop performance of either direct or indirect approaches can be superior depending on
different factors including the data length (136). Deriving formal closed-loop results on this
gap, e.g., considering stability or closed-loop performance guarantees but also conservatism
of constraint tightenings, is an open research problem. To the best of the authors’ knowl-
edge, the existing SPC literature does not contain closed-loop guarantees under assumptions
comparable to the direct data-driven MPC results covered in this review, in particular in
the presence of noise or nonlinearities. Deriving such guarantees may provide a meaningful
step towards understanding the interplay between SPC and direct data-driven MPC.
In terms of complexity, indirect approaches have the advantage of being less com-
putationally demanding, whereas the number of decision variables in direct approaches
grows with the data length, compare the optimization problems 4 and 8. This has mo-
tivated contributions on complexity reduction in direct data-driven MPC, e.g., based on
wavelets (137), trajectory segmentation (138), singular value decomposition (139), alter-
native data-driven system representations (140), iterative solvers (141), and differentiable
convex programming (142). Finally, another noteworthy distinction arises when designing
indirect data-driven MPC schemes via identified state-space models, which admit several
inherent differences to the multi-step predictor based on the Fundamental Lemma (143).
Data-driven MPC is a promising modern control approach with remarkable empirical
performance and a solid theoretical foundation. We conclude the review by discussing open
research directions that may further enhance the theoretical understanding and reliability
of data-driven MPC, thereby simplifying its application in challenging control problems.

FUTURE ISSUES
1. Data-driven MPC for nonlinear systems: Data-driven methods are most useful in
cases where first-principles models are hard to obtain, e.g., for complex systems
with nonlinear dynamics. Current nonlinear data-driven MPC approaches are either
tailored to specific system classes, which limits their practicality, or employ online
data updates, which requires PE closed-loop trajectories. Developing a unifying

18 Berberich & Allgöwer

 data-driven MPC framework in the nonlinear regime, maintaining both rigorous
guarantees as well as practical applicability, is an important future direction.
2. Online data updates in data-driven MPC: The overwhelming majority of existing
data-driven MPC approaches focuses on a setup where the data used for prediction
via Hankel matrices are collected offline. On the other hand, if new data samples
are obtained in closed-loop operation, it is desirable to exploit the information
contained in the data to improve the performance. Understanding the role of data
in data-driven MPC, in particular in adaptive setups where new samples enter or
old data samples are deleted, is a crucial future research challenge.
3. Direct vs. indirect data-driven MPC: While the interplay between direct and indi-
rect data-driven control has received substantial attention, an ultimate assessment
of their respective advantages and drawbacks requires further research. One promis-
ing direction is to transfer performance bounds, e.g., from (136), to more general
scenarios such as bounded noise, nonlinear systems, or closed-loop performance.
4. Transferring open-loop results to closed-loop formulations: Several effects in data-
driven MPC are by now well-understood in open-loop optimal control formulations,
but have not been researched extensively w.r.t. closed-loop guarantees, e.g., reg-
ularization. On the other hand, when applying data-driven MPC, typically the
closed-loop performance is the main object of interest. Thus, transferring open-
loop to closed-loop results is another promising future research avenue.

DISCLOSURE STATEMENT
The authors are not aware of any affiliations, memberships, funding, or financial holdings
that might be perceived as affecting the objectivity of this review.

ACKNOWLEDGMENTS
The authors are thankful to Tim Martin for helpful comments. F. Allgöwer is thankful that
this work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research
Foundation) under Germany’s Excellence Strategy - EXC 2075 - 390740016 and within
grant AL 316/15-1 - 468094890. The authors would like to thank the Stuttgart Center for
Simulation Science (SimTech) for the support.

LITERATURE CITED
1. Ljung L. 1987. System Identification: Theory for the User. Prentice-Hall, Englewood Cliffs,
NJ
2. Tsiamis A, Ziemann I, Matni N, Pappas GJ. 2023. Statistical learning theory for control: a
finite-sample perspective. IEEE Control Systems Magazine 43(6):67–97
3. Hou ZS, Wang Z. 2013. From model-based control to data-driven control: Survey, classification
and perspective. Information Sciences 235:3–35
4. Willems JC, Rapisarda P, Markovsky I, De Moor B. 2005. A note on persistency of excitation.
Syst. Contr. Lett. 54:325–329
5. Markovsky I, Dörfler F. 2021. Behavioral systems theory in data-driven analysis, signal pro-
cessing, and control. Annual Reviews in Control 52:42–64

www.annualreviews.org • 19
 6. Rawlings JB, Mayne DQ, Diehl MM. 2020. Model Predictive Control: Theory, Computation,
and Design. Nob Hill Pub. 3rd printing
7. Hewing L, Wabersich KP, Menner M, Zeilinger MN. 2020. Learning-based model predictive
control: Toward safe learning in control. Annual Review of Control, Robotics, and Autonomous
Systems 3:269–296
8. Adetola V, Guay M. 2011. Robust adaptive MPC for constrained uncertain nonlinear systems.
Int. J. Adaptive Control and Signal Processing 25(2):155–167
9. Tanaskovic M, Fagiano L, Smith R, Morari M. 2014. Adaptive receding horizon control for
constrained MIMO systems. Automatica 50(12):3019–3029
10. Lu X, Cannon M, Koksal-Rivet D. 2021. Robust adaptive model predictive control: perfor-
mance and parameter estimation. Int. J. Robust and Nonlinear Control 31(18):8703–8724
11. Yang H, Li S. 2015. A data-driven predictive controller design based on reduced hankel matrix.
In Proc. Asian Control Conference, pp. 1–7
12. Coulson J, Lygeros J, Dörfler F. 2019. Data-enabled predictive control: in the shallows of the
DeePC. In Proc. European Control Conf. (ECC), pp. 307–312
13. Elokda E, Coulson J, Lygeros J, Dörfler F. 2021. Data-enabled predictive control for quad-
copters. Int. J. Robust and Nonlinear Control 31(18):8916–8936
14. Berberich J, Köhler J, Müller MA, Allgöwer F. 2021. Data-driven model predictive control:
closed-loop guarantees and experimental results. at-Automatisierungstechnik 69(7):608–618
15. Carlet PG, Favato A, Bolognani S, Dörfler F. 2022. Data-driven continuous-set predictive
current control for synchronous motor drives. IEEE Trans. Power Electronics 37(6):6637–
6646
16. Fawcett RT, Afsari K, Ames AD, Hamed KA. 2022. Toward a data-driven template model for
quadrupedal locomotion. IEEE Robotics and Automation Letters 7(3):7636–7643
17. Müller D, Feilhauer J, Wickert J, Berberich J, Allgöwer F, Sawodny O. 2022. Data-driven
predictive disturbance observer for quasi continuum manipulators. In Proc. 61st IEEE Conf.
Decision and Control (CDC), pp. 1816–1822
18. Hemming S, de Zwart F, Elings E, Petropoulou A, Righini I. 2020. Cherry tomato production
in intelligent greenhouses - sensors and AI for control of climate, irrigation, crop yield, and
quality. Sensors 20(22):6430
19. Wang J, Lian Y, Jiang Y, Xu Q, Li K, Jones CN. 2023. Distributed data-driven predictive con-
trol for cooperatively smoothing mixed traffic flow. Transportation Research Part C: Emerging
Technologies 155:104274
20. Wang J, Zheng Y, Li K, Xu Q. 2023. DeeP-LCC: data-enabled predictive leading
cruise control in mixed traffic flow. IEEE Trans. Control Systems Technology Doi:
10.1109/TCST.2023.3288636
21. Shang X, Wang J, Zheng Y. 2023. Smoothing mixed traffic with robust data-driven predictive
control for connected and autonomous vehicles. arXiv:2310.00509
22. Rimoldi A, Cenedese C, Padoan A, Dörfler F, Lygeros J. 2023. Urban traffic congestion control:
a DeePC change. arXiv:2311.09851
23. Huang L, Coulson J, Lygeros J, Dörfler F. 2019. Data-enabled predictive control for grid-
connected power converters. In Proc. 58th IEEE Conf. Decision and Control (CDC), pp.
8130–8135
24. Huang L, Zhen J, Lygeros J, Dörfler F. 2021. Quadratic regularization of data-enabled pre-
dictive control: theory and application to power converter experiments. IFAC-PapersOnLine
54(7):192–197
25. Huang L, Coulson J, Lygeros J, Dörfler F. 2021. Decentralized data-enabled predictive con-
trol for power system oscillation damping. IEEE Trans. Control Systems Technology Doi:
10.1109/TCST.2021.3088638
26. Lian Y, Shi J, Koch MP, Jones CN. 2021. Adaptive robust data-driven building control via
bi-level reformulation: an experimental result. arXiv:2106.05740

20 Berberich & Allgöwer

27. O’Dwyer E, Falugi P, Shah N, Kerrigan E. 2022. Automating the data-driven predictive con-
trol design process for building thermal management. In Proc. 35th Int. Conf. on Efficiency,
Cost, Optimization, Simulation and Environmental Impact of Energy Systems (ECOS). Doi:
10.11581/dtu.00000267
28. Di Natale L, Lian Y, Maddalena ET, Shi J, Jones CN. 2022. Lessons learned from data-driven
building control experiments: contrasting Gaussian process-based MPC, bilevel DeePC, and
deep reinforcement learning. In Proc. 61st IEEE Conf. Decision and Control (CDC), pp.
1111–1117
29. Behrunani V, Zagorowska M, Hudoba de Badyn M, Ricca F, Heer P, Lygeros J. 2023.
Degradation-aware data-enabled predictive control of energy hubs. arXiv:2307.01543
30. Chen K, Zhang X, Lin X, Zheng Y, Yin X, et al. 2022. Data-enabled predictive control for
fast charging of Lithium-Ion batteries with constraint handling. arXiv:2209.12862
31. Mahdavipour P, Wieland C, Spliethoff H. 2022. Optimal control of combined-cycle power
plants: a data-enabled predictive control perspective 55(13):91–96
32. Bilgic D, Koch A, Pan G, Faulwasser T. 2022. Toward data-driven predictive control of multi-
energy distribution systems. Electric Power Systems Research 212:108311
33. Schmitz P, Engelmann A, Faulwasser T, Worthmann K. 2022. Data-driven MPC of descriptor
systems: a case study for power networks. IFAC-PapersOnLine 55(30):359–364
34. Yin M, Cai H, Gattiglio A, Khayatian F, Smith RS. 2024. Data-driven predictive control for
demand side management: Theoretical and experimental results. Applied Energy 353:122101
35. Schmitt L, Beerwerth J, Bahr M, Abel D. 2023. Data-driven predictive control with online
adaption: application to a fuel cell system. IEEE Trans. Control Systems Technology Doi:
10.1109/TCST.2023.3293790
36. Markovsky I, Huang L, Dörfler F. 2023. Data-driven control based on the behavioral approach:
from theory to applications in power systems. IEEE Control Systems
37. Verheijen PCN, Breschi V, Lazar M. 2023. Handbook of linear data-driven predictive control:
Theory, implementation and design. Annual Reviews in Control 56:100914
38. van Waarde HJ, De Persis C, Camlibel MK, Tesi P. 2020. Willems’ fundamental lemma
for state-space systems and its extension to multiple datasets. IEEE Control Systems Lett.
4(3):602–607
39. Berberich J, Iannelli A, Padoan A, Coulson J, Dörfler F, Allgöwer F. 2023. A quantitative
and constructive proof of Willems’ Fundamental Lemma. In Proc. American Control Conf.
(ACC), pp. 4155–4160
40. Markovsky I, Rapisarda P. 2008. Data-driven simulation and control. Int. J. Control
81(12):1946–1959
41. Maupong TM, Mayo-Maldonado JC, Rapisarda P. 2017. On Lyapunov functions and data-
driven dissipativity. IFAC-PapersOnLine 50(1):7783–7788
42. Romer A, Berberich J, Köhler J, Allgöwer F. 2019. One-shot verification of dissipativity prop-
erties from input-output data. IEEE Control Systems Lett. 3(3):709–714
43. De Persis C, Tesi P. 2020. Formulas for data-driven control: stabilization, optimality and
robustness. IEEE Trans. Automat. Control 65(3):909–924
44. Berberich J, Koch A, Scherer CW, Allgöwer F. 2020. Robust data-driven state-feedback design.
In Proc. American Control Conf. (ACC), pp. 1532–1538
45. van Waarde HJ, Eising J, Trentelman HL, Camlibel MK. 2020. Data informativity: a new
perspective on data-driven analysis and control. IEEE Trans. Automat. Control 65(11):4753–
4768
46. Martin T, Schön TB, Allgöwer F. 2023. Guarantees for data-driven control of nonlinear systems
using semidefinite programming: A survey. Annual Reviews in Control :100911
47. De Persis C, Tesi P. 2023. Learning controllers for nonlinear systems from data. Annual Reviews
in Control :100915
48. van Waarde HJ, Eising J, Camlibel MK, Trentelman HL. 2023. The informativity approach

www.annualreviews.org • 21
 to data-driven analysis and control. IEEE Control Systems Magazine 43(6):32–66
49. Raff T, Huber S, Nagy ZK, Allgöwer F. 2006. Nonlinear model predictive control of a four
tank system: An experimental stability study. In Proc. IEEE Int. Conf. Control Applications
(CCA), pp. 237–242
50. Grüne L. 2012. NMPC without terminal constraints. In Proc. IFAC Conf. Nonlinear Model
Predictive Control, pp. 1–13
51. Berberich J, Köhler J, Müller MA, Allgöwer F. 2021. Data-driven model predictive control
with stability and robustness guarantees. IEEE Trans. Automat. Control 66(4):1702–1717
52. Berberich J, Köhler J, Müller MA, Allgöwer F. 2021. On the design of terminal ingredients
for data-driven MPC. IFAC-PapersOnLine 54(6):257–263
53. Goodwin GC, Sin KS. 2014. Adaptive filtering prediction and control. Courier Corporation
54. Koch A, Berberich J, Allgöwer F. 2022. Provably robust verification of dissipativity properties
from data. IEEE Trans. Automat. Control 67(8):4248–4255
55. Berberich J, Scherer CW, Allgöwer F. 2023. Combining prior knowledge and data for robust
controller design. IEEE Trans. Automat. Control 68(8):4618–4633
56. van Waarde HJ, Eising J, Camlibel MK, Trentelman HL. 2024. A behavioral approach to data-
driven control with noisy input-output data. IEEE Trans. Automat. Control 69(2):813–827
57. Alsalti M, Lopez VG, Müller MA. 2023. Notes on data-driven output-feedback control of linear
MIMO systems. arXiv:2311.17484
58. Bongard J, Berberich J, Köhler J, Allgöwer F. 2023. Robust stability analysis of a simple data-
driven model predictive control approach. IEEE Trans. Automat. Control 68(5):2625–2637
59. Lazar M. 2021. A dissipativity-based framework for analyzing stability of predictive controllers.
IFAC-PapersOnLine 54(6):159–165
60. Lazar M, Verheijen PCN. 2023. Generalized data-driven predictive control: merging subspace
and Hankel predictors. Mathematics 11(9):2216
61. Limón D, Alvarado I, Alamo T, Camacho EF. 2008. MPC for tracking piecewise constant
references for constrained linear systems. Automatica 44(9):2382–2387
62. Berberich J, Köhler J, Müller MA, Allgöwer F. 2020. Data-driven tracking MPC for changing
setpoints. IFAC-PapersOnLine 53(2):6923–6930
63. Cai C, Teel AR. 2008. Input–output-to-state stability for discrete-time systems. Automatica
44(2):326–336
64. Favoreel W, Moor BD, Gevers M. 1999. SPC: Subspace predictive control. IFAC Proceedings
Volumes 32(2):4004–4009
65. Huang B, Kadali R. 2008. Dynamic modeling, predictive control and performance monitoring:
a data-driven subspace approach. Springer
66. Berberich J, Köhler J, Müller MA, Allgöwer F. 2020. Robust constraint satisfaction in data-
driven MPC. In Proc. 59th IEEE Conf. Decision and Control (CDC), pp. 1260–1267
67. Grüne L, Palma VG. 2015. Robustness of performance and stability for multistep and updated
multistep MPC schemes. Discrete and Continuous Dynamical Systems 35(9):4385–4414
68. Berberich J. 2022. Stability and robustness in data-driven predictive control. Ph.D. thesis,
University of Stuttgart
69. Berberich J, Köhler J, Müller MA, Allgöwer F. 2022. Stability in data-driven MPC: an inherent
robustness perspective. In Proc. 61st IEEE Conf. Decision and Control (CDC), pp. 1105–1110
70. Berberich J, Köhler J, Müller MA, Allgöwer F. 2022. Linear tracking MPC for nonlinear sys-
tems part II: the data-driven case. IEEE Trans. Automat. Control 67(9):4406–4421Extended
version on arXiv:2105.08567
71. Yu S, Reble M, Chen H, Allgöwer F. 2014. Inherent robustness properties of quasi-infinite
horizon nonlinear model predictive control. Automatica 50(9):2269–2280
72. Klöppelt C, Berberich J, Allgöwer F, Müller MA. 2022. A novel constraint tightening ap-
proach for robust data-driven predictive control. Int. J. Robust and Nonlinear Control Doi:
10.1002/rnc.6532

22 Berberich & Allgöwer

73. Coulson J, Lygeros J, Dörfler F. 2022. Distributionally robust chance constrained data-enabled
predictive control. IEEE Trans. Automat. Control 67(7):3289–3304
74. Yin M, Iannelli A, Smith RS. 2021. Maximum likelihood signal matrix model for data-driven
predictive conntrol. In Proc. 3rd Conf. Learning for Dynamics and Control (L4DC), vol. 144,
pp. 1004–1014. PMLR
75. Yin M, Iannelli A, Smith RS. 2021. Maximum likelihood estimation in data-driven modeling
and control. IEEE Trans. Automat. Control Doi: 10.1109/TAC.2021.3137788
76. Yin M, Iannelli A, Smith RS. 2023. Stochastic data-driven predictive control: regularization,
estimation, and constraint tightening. arXiv:2312.02758
77. Breschi V, Chiuso A, Formentin S. 2023. Data-driven predictive control in a stochastic setting:
a unified framework. Automatica 152:110961
78. Breschi V, Fabris M, Formentin S, Chiuso A. 2023. Uncertainty-aware data-driven predictive
control in a stochastic setting. IFAC-PapersOnLine 56(2):10083–10088
79. Breschi V, Chiuso A, Fabris M, Formentin S. 2023. On the impact of regularization in data-
driven predictive control. arXiv:2304.00263
80. Chiuso A, Fabris M, Breschi V, Formentin S. 2023. Harnessing the final control error for
optimal data-driven predictive control. arXiv:2312.14788
81. Sader M, Wang Y, Huang D, Shang C, Huang B. 2023. Causality-informed data-driven pre-
dictive control. arXiv:2311.09545
82. Klädtke M, Schulze Darup M. 2024. Towards a unifying framework for data-driven predictive
control with quadratic regularization. arXiv:2404.02721
83. Huang L, Lygeros J, Dörfler F. 2022. Robust and kernelized data-enabled predictive control
for nonlinear systems. arXiv:2206.01866
84. Dörfler F, Coulson J, Markovsky I. 2023. Bridging direct & indirect data-driven control for-
mulations via regularizations and relaxations. IEEE Trans. Automat. Control 68(2):883–897
85. Shang X, Zheng Y. 2023. Convex aproximations for a bi-level formulation of data-enabled
predictive control. arXiv:2312.15431
86. Klädtke M, Schulze Darup M. 2023. Implicit predictors in regularized data-driven predicive
control. IEEE Control Systems Lett. 7:2479–2484
87. Mattsson P, Schön TB. 2023. On the regularization in DeePC. IFAC-PapersOnLine 56(2):625–
631
88. Mattsson P, Bonassi F, Breschi V, Schön TB. 2024. On the equivalence of direct and indirect
data-driven predictive control approaches. arXiv:2403.05860
89. Pan G, Ou R, Faulwasser T. 2022. On a stochastic fundamental lemma and its use for data-
driven MPC. IEEE Trans. Automat. Control 68(10):5922–5937
90. Faulwasser T, Ou R, Pan G, Schmitz P, Worthmann K. 2023. Behavioral theory for stochastic
systems? A data-driven journey from Willems to Wiener and back again. Annual Reviews in
Control 55:92–117
91. Pan G, Ou R, Faulwasser T. 2022. Towards data-driven stochastic predictive control.
arXiv:2212.10663
92. Pan G, Ou R, Faulwasser T. 2022. On data-driven stochastic output-feedback predictive con-
trol. arXiv:2211.17074
93. Pan G, Ou R, Faulwasser T. 2023. Data-driven stochastic output-feedback predictive control:
recursive feasibility through interpolated initial conditions. In Proc. 5th Conf. Learning for
Dynamics and Control (L4DC), vol. 211, pp. 980–992. PMLR
94. Ou R, Pan G, Faulwasser T. 2022. Data-driven multiple shooting for stochastic optimal control.
IEEE Control Systems Lett. 7:313–318
95. Kerz S, Teutsch J, Brüdigam T, Wollherr D, Leibold M. 2023. Data-driven tube-based stochas-
tic predictive control. IEEE Open Journal of Control Systems 2:185–199
96. Teutsch J, Kerz S, Wollherr D, Leibold M. 2024. Sampling-based stochastic data-driven pre-
dictive control under data uncertainty. arXiv:2402.00681

www.annualreviews.org • 23
 97. Allibhoy A, Cortés J. 2021. Data-based receding horizon control of linear network systems.
IEEE Control Systems Lett. 5(4):1207–1212
98. Alonso CA, Yang F, Matni N. 2022. Data-driven distributed and localized model predictive
control. IEEE Open Journal of Control Systems 1:29–40
99. Köhler M, Berberich J, Müller MA, Allgöwer F. 2022. Data-driven distributed MPC of dy-
namically coupled linear systems. IFAC-PapersOnLine 55(30):365–370
100. Breschi V, Sassella A, Formentin S. 2023. On the design of regularized explicit predictive
controllers from input-output data. IEEE Trans. Automat. Control 68(8):4977–4983
101. Xie Y, Berberich J, Allgöwer F. 2023. Linear data-driven economic MPC with generalized
terminal constraint. IFAC-PapersOnLine 56(2):5512–5517
102. Liu W, Sun J, Wang G, Bullo F, Chen J. 2023. Data-driven resilient predictive control under
denial-of-service. IEEE Trans. Automat. Control 68(8):4722–4737
103. Liu W, Sun J, Wang G, Bullo F, Chen J. 2023. Data-driven self-triggered control via trajectory
prediction. IEEE Trans. Automat. Control 68(11):6951–6958
104. Deng L, Shu Z, Chen T. 2024. Event-triggered robust MPC with terminal in-
equality constraints: a data-driven approach. IEEE Trans. Automat. Control Doi:
10.1109/TAC.2024.3357417
105. Bajelani M, van Heusden K. 2023. Data-driven safety filter: An input-output perspective.
arXiv:2309.00189
106. Bajelani M, van Heusden K. 2024. From raw data to safety: reducing conservatism by set
expansion. arXiv:2403.15883
107. Schmitz P, Lanza L, Worthmann K. 2023. Safe data-driven reference tracking with prescribed
performance. In Proc. 27th Int. Conf. on System Theory, Control and Computing (ICSTCC),
pp. 454–460
108. Khaledi M, Tooranjipour P, Kiumarsi B. 2023. Data-driven safety-certified predictive control
for linear systems. IEEE Control Systems Lett. 7:3687–3692
109. van Waarde HJ, Camlibel MK, Mesbahi M. 2022. From noisy data to feedback controllers:
non-conservative design via a matrix S-lemma. IEEE Trans. Automat. Control 67(1):162–175
110. Nguyen HH, Friedel M, Findeisen R. 2023. LMI-based data-driven robust model predictive
control. IFAC-PapersOnLine 56(2):4783–4788
111. Xie Y, Berberich J, Allgöwer F. 2024. Data-driven min-max MPC for linear systems: robust-
ness and adaptation. arXiv:2404.19096
112. Schmitz P, Faulwasser T, Worthmann K. 2022. Willems’ fundamental lemma for linear de-
scriptor systems and its use for data-driven output-feedback MPC. IEEE Control Systems
Lett. 6:2443–2448
113. Zhang K, Zuliani R, Balta EC, Lygeros J. 2024. Data-enabled predictive iterative control.
IEEE Control Systems Lett. Doi: 10.1109/LCSYS.2024.3408073
114. Wolff TM, Lopez VG, Müller MA. 2022. Robust data-driven moving horizon estimation for
linear discrete-time systems. arXiv:2210.09017
115. Berberich J, Allgöwer F. 2020. A trajectory-based framework for data-driven system analysis
and control. In Proc. European Control Conf. (ECC), pp. 1365–1370
116. Rueda-Escobedo JG, Schiffer J. 2020. Data-driven internal model control of second-order dis-
crete Volterra systems. In Proc. 59th IEEE Conf. Decision and Control (CDC), pp. 4572–4579
117. Verhoek C, Tóth R, Haesart S, Koch A. 2021. Fundamental Lemma for data-driven analysis
of linear parameter-varying systems. In Proc. 60th IEEE Conf. Decision and Control (CDC),
pp. 5040–5046
118. Martinelli A, Gargiani M, Draskovic M, Lygeros J. 2022. Data-driven optimal control of affine
systems: a linear programming perspective. arXiv:2203.12044
119. Mishra VK, Markovsky I, Fazzi A, Dreesen P. 2021. Data-driven simulation for NARX sys-
tems. In Proc. European Signal Processing Conference, pp. 1055–1059
120. Yuan Z, Cortés J. 2022. Data-driven optimal control of bilinear systems. IEEE Control Systems

24 Berberich & Allgöwer

 Lett. 6:2479–2484
121. Li R, Simpson-Porco JW, Smith SL. 2022. Data-driven model predictive control for linear
time-periodic systems. In Proc. 61st IEEE Conf. Decision and Control, pp. 3661–3668
122. Alsalti M, Lopez VG, Berberich J, Allgöwer F, Müller MA. 2023. Data-based control of feed-
back linearizable systems. IEEE Trans. Automat. Control 68(11):7014–7021
123. Lazar M. 2023. Basis functions nonlinear data-enabled predictive control: consistent and com-
putationally efficient formulations. arXiv:2311.05360
124. Lian Y, Wang R, Jones CN. 2021. Koopman based data-driven predictive control.
arXiv:2102.05122
125. Huang L, Zhen J, Lygeros J, Dörfler F. 2023. Robust data-enabled predictive control: tractable
formulations and performance guarantees. IEEE Trans. Automat. Control 68(5):3163–3170
126. Molodchyk O, Faulwasser T. 2024. Exploring the links between the Fundamental Lemma and
kernel regression. arXiv:2403.05368
127. Alsalti M, Lopez VG, Berberich J, Allgöwer F, Müller MA. 2023. Data-driven nonlinear pre-
dictive control for feedback linearizable systems. IFAC-PapersOnLine 56(2):617–624
128. Alsalti M, Lopez VG, Berberich J, Allgöwer F, Müller MA. 2022. Practical exponential stability
of a robust data-driven nonlinear predictive control scheme. arXiv:2204.01150 Supplementary
material
129. Verhoek C, Abbas HS, Tóth R, Haesart S. 2021. Data-driven predictive control for linear
parameter-varying systems. IFAC-PapersOnLine 54(8):101–108
130. Verhoek C, Berberich J, Haesaert S, Tóth R, Abbas HS. 2023. A linear parameter-varying
approach to data predictive control. arXiv:2311.07140
131. Berberich J, Köhler J, Müller MA, Allgöwer F. 2022. Linear tracking MPC for nonlinear
systems part I: the model-based case. IEEE Trans. Automat. Control 67(9):4390–4405
132. Dörfler F. 2023. Data-driven control: part two of two: hot take: why not go with models?
IEEE Control Systems Magazine 43(6):27–31
133. Fiedler F, Lucia S. 2021. On the relationship between data-enabled predictive control and
subspace predictive control. In Proc. European Control Conf. (ECC), pp. 222–229
134. Lazar M, Verheijen PCN. 2022. Offset-free data-driven predictive control. In Proc. 61st IEEE
Conf. Decision and Control (CDC), pp. 1099–1104
135. Smith RS, Abdalmoaty M, Yin M. 2024. Optimal data-driven prediction and predictive control
using signal matrix models. arXiv:2403.15329
136. Krishnan V, Pasqualetti F. 2021. On direct vs indirect data-driven predictive control. In Proc.
60th IEEE Conf. Decision and Control (CDC), pp. 736–741
137. Sathyanarayanan KK, Pan G, Faulwasser T. 2023. Towards data-driven predictive control
using wavelets. IFAC-PapersOnLine 56(2):632–637
138. O’Dwyer E, Kerrigan EC, Falugi P, Zagorowska M, Shah N. 2023. Data-driven predictive con-
trol with improved performance using segmented trajectories. IEEE Trans. Control Systems
Technology 31(3):1355–1365
139. Zhang K, Zheng Y, Shang C, Li Z. 2023. Dimension reduction for efficient data-enabled pre-
dictive control. IEEE Control Systems Lett. 7:3277–3282
140. Alsalti M, Barkey M, Lopez VG, Müller MA. 2023. Sample- and computationally efficient
data-driven predictive control. arXiv:2309.11238
141. Schmitz P, Schaller M, Voigt M, Worthmann K. 2024. Fast and memory-efficient optimization
for large-scale data-driven predictive control. arXiv:2402.13090
142. Zhou Y, Lu Y, Li Z, Yan J, Mo Y. 2024. Learning-based efficient approximation of data-enabled
predictive control. arXiv:2404.16727
143. Köhler J, Wabersich KP, Berberich J, Zeilinger MN. 2022. State space models vs. multi-step
predictors in predictive control: Are state space models complicating safe data-driven designs?
In Proc. 61st IEEE Conf. Decision and Control (CDC), pp. 491–498

www.annualreviews.org • 25
NASA/TM—2007-215007

Feedback Control Systems Loop Shaping

Design With Practical Considerations
George Kopasakis
Glenn Research Center, Cleveland, Ohio

September 2007
 NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the • CONFERENCE PUBLICATION. Collected
advancement of aeronautics and space science. The papers from scientific and technical
NASA Scientific and Technical Information (STI) conferences, symposia, seminars, or other
program plays a key part in helping NASA maintain meetings sponsored or cosponsored by NASA.
this important role.
• SPECIAL PUBLICATION. Scientific,
The NASA STI Program operates under the auspices technical, or historical information from
of the Agency Chief Information Officer. It collects, NASA programs, projects, and missions, often
organizes, provides for archiving, and disseminates concerned with subjects having substantial
NASA’s STI. The NASA STI program provides access public interest.
to the NASA Aeronautics and Space Database and its
public interface, the NASA Technical Reports Server, • TECHNICAL TRANSLATION. English-
thus providing one of the largest collections of language translations of foreign scientific and
aeronautical and space science STI in the world. technical material pertinent to NASA’s mission.
Results are published in both non-NASA channels and
by NASA in the NASA STI Report Series, which Specialized services also include creating custom
includes the following report types: thesauri, building customized databases, organizing
and publishing research results.
• TECHNICAL PUBLICATION. Reports of
completed research or a major significant phase For more information about the NASA STI
of research that present the results of NASA program, see the following:
programs and include extensive data or theoretical
analysis. Includes compilations of significant • Access the NASA STI program home page at
scientific and technical data and information http://www.sti.nasa.gov
deemed to be of continuing reference value.
NASA counterpart of peer-reviewed formal • E-mail your question via the Internet to
professional papers but has less stringent help@sti.nasa.gov
limitations on manuscript length and extent of
graphic presentations. • Fax your question to the NASA STI Help Desk
at 301–621–0134
• TECHNICAL MEMORANDUM. Scientific
and technical findings that are preliminary or • Telephone the NASA STI Help Desk at
of specialized interest, e.g., quick release 301–621–0390
reports, working papers, and bibliographies that
contain minimal annotation. Does not contain • Write to:
extensive analysis. NASA Center for AeroSpace Information (CASI)
7115 Standard Drive
• CONTRACTOR REPORT. Scientific and Hanover, MD 21076–1320
technical findings by NASA-sponsored
contractors and grantees.
NASA/TM—2007-215007

Feedback Control Systems Loop Shaping

Design With Practical Considerations
George Kopasakis
Glenn Research Center, Cleveland, Ohio

National Aeronautics and

Space Administration

Glenn Research Center

Cleveland, Ohio 44135

September 2007
 Trade names and trademarks are used in this report for identification
only. Their usage does not constitute an official endorsement,
either expressed or implied, by the National Aeronautics and
Space Administration.

This work was sponsored by the Fundamental Aeronautics Program

at the NASA Glenn Research Center.

Level of Review: This material has been technically reviewed by technical management.

Available from
NASA Center for Aerospace Information National Technical Information Service
7115 Standard Drive 5285 Port Royal Road
Hanover, MD 21076–1320 Springfield, VA 22161

Available electronically at http://gltrs.grc.nasa.gov

 Feedback Control Systems Loop Shaping
Design With Practical Considerations
George Kopasakis
National Aeronautics and Space Administration
Glenn Research Center
Cleveland, Ohio 44135

Abstract
This paper describes loop shaping control design in feedback control systems, primarily from a
practical stand point that considers design specifications. Classical feedback control design theory, for
linear systems where the plant transfer function is known, has been around for a long time. But it’s still a
challenge of how to translate the theory into practical and methodical design techniques that
simultaneously satisfy a variety of performance requirements such as transient response, stability, and
disturbance attenuation while taking into account the capabilities of the plant and its actuation system.
This paper briefly addresses some relevant theory, first in layman’s terms, so that it becomes easily
understood and then it embarks into a practical and systematic design approach incorporating loop
shaping design coupled with lead-lag control compensation design. The emphasis is in generating simple
but rather powerful design techniques that will allow even designers with a layman’s knowledge in
controls to develop effective feedback control designs.

Introduction
Classical feedback control is challenging even to control designers, and translating the theory to
easily understood and encompassing design tools is not an easy task. This is the reason that many control
designers to date still employ ad-hoc control design techniques like normally done with PID control
design. Such ad-hoc control design techniques primarily rely on manipulating the control gains to achieve
some satisfactory time domain response, without much insight on how to achieve good overall
performance within the limitations of the control design relative to the plant.
Being able to relate the hardware designs, such as the actuator speeds, into the controller design
limitations is a practical matter, which can not be overlooked. In aviation systems actuators like
hydraulics, are the means by which control is applied. If the control designer understands how these
actuators influence the control design, better control designs can be achieved. Understanding this relation
between the hardware and control designs also enables the control designer to negotiate the control
requirements or justify the need for better (faster) actuators.
The idea of Loop Shaping in classical feedback control system design is not new. But there is not that
much written about this subject in terms of methodical design techniques that simultaneously satisfy a
variety of control requirements and also offer insight into the limitations of the control design in relation
to the plant. References 1 and 2 discuss loop shaping in terms of sensitivity functions, but they do not
present a methodical control design approach for loop shaping and a clear path of how the control design
is related to the nature of the plant. There is also a considerable amount of work done in H ∞ loop
shaping (ref. 3 is an example of that). But with H ∞ it is difficult to incorporate quantifiable stability
requirements in the design for the computation of the control bounds (singular values). Also, satisfying
the control bounds doesn’t preclude the existence of a better control law that offers better performance,
making it difficult to understand the limitations of the control design relative to the plant at hand and the
requirements.

NASA/TM—2007-215007 1
 Other than ad-hock control design techniques, classical control techniques like Root Locus (ref. 4) is
perhaps the most widely used linear feedback control design method. In root locus feedback control the
poles and zeros can be placed to give a certain time domain response. But with root locus the correlation
between the time and frequency domain responses can be described qualitatively at best and additional
frequency domain analysis would be needed to quantify stability, disturbance rejection, and generally to
understand the limitations of the control design process. On the other hand, a loop shaping design can
accomplish all that and it can still remain in many regards a powerful and intuitive design process. The
loop shaping control technique described here assumes that the plant transfer function is known. But other
than that, the design methodology as described here is generally applicable.
In this paper some control concepts are mentioned occasionally, like lead-lag compensation, stability
margins, closed loop gain (CLG), settling time and so on, without reference. These concepts are standard
classical control concepts that can be found (for more information) in any book on linear control systems
design, like reference 4.
The paper is organized as follows. A background of the theory relative to feedback control and loop
shaping is first discussed in brief; the natural response of the system and the control bandwidth. Next the
control design relative to the plant, more specifically the actuation system speed relative to the control
bandwidth design is discussed. This is followed by the loop shaping design procedure; general guidelines,
how to incorporate control system requirements into the loop shaping design, and how these requirements
relate to design limitations. Lastly, the feedback controller design procedure is outlined relative to the
loop shaping design and contrasted with more traditional designs like PID control, with the use of design
examples.

Loop Shaping Control Design Background

This section discusses frequency domain closed loop control design, given a plant transfer function,
actuator(s) rates, and time and frequency domain control design requirements like transient response,
disturbance attenuation, and stability. The introductory discussion in this section may seem premature.
But the aim is for the reader to keep the material in this section in mind when some background control
concepts are presented later, followed by a more detailed discussion in this area.
A traditional feedback control system as shown in Figure 1, with a plant transfer function Gp(s) and a
controller transfer function Gc(s) has input to output transfer function

C (s) GC ( s )GP ( s )
= (1)
R( s ) 1 + GC ( s )G P ( s )

The CLG of the feedback control system in Figure 1 is

L(s) = Gc(s)Gp(s) (2)

Figure 1.—Feedback Control System Diagram

NASA/TM—2007-215007 2
 In general, the CLG of a feedback control system is L(s)=GC(s)GP(s)H(s), where H(s) is the feedback
transfer function. Loop shaping mainly consists of designing the frequency domain response of L(s) to
satisfy control system requirements and then given the transfer function of the plant Gp(s), calculate or
determine the controller transfer function Gc(s), so that the desirable CLG transfer function L(s) is
realized. The reason for shaping the loop gain L(s), is because the denominator of Equation (1), often
referred to as the characteristic equation, is the equation that determines the stability properties of the
feedback control system, as well as disturbance attenuation and response time characteristics.
A desired shape for CLG, L(s), somewhat independent of the control system specifications is shown
in Figure 2. Such a typical design would have a pole at the origin for high CLG and zero steady-state
error, followed by a zero to maintain relative high gain at the mid-frequency range for disturbance
attenuation, followed by a pole to attenuate the gain at higher frequencies in order not to exceed the
actuator rate(s). Such a design would also have adequate stability margins >= to 90° phase margin and
infinite gain margin since the phase doesn’t cross 180° (more detail on stability margins will be discussed
later). The loop gain displayed below is an example of that.
The feedback control diagram for this system is shown in Figure 3. The forward loop transfer
function starting from the gain k to the transfer function output depicts the desirable CLG transfer
function as opposed to normally separately displayed controller and plant transfer functions.

Figure 2.—Typical CLG Design of a Feedback Control System

Figure 3.—Feedback Control Diagram for the CLG in Equation (2) and Figure 3

NASA/TM—2007-215007 3
 wp1 ( s + wz1) ( s / 20 + 1)
For this control diagram L( s ) = k = 110 and the input to output transfer
wz1 s ( s + wp1) s ( s / 200 + 1)
function in Equation (1) can be rewritten as

C (s) L( s )
= (3)
R( s ) 1 + L( s )

Figure 2 can be reproduced by the following commands in MATLAB®

>> num=110*[1/20 1];

>> den=[1/200 1 0];
>> bode(num,den);
>> grid

Closed Loop Design Procedure Based on Specifications

A feedback controller design procedure would need to incorporate information of the speed of the
actuators such that certain limits placed on actuator rates (i.e., magnitude response over time) will not be
exceeded. Exceeding actuator rates can have negative consequences such as shortening the actuator
lifetime. Also if the controller demands excessive actuator rates, with no limits placed, this can cause the
controller to overdrive and saturate the system. If actuator rate limits are incorporated in the control
design, hitting these limits causes the system to become nonlinear, which can complicate the control
system analysis. To avoid these difficulties, the actuator rates can be incorporated in the control design
right from the start, and incorporation of actuator rate limits could serve as lasts resorts, if for some reason
the controller fails to do its job.
The control system design must also have the ability to attenuate low frequency disturbances. As is
often the case with control systems, disturbances or noise to the system can not be totally avoided. Thus a
good control design also deals with the system disturbances, within the capability of the actuation
hardware. Sometimes attenuation requirements, as all other control requirements, can be explicitly stated
or else they become implicit in the design process. Knowing how to design the control system in a
systematic way, within the capability of the hardware actuation system, can empower the control system
design engineer to ask for more hardware capability in order for the control design to meet system
requirements or negotiate requirements.

Natural Frequency of Closed Loop Systems

Before we embark into a feedback control design process using loop shaping design, some
fundamental concepts need to be introduced first to help facilitate the discussion.
In order to incorporate the actuator rates into the feedback control design, it is important to
understand the natural response or the natural frequency of feedback control systems. The response of a
linear sufficiently damped dynamic system to a unit step input can be approximated with an exponential
response as

c(t ) = (1 − e − ω n t ) (4)

where ωn is the natural frequency of the system.

NASA/TM—2007-215007 4
 Figure 4.—Exponential System Simulation—Approximation to
a Damped Dynamic System Response

Figure 5.—Exponential Response

Figure 4 shows the simulation of Equation (4) and Figure 5 shows the exponential response and the
rate. The rate of the system is ~0.1/1e–4 = 1000, which is the same as ωn in the simulation. For this
approximation to a dynamic system, this rate would be the rate at which the actuation system would also
need to respond to control the system. The simulation shown in Figure 4 was done using SIMULINK®
and the plot was produced with the following command in MATLAB®

>> plot(simout.signals.values',simout1.signals.values')
The question becomes what is the natural frequency of a dynamic system. For first and second order
systems of the type

C (s) ω
=K (5)
R( s) s+ω

ω2
or K (6)
s 2 + 2ςω + ω2

the natural frequency is ω. For a feedback control system (Eq. (3)), however, the natural frequency of the
system is not so obvious. The closed loop system response can be expressed in terms of magnitudes. The
output reaches the magnitude of the reference input, in a step response, when the C ( jω) ≅ R( jω) or

NASA/TM—2007-215007 5
 C ( s)
when the output ratio ≅ 1 . Based on that the system response in Equation (3) approaches unity
R( s )
when

C ( s) L( jω)
≅ ≅1 (7)
R ( s ) 1 + L( jω)

For the transfer function in Equation (7), the natural frequency would be the frequency where the
response breaks from the horizontal asymptote (rolls off 3 dB from the magnitude it starts out - typically
the magnitude starts out at 1 (0 dB). But the question still remains what is the frequency where the
magnitude of Equation (7) rolls off 3 dB in magnitude. As it is often the case in feedback control design,
the CLG L employs an integrator to drive the steady-state error to zero and also to accomplish high
disturbance attenuation at low frequencies. For an integrator in the control design, as s → 0 (equivalently
as t → ∞ ) the magnitude ratio in Equation (7) becomes unity. But in determining the natural frequency of
the closed loop system, we are not interested what happens in the limit sense, after all the modes have
settled. Rather, the interest is to find the point when the magnitude ratio first starts to approximate unity,
starting from s = ∞ (equivalently starting from t = 0 ). By inspection of Equation (7), it can be seen that
the ratio first starts to approach unity when L( jω) ≅ 1 and 1 + L( jω) ≅ 1 . Remember from controls
1
theory that when t equals one time constant, τ c = , the system response reaches 1 − e −1 or 63% of its
ωn
final value. Also, 1 + L( jω) is a vector summation and for instance, when L(jω) = 1〈120° or 60° phase
margin (stability will be discussed later), then 1 + L( jω) = 1 . Therefore, the natural frequency of the
closed loop system (for all practical purposes) is the frequency where the CLG L crosses the 0 dB axis in
frequency (or magnitude of 1). This cross-over frequency is commonly referred to as the bandwidth of the
system.
As an example consider the feedback control system shown in Figure 3. In this system a CLG has
been designed as

wp1 s + wz1 ( s / wz1 + 1)

L( s ) = K =K (8)
wz1 s ( s + wp1) s ( s / wp1 + 1)

This rather simplified CLG does not necessarily imply that either the plant transfer function or the
controller design is trivial. Instead, as it will be discussed later, the shape of the CLG can be chosen to
satisfy system performance criteria. Once this is done, the controller can be calculated (also by knowing
the plant transfer function). The bode plot of the CLG in Equation (8), with K=110, wz1=20 and
wp1=200, is shown in Figure 6 as well as the corresponding closed loop transfer function (CLTF) gain of
Equation (3). It can be seen in this figure that the 3 dB roll-off frequency coincides with the cross-over
frequency of the CLG. This establishes the relation in terms of natural frequencies between the CLTF and
the CLG, which is our main interest here for loop shaping control design. This figure also demonstrates
why it makes sense to shape the CLG instead of the CLTF and that is because unlike the CLG, the CLTF
is inherently flat in most of the frequency range of the control design. Figure 7 shows the response of the
system to unit step input and the initial rate of the system which is approximately 1000, the same as the
cross-over frequency of the CLG in Figure 6.

NASA/TM—2007-215007 6
 Figure 6.—Bode Plot of the Closed Loop Gain L(s)

Figure 7.—Closed Loop System Response to a Unit Step

This simulation in Figure 6 was produced with the following additional commands (i.e., additional to
the commands used for Figure 2) in MATLAB®

>> num2=1100*[1 20];

>> den2=[1 1300 22000];
>> hold on;
>> bode(num2,den2)

The num2 and den2 values were obtained by calculating Equation (3) with the corresponding values
C (s) L( s ) 1100( s + 20)
of L(s), which reduces to = = . Using actuator rate limit spec(s) and
R( s ) 1 + L( s ) S + 1300s + 22000
2

with the understanding of the natural frequency ωn of a closed loop control system discussed above, the
design of the feedback controller can begin. But first a little more discussion is needed to more fully
understand the natural response of a closed loop control system. The natural frequency of the closed loop
control system also depends on the lowest frequency zero of the CLG, L(s). The role that the lowest
frequency zero plays in the response of the system is that the frequency of the zero in rad/sec becomes the
time constant 1 / ω z , which is the time it takes for a pole in the system to reach that lowest frequency zero.

NASA/TM—2007-215007 7
Therefore, this zero also influences the response of the system. Both the cross-over frequency time
constant and the time constant of the lowest frequency zero will influence the overall time response of the
system. The cross-over frequency of the system, which would be normally higher than the frequency of
the lowest frequency zero, will therefore, influence the initial response of the system. The lowest
frequency zero will dictate the final response of the system before the system response settles. If it turns
out that the CLG has no zero below the cross-over frequency, than the system time response will only
depend on the cross-over frequency, However, a zero placed before (actually well before) the cross-over
frequency is a good design practice because it helps to boost the mid-frequency gain for disturbance
rejection, increase the phase margin for stability, and also increase the control bandwidth.
Figure 8 shows a portion of Figure 7 that emphasizes the response of the closed loop system that is
influenced by the natural frequency of the lowest frequency zero. The time constant of this natural
frequency is τ = 1 / wz1 = 0.05 sec. In Figure 8, the response due to the natural frequency of the zero starts
when the output reaches approximately 0.85. Thus in 1 time constant, starting from when the response
reaches 0.85, the output should reach, 0.85 + (1 – 0.85) × 0.63 = 0.944, which is about right. The shape of
the second order response in Figure 7 may not be typical compared to customary, smooth damped
responses as shown in Figure 5. But it serves two purposes; one is to help maintain high mid-frequency
gain (see Figure 6) to attenuate disturbances, as will be discussed later; two it helps the stability of the
system by boosting the phase (full + 90°) before the cross-over; besides the fact that it also serves the
purpose here to separate and illustrate the natural frequencies of closed loop systems. If this system did
not have sufficient phase margin, the response of the system would have been underdamped and the
response due to the lowest frequency zero would have being masked out (i.e., if there was any visible
evidence left it would be further out in time just before the response settles).

Incorporating Actuator Rates in the Feedback Control Design

As was discussed above, the cross-over frequency of the system (normally higher than the frequency
of the lowest frequency zero) determines the initial response of the closed loop system. The system
response rate (magnitude response over time) at the initial stages of the response is approximately equal
1
to τ co = for a unit step input, where τ co is the time constant of the cross-over frequency ωco. For a
ωco
non-unit step response, the initial system response rate would be rm/τco, where rm is the maximum
allowable magnitude of the step input. For the actuator limit not to be exceeded, rm/τco is set equal to the

Figure 8.—Closed Loop System Response to a Unit Step Expanded

to Show the Natural Response Due to Closed Loop Zero

NASA/TM—2007-215007 8
actuator rate limit, arl (arl is magnitude units per sec). Therefore, the cross-over frequency, ωco, can be
computed based on the actuator rate limit and the input step size as ωco = arl / rm. If there is other than one-
to-one correspondence between the actuator control rate and system response rate than the computed
cross-over frequency would need to be adjusted as

ωco = C r arl / rm (9)

where C r = cr ar is the rates ratio of system output response with respect to the actuator input. In case
the rate limiting factor is not the actuator itself, but rather it is the mechanical device that the actuator
controls, then this mechanical device becomes the effective rate for the calculations involving the actuator
rate. It is not necessary to specify the system with only step input responses, it could be a ramp instead; in
case of a ramp, if the ramp rate is faster than the actuator rate the ramp input can be treated like a step for
this analysis; if the ramp rate is slower than the actuator rate, then the ramp input will not be a limiting
factor for calculating the system bandwidth, ωco.
The control system can respond to disturbances (i.e., attenuate disturbances) with frequencies up to
the system bandwidth, ωco. That is why system bandwidth is a commonly used term in control systems. If
no input step response or ramp input characteristics are specified, or if the ramp input is not a limiting
factor compared to actuator rate limit, then the bandwidth of the system can be tailored to disturbance
attenuation requirements. Sometimes control design requirements are not explicitly stated, but in such
cases control requirements become implicit in the design process. In some control system designs, the
primary purpose is to maintain the output at a scheduled reference point and the controller design
emphasis is to attenuate disturbances. In such cases, disturbance attenuation with actuator control rates
could be used to determine the control bandwidth. In classical feedback control systems, where noise or
disturbance is summed at the output, the output-to-noise transfer function is

C (s) 1
= (10)
N ( s) 1 + L( s )

where N is the noise input or disturbance to the system. Comparing this transfer function to Equation (3),
it can be seen that the analysis for determining the natural frequency or the desired bandwidth of the
closed loop system is the same for the system input as it is for the noise. Even if the noise summation
occurs at a different place (i.e., somewhere else than at the systems output), the numerator of Equation
(10) changes, but the analysis is still valid. Based on that, for a step or a ramp type disturbance (i.e.,
knowing the maximum magnitude of the disturbance) the same design considerations would apply for
calculating ωco as those discussed for the input step and input ramp. Figure 9(a) shows a closed loop step
response to a unit step input followed by a unit step disturbance applied at 0.2 sec, for the same system
shown in Figure 3. Both the responses (i.e., the response to the input and the response to the disturbance)
are similar. Figure 9(b) which is an expanded part of Figure 9(a), shows that the control rate (actuation
rate) due to the response of the disturbance is the same as that for the unit step input shown in Figure 7.
Also, it can be seen from Equation (10) and Figure 6 that the control system can attenuate
disturbances to varying degree up to the cross-over frequency, as L( jω) 1 below this frequency. Above
the cross-over frequency L( jω) ≺ 1 the response to noise in Equation (10) becomes unity and no
disturbance attenuation can be expected above this frequency.
For a unit amplitude sinusoidal disturbance the controller can be designed with a bandwidth the same
as that determined based on the unit step input and the actuator rate. Then the system can attenuate unit
magnitude sinusoidal disturbances up to the frequency ωco without exceeding the actuator rate limit. But
if design specs call for sinusoidal disturbance of frequency ωco, other than unit magnitude, then the unit
step input design considerations can be employed again, and the

NASA/TM—2007-215007 9
 Figure 9.—(a) Closed Loop Control Response to a Unit Step Followed by a
Unit Step Disturbance; (b) Expanded view of Disturbance Response

cross-over frequency should be adjusted based on Equation (9). Especially if the sinusoidal disturbance
exceeds unity in order not to exceed the actuation rates. So knowing or anticipating what are the
magnitudes, frequency, and attenuation requirements for disturbances together with the actuator rates, the
design of the control system can proceed. For explicit sinusoidal disturbance attenuation specifications,
the design procedure will be discussed later.
In terms of how much the controller can attenuate disturbances below the cross-over frequency, the
attenuation magnitude is equal to the CLG ⏐L(jω)⏐, at that particular frequency based on Equation (10).
Near the cross-over frequency the attenuation is equal to the magnitude ⏐1 + L(jω)⏐ ≈ 1. And beyond that
there is not attenuation as ⏐1 + L(jω)⏐ ≈ 1.For the feedback control system discussed above, a sinusoidal
disturbance of 50 rad/sec is simulated. According to Figure 6, the magnitude of the CLG is ~16 dB at that
frequency. So we can expect that a disturbance at this frequency will be attenuated by about 1016/20 = 6.3
(the dB magnitude is based on 20log10(x)). Figure 10 shows the disturbance with amplitude of 1 and the
output response due to this disturbance with amplitude of about 0.15, which is what was expected out of
the performance of the system. Next a disturbance is simulated right at the cross-over frequency of 1000
Hz, where we expect to see practically no attenuation. In Figure 11 we see that the attenuation of the
control system at this frequency is instead about 1/0.64 = 1.56. As was discussed before, it can be
considered that a control system can attenuate disturbances up to the cross-over frequency, with
practically no attenuation provided at the cross-over frequency. But what is the exact attenuation at the
cross-over frequency depends on the stability margins of the system at this point. For instance, for this
system the phase of the system at the cross-over frequency is about 77° according to Figure 6. Thus the

NASA/TM—2007-215007 10
 Figure 10.—Disturbance Attenuation of the Control Figure 11.—Disturbance Attenuation of the Control
System at 50 rad/sec System at 1000 rad/sec

magnitude of the denominator in Equation (10) is 1 + L( jω) ≅ 1 + 1 ≺ 103° = 1.55 . This means that at this
frequency, according to Equation (10), any disturbance will be reduced or attenuated by a factor of about
1.55, which agrees with the simulation results in Figure 11.

Control System CLG Shaping

So far the ground work was laid on how to handle control system actuation rates and determine the
limits of the control bandwidth or cross-over frequency. Also, some discussion was carried out as to how
the magnitude of the CLG influences disturbance attenuation at different frequencies. Next attention will
be paid to the CLG design influenced by explicit or implicit control design requirements.
As discussed earlier the CLG can be designed or shaped based on explicit or implicit control system
requirements such as actuation rates, disturbance attenuation, stability, and maybe transient response.
Once the desired shape of the CLG has been derived, the feedback controller can be computed (covered in
the next section) by also taking into account the plant transfer function.
In order to understand how to shape the CLG based on disturbance attenuation or rejection
requirements, we must first understand the limits of the control system design. That is what would be the
achievable disturbance rejection given the control system bandwidth, which has been determined based
on actuation system rates. Let’s say that the actuation system response rate limit and the input or
disturbance step or rate calls for a control system bandwidth not to exceed 1000 rad. Also, let’s say that
additional disturbance requirements call that a disturbance at 100 rad needs to be attenuated by 20 dB or
by a factor of 10. In that case, if we design the mid-frequency gain to be 20 dB and we place a pole at 100

NASA/TM—2007-215007 11
 Figure 12.—Bode Plot of Closed Loop Gain Shaping with 20 dB Gain
at Mid-Frequency and Cross-over (Bandwidth) of 1000 rad/sec

rad, then the CLG will cross the 0 dB axis at 1000 rad (how we go about designing the lower and mid-
frequency gain will be discuss later). Remember, that a pole gives a slope of –20 dB/decade (a zero will
give a slope of +20/decade) starting at the frequency where the pole is placed. This means that a pole
placed at 100 rad where the gain is 20 dB will bring the gain to zero at 1000 rad. Figure 12 shows this
portion of the design with
100
L( s ) = 10 (11)
s + 100
In reality as shown in Figure 12, at the frequency where the pole is placed the gain rolls off by 3 dB, and
for a zero the gain will be up by 3 dB. For this design, this means that for the gain to cross-over at 1000
rad the actual gain at 100 rad will be 17 dB instead of 20.
If we continued with this design and we selected to put a zero somewhere at low frequency (i.e., less
than 100 rad), and a pole at zero, then the phase at cross-over would be expected to be –90° (i.e., –90° –
pole at origin, +90° - zero at low frequency –90° pole at 100 rad/sec ). This will give sufficient phase
margin for stability, as typically greater than 60° will be adequate (somewhat damped response). The
phase margin for stability is defined as M φ = 180 − φ , where φ is the phase of the CLG at the cross-over
frequency.
Now let’s say that instead of 20 dB attenuation at the mid-frequency range, 17 dB at 100 rad/sec with
the same cross-over frequency, the requirement was for 26 dB or a factor of 20 in disturbance attenuation.
We can still cross-over at 1000 rad/sec if an additional pole is placed at 500 rad/sec. The reason is, that a
pole gives a reduction in gain of 20 dB/decade or 6 dB/octave (1000 rad/sec is one octave above 500
rad/sec). Therefore, the pole at 100 rad/sec gives us 20 dB gain reduction at 1000 rad/sec with an
additional 6 dB contribution from the pole at 500 rad/sec, which gives us a total of 26 dB reduction at
1000 rad/sec that meets the requirement. But we need to also check the phase margin with this design.
From before, we had 90° phase margin. Adding the phase contribution of the additional pole at 500 Hz,
we will get 45° contribution at 500 Hz (the phase of poles and zeros starts one decade below the
frequency and ends about one decade above – total 90° shift) and approximately another 14° contribution
one octave away, at 1000 rad/sec. Thus, this design will make the phase margin about 180 – 90 – (45 –
14) = 31°. This design will not satisfy typical stability requirements, and the response will start out
somewhat oscillatory with little margin for error. Therefore, while 20 dB mid-frequency gain (17 dB gain

NASA/TM—2007-215007 12
 Figure 13.—Closed Loop Gain with Cross-over at
1000 rad/sec and 26 dB mid-frequency gain

Figure 14.—Response of the closed Loop System with 1000 rad/sec

cross-over, 26 dB mid-frequency gain (phase margin ~ 31°)

one decade below the cross-over frequency due to the pole roll-off) is reasonable for the control system
design for disturbance attenuation, 26/23 dB exceeds the capabilities of a feedback control system. The
limit of the control design is somewhere between the 20 and less than 26 dB mid-frequency gain,
depending on what are the stability requirements that includes the gain margin. Gain margin is the gain
difference between the cross-over frequency gain (i.e., the 0 dB gain) and when the CLG L( jω) crosses
180°. Figure 13 and Figure 14 show the CLG of such a design and the closed loop response (i.e., with
cross-over of 1000 rad/sec and mid-frequency gain of 26 dB) as

(1 / wz1 + 1)
L( s ) = K (12)
s ( s / wp1 + 1)( s / wp 2 + 1)

For this case, K=400, wz1=20, wp1=100, wp2=500. Note: the response in Figure 14 has significant
overshoot and typically it would not be acceptable. But it serves the tutorial purpose here of
demonstrating the control design limitations.

NASA/TM—2007-215007 13
 The low frequency zero dictates the final time response of the closed loop system as was discussed
earlier. If the plant transfer function has a zero at lower frequency, but that zero doesn’t show up in the
CLG transfer function because a pole has been placed on top of this zero, still that low frequency zero
will dictate the final time response of the system. Sometimes the overall time response of the system is
specified, which dictates the placement of the low frequency zero. Other times it is left up to the control
system designer to design the speed and the settling time with which the system responds, within the
capabilities of the plant. If the overall response of the system is not specified, the lowest frequency zero in
the CLG needs to be placed at a frequency less than a decade below the cross over frequency. This is
done in order to get the full benefit of +90° phase shift to help with stability. Then the zero can be placed
as low in frequency, without compromising the final speed/settling time with which the system needs to
respond, and to maintain a high mid-frequency gain for disturbance rejection.
Of course, the pole at the origin is important for zero steady-state error and also for high CLG at low
frequencies for disturbance rejection. In practice, it is also beneficial to place another pole, as in Equation
(12), shortly before or after the cross-over frequency, which causes the gain to decrease by 40 dB/decade
starting from that point in frequency. The reason is that if any unmodeled plant zero exists in this frequency
range shortly after cross-over, even if it does not compromise system stability (i.e., absolute stability –
defined at singular frequency, at the cross-over) as was discussed earlier, it can compromise conditional
stability; the gain (lingering) near the neighborhood of the cross-over frequency or equivalently near the -1
radius circle on the Nyquist plot (where system stability again depends on the phase).

Loop Shaping Design Procedure: Example

For demonstration of loop shaping design let’s pick a design example with the following control
system design requirements:
Actuation rate limit to a unit step input, for a one-to-one control to output ratio – 1000 per second
Mid-frequency disturbance attenuation of 23 dB/20 dB at 100 rad/sec
Stability phase margin of ≥45°and gain margin of ≥10 dB
One time constant response time≤0.02 sec
Settling time to 2% error of ≤0.1 sec
Notice, an overshoot requirement was not given in this example. Had we also been given an
overshoot requirement, this would be somewhat redundant with the phase margin requirement. That is
because the 45° requirement translates to the magnitude of the CLTF vector representation of
1 / (1 ≺ 0° + 1 ≺ 135°) = 1 / 0.765 = 1.307 or 30.7% overshoot. This calculation can be used as a rough
approximation for overshoot, or an explicit overshoot requirement could be given for completeness.
However, the stability requirement as listed here, alone, would not guarantee stability and we’re not
talking here about system sensitivity. That is because the 45°/10 dB requirement refers to absolute
stability as mentioned before and doesn’t cover conditional stability. More on this topic will be discussed
later in this section.
Based on these requirements we first calculate the cross-over frequency based on Equation (9), ωco =
Crarl/rm with Cr = 1 and arl = 1000 units/sec, which gives us ωco = 1000 rad/sec. Based on the mid-
frequency disturbance attenuation requirement of 23 dB; with mid-frequency CLG design of 23 dB, we
can place a pole at 100 rad/sec which will reduce the gain by 20 dB at 1000 rad/sec. If we put another
pole at 1000 rad/sec, this is expected to reduce the gain by another 3 dB at this frequency (remember a
pole reduces the gain by 3 dB at the frequency where it is placed. With a pole also placed at the origin and
a zero placed somewhere below 100 rad/sec, we would expect to have a phase at cross-over of
φ cr = −90 + 90 − 90 − 45 = −135 , which gives a stability phase margin of M φ = 180 − φ cr = 45° . This
meets the stability phase margin requirement. This design would meet the gain margin requirement as
well (infinite gain margin) since the phase does not cross the 180° axis.

NASA/TM—2007-215007 14
 The next question is at what frequency to place the zero. Remember the lowest frequency zero also
influences the time response. The one time constant response time of 0.02 sec, which happens to be
approximately 63% of the final value of the response can easily be met in this design, since this time
constant, τc, is primarily influenced by the cross-over frequency and τc ≈ 1/ωco = 0.001. So this
requirement does not influence the placement of the closed loop zero. Looking into the settling time
requirement of 2% error within 0.1 sec; a 2% error requirement in controls theory for a dominant second
order response is achieved within 4 time constants. Assuming, that the low frequency zero is well below
the cross-over frequency (as is the case here), so that the two time constants in the response are separated,
and assuming that the low frequency zero starts dominating the response after the response reaches about
80% of its final value, then this 2% error would translate to 0.02/(1–0.8)×100%=10% error criteria on the
lowest frequency zero. Again from controls theory, this translates to about 2 time constants for the lowest
frequency zero. Also, the 80% response due to the cross-over frequency time constant takes about 0.002
sec. Therefore, 1 time constant (τcz) for the low frequency zero based on the settling time specification
would be τcz = [(0.1 – 0.002)/2] ≈ 0.05 sec and the zero frequency would be wz1 = 1/τcz = 20 rad/sec.
Thus, we could place the zero at 25 rad/sec to leave some margin. Of course, the design would need to be
simulated and make any final adjustments if necessary.
Also, as customary in feedback control design, a pole will be placed at the origin for the reasons
discussed before. For the gain, K, of the CLG transfer function, we know that the spec calls for 23 dB
mid-frequency attenuation for disturbance. Therefore, at the zero frequency of 25 rad/sec the gain of the
CLG transfer function needs to be 23 dB as

K
L( jω) = = 23dB (13)
s jω = 25

which gives K = ( 25)10 23 / 20 = 353.13 . In actuality because the zero breaks 3 dB above (as discussed)
before, the gain of the transfer function would be 26 dB at the frequency of the zero. A simulation of
L( s ) = K ( s / 25 + 1) / s is shown in Figure 15.
At this point based on the requirements provided, we have designed the CLG transfer function L(s) as

( s / 25 + 1)
L( s ) = 353.13 (14)
s ( s / 100 + 1)( s / 1000 + 1)

Figure 15.—Bode plot of the Closed Loop Gain Transfer Function, L( s ) = K ( s / 25 + 1) / s

NASA/TM—2007-215007 15
 Figure 16 shows the bode plot simulation of the designed CLG L(s) in Equation (14), which meets the
requirements in terms of mid-frequency gain for disturbance attenuation, cross-over frequency for
actuation rate and stability phase and gain margins, and also very closely conforms with the design
procedure. One thing that remains is to check if the design meets the time requirements for a step
response and the actuation control rate. Figure 17 shows the time response of this design to a unit step
input. As seen the response is somewhat underdamped due to the low phase margin. The system well
meets both the one time constant response time as well as the 2% settling time spec within 0.1 sec. In fact,
as shown in Figure 17, the error is approximately 0.5% at 0.1 sec. We assumed before in the design that
the two natural frequencies in the response are separated or distinct and that the response due to the zero
starts about when the response reaches 80% of its final value. However, that assumption was incorrect
due to the underdamped nature of the cross-over frequency mode which dominates the response beyond

Figure 16.—Designed Closed Loop Gain of equation (14)

Figure 17.—Step Response of the Designed Closed Loop System

NASA/TM—2007-215007 16
80% of its final value. This means that for this system the zero can be moved at a lower frequency and the
system still meets the time response requirements. The maximum system rate (actuation rate) as shown in
Figure 18 is about 500 units/sec. as opposed to 1000 for which the cross-over frequency was designed for,
seemingly due to the additional integration (additional pole compared to Figure 7). As a result, the cross-
over frequency could possibly be moved higher, which would improve the design margins. With the
cross-over frequency selected, designing to increase phase margin and also increase mid-frequency
disturbance attenuation work against each other. Had it not been for the slowing of the response
(actuation) rate in the process as shown in Figure 18, the design specs would have constitute the limits of
what a control design can do given the actuation rate, mid-frequency disturbance attenuation, and any,
even slightly reasonable phase margin. This is why this example was chosen to also demonstrate control
design limitations.
Figure 19 shows an encompassing specification of system stability (i.e., it includes both absolute and
conditional stability), in that it specifies an exclusion zone inside which the trajectory of the CLG is not
allowed. This exclusion zone (offered as an example) is bordered by the lines of 60° phase margin and 6
dB gain margin (equivalent to 0.5 radius circle). Note: the gain margin for the exclusion zone does not

Figure 18.—Step Response to Show System Rate

Figure 19.—Nyquist Plot Showing Exclusion Zone for Stability

Specifications and the Trajectory of the CLG in Equation (14)

NASA/TM—2007-215007 17
have the same interpretation as the gain margin for absolute stability. The plot also shows the trajectory of
Equation (14), which in this case does not satisfy this exclusion zone criterion for stability. For
comparison the 45° phase margin lines are also shown in green color. The Script for generating this
Nyquist plot is covered in Appendix I. Generally speaking, if there are no resonances evident near the
cross-over frequency of the CLG (the requirement can call for no resonance in the CLG), as in Figure 16,
stability should be fine only with the absolute stability specifications.

Loop Shaping Design Procedure Steps

A design procedure for the closed loop shaping would go as follows:

1. Choose the bandwidth of the control system based on hardware actuation speed
2. Based on the mid-frequency disturbance attenuation requirements choose the mid-frequency gain
and appropriately place one or two poles before the cross-over in order to bring the gain to 0 dB
at the cross-over.
3. Check the phase margin at the cross-over; also given lower frequency desired poles and zeros
(like a pole at the origin and a zero at a frequency less than a decade below the cross-over). If
phase margin is not met, readjust the design, like lowering the mid-frequency gain.
4. Compute the lower zero frequency by the settling time requirements of the step response.
5. Calculate the gain of the CLG transfer function based on the mid-frequency gain at the lower
frequency zero.
6. Simulate the bode plot of the CLG and the closed loop system response and make adjustments if
necessary.

Note: This design procedure does not discuss margins. But normally the design will incorporate
cross-over frequency margin for actuator rate uncertainty or control operation uncertainty; Margin in the
phase margin for plant phase uncertainty; Mid-frequency gain margin for disturbance uncertainty; Margin
in the lower frequency zero placement for response settling time; A pole placement shortly above cross-
over frequency if the slope at cross-over is –20 dB/decade to prevent a possibly unmodeled plant zero at
that frequency to have unforeseeable consequences on stability. The phase and gain margins would need
to be rechecked for additional pole placement at this frequency range.

Lead-Lag Compensation—Controller Design Procedure for Loop Shaping

Since we designed the CLG of the feedback control system to meet the system requirements, the next
step is to design the feedback controller that would give the desired CLG by also knowing the plant
transfer function.
As an example let’s pick a plant transfer function as

1 2
GP (s) = (15)
s 2 / ω2A + 2 s / ω A + 1 s 2 / ω2P + 2 s / ω P + 1

Let’s say that the first part of the transfer function is due to the actuator with a frequency of 1000 rad/sec
and the second part is due to the rest of the plant with a frequency of 70 rad/sec. A bode plot of this
transfer function superimposed on the desired CLG designed is shown in Figure 20. The controller needs
to be designed such that the plant together with the controller achieves the desired CLG shown in this
figure. This entails that we appropriately add the controller gain, poles and zeros so that the controller
transfer function together with the plant transfer function is raised in gain to match the desired CLG
transfer function. Hopefully in this process the phase is matched as well.

NASA/TM—2007-215007 18
 Figure 20.—Designed Closed Loop Gain and Plant Transfer Function

Figure 21.—Expanding Figure 20

Figure 21 shows a portion of Figure 20 expanded in the lower frequency. At 0.1 rad/sec we pick a
data point. As shown the gain of the CLG at this point is 71 dB or 3548 while the plant transfer function
gain is 6 dB or 2 in a linear scale. If we add a pole to the controller at the origin, then the gain of the pole
at 0.1 rad/sec would be 1/0.1 = 10 or 20 dB. Adding the gain of the plant at this frequency and subtracting
the result from where we want the CLG to be [71 – (20 + 6)] = 45 dB or 1045/20 = 177.8 for the controller
gain. In another way, the gains at low frequency can be equated as follows:

⎡ K P KC ⎤
⎢ = K CLG ⎥ (16)
⎢⎣ s ⎥⎦ s = j 0.1

where KC and KP and KCLG are the plant, the controller, and the closed loop gains at low frequencies.
Based on Equation (16), KC =177.8. After the DC (low frequency) control gain has been determined, the
relation G P ( s ) GC ( s ) = L( s ) can be used to calculate the transfer function GC (s) as a function of
frequency.
So far we have designed the controller gain and its first pole at the origin. Now the designed CLG,
based on the control design so far, matches the desired CLG at low frequency. These two CLG’s will
continue to match if we place the controller low frequency zero at the desired CLG low frequency zero.
Thus, so far we have the controller design as

s / 25 + 1
Gc ( s) = 177.4 (17)
s

NASA/TM—2007-215007 19
 Figure 22.—Comparison of desired CLG with
Controller CLG Design at this Point

The Bode plot simulation of this design including the gain of the plant (i.e., gain of 2) against the desired
CLG is shown Figure 22. This figure shows that the design matches well so far at the low frequency. Next
we see that the plant has a double pole with a gain of 1 at 70 rad. We would like to get rid of the effect of
this double pole on the CLG because we want to maintain the gain at the mid-frequency region at the
specified level of ~ 23 dB. We can do that by placing a zero just before the double pole frequency and
another zero shortly after. That is a matter of practice, by forcing the poles to split one in each direction
on the root locus. But also placing a double zero at the double pole location, for one thing we may not get
this exactly right, but also we may need to worry about the damping of a double zero. (If a set of double
poles of the plant were underdamped, then a notch filter could be used instead of the two separate zeros.
To accomplish this, a double zero could be used with proper damping to filter out the peaking action of
the double pole. Next we need to place a pole at 100 rad/sec to restore the desired shape of the CLG. In
addition, there is a double pole at 1000 rad/sec, where in the designed CLG we originally had a single
pole at that frequency. Thus, we need to place a zero at 1000 rad/sec to eliminate the effect of one of the
poles. So far if we are counting, we have placed four zeros and 2 poles in the controller design. In order to
make the controller design proper, we need a list two more poles in the controller design. However, we
don’t want these two additional poles to affect the closed loop design below the cross-over frequency. Or
else, the CLG derived from the controller design and the plant will not match the desired CLG based on
the specifications. Therefore, we can place the first pole at 10,000 rad/sec at the minimum, and the other
pole at let’s say at 15,000 rad/sec.
Based on that, the controller design will be

( s / 25 + 1)( s / 60 + 1)( s / 80 + 1)( s / 1000 + 1)

Gc ( s) = 177.4 (18)
s( s / 100 + 1)( s / 10,000 + 1)( s / 15,000 + 1)

Figure 23 shows a comparison of the desired CLG against the CLG based on the controller design. As
seen, the desired and the calculated CLG’s based on the controller design match very well, except slightly
at the crossover where we lost some phase. The high frequency, beyond the cross-over frequency, is not
as important to accurately match in this design since the control system performance is not impacted by
the CLG design much beyond the cross-over frequency. In terms of magnitude, it’s important for the gain
to keep rolling-off after the crossover (preferably by 40 dB) in order to avoid the possibility of any
unmodeled plant zeros near the cross-over from causing stability problems. In this design the phase
margin is a little less than 45°, called by the spec, and the gain margin is about 15 dB. Had we designed

NASA/TM—2007-215007 20
 Figure 23.—Comparison of Desired CLG versus CLG
based on the controller Design

Figure 24.—Designed closed Loop Step Response Figure 25.—Output (0.1 peak) Due to a
Disturbance (1 peak) at 100 rad/sec

some margin in the beginning, it would have helped, but again the way the specs were written, the control
design was expected to be borderline. Again, this was done purposely to show the limitations in the
control design.
A step response of this design is shown in Figure 24, which shows a little more overshoot compared
to Figure 17 as predicted with the slightly reduced phase margin of the design. The system rate (actuation
rate) is approximately 750 units/sec. Figure 25 shows disturbance attenuation of this closed loop system
design for a disturbance of 1 peak amplitude at 100 rad, with 10 times or 20 dB disturbance attenuation,
which matches that of the CLG design.
Some adjustments can be made to boost the phase margin a little and still meet the design
specifications, but at the expense of increased actuation rate. This can be done by shifting the pole at 100
up in frequency, or/and shifting the zero at 1000 lower in frequency. However, doing that in an add hoc
way, without systematically rethinking and strategizing a revised design could be problematic. For
instance, shifting the zero to the left in frequency will boost the phase. But this will cause the slope of the
CLG to fall to zero right before the cross-over, which can cause stability problems if the phase of the
plant is slightly different than what we anticipated at this frequency range. Instead, we can move the

NASA/TM—2007-215007 21
controller pole from 100 to let’s say 115 rad, which by itself will move the cross-over frequency slightly
up, extend a little the mid-frequency disturbance attenuation range which will give us slightly more phase
margin at the cross-over. And instead of moving the 1000 rad zero to the left in frequency, we will add
another zero at somewhat higher than 1000 rad (let’s say at 1300 rad) in order to get an additional boost in
phase before the cross over. Adding another zero in the design necessitates an additional pole to keep the
controller TF proper. Since we would like the CLG to transition, preferably with a –40 dB/decade slope
shortly before (or at least shortly after the cross-over), we can place this pole at 5,000 rad. Had we placed
this pole near the zero at 1300 rad, this will cancel out the phase boost we get from the zero and will do us
no good in terms of the phase margin.
Making these adjustments in the controller design the new controller transfer function is

( s / 25 + 1)( s / 60 + 1)( s / 80 + 1)( s / 1000 + 1)( s / 1300 + 1)

Gc ( s ) = 177.8 (19)
s ( s / 115 + 1)( s / 5000 + 1)( s / 10000 + 1)( s / 15000 + 1)

The feedback control system diagram is shown in Figure 26.

Figure 27 shows the CLG of this control system design. As it can be seen in this figure compared to
the previous design in Figure 23 the system bandwidth has been extended, the mid-frequency disturbance
rejection range has been extended slightly as well, and the phase margin has been improved to about 60°.
The gain margin still remains at about 15 dB. The improvement in the phase margin can be seen in the
system step response in Figure 28 compared to Figure 24, at the expense of the actuation system rate
which is now about 1000 units/sec. If the mid-frequency attenuation gain could be relaxed a little, both
the stability margins as well as the actuation rates of the system could be comfortably designed for with
adequate stability margins.

Figure 26.—Designed Controller Feedback Control System Diagram

Figure 27.—CLG of Redesigned Control System

NASA/TM—2007-215007 22
 Figure 28.—Step Response of Redesigned Controller
Design, Also Showing Actuation Rate

This control design and redesign example in this section demonstrates the powerful and methodical
design techniques for loop shaping coupled with lead-lag compensation in linear feedback control
systems, the quantitative relations between the time and frequency domain design, the limitations of
control designs in terms of the specifications and the plant, and compromises that can be made to improve
the design.

Comparison of Lead-Lag Control Design Versus PID

As was mentioned before, PID control is in reality a subset of lead-lag compensation. One zero and
one pole at the origin is utilized in PI control and a double zero and one pole at the origin is utilized in
PID. In lead-lag compensation, the objective is to compensate using zeros and poles as necessary without
restricting a-priory the structure or order of the controller transfer function.
Let’s say that we wanted or we were restricted to accomplish the control design in the previous
section using PI or PID control. A PI controller can be expressed as

GC(s) = KP + KI /s (20)

where KP and KI are the proportional and integral gains of the controller. Summing the terms into a
common denominator and extracting the control proportional gain and frequencies gives

[ s /( K I / K P ) + 1]
GC ( s ) = K I (21)
s

From Equation (21) we can see that the PI control structure has a transfer function with a proportional
gain KI , a zero at frequency KI /KP rad and a pole at the origin. Therefore, if we were restricted to use a PI
controller, Equation (21), there is no way we can accomplish the design in Equation (19) with only one
zero and one pole and satisfy the requirements. If we were to use a PID control structure as

GC(s) = KP + KI /s + KD s (22)

where KD is the derivative gain. Manipulating Equation (22) similarly to Equation (20), gives

[ s 2 /( K I / K D ) + K P s / K I + 1]
GC ( s ) = K I (23)
s

NASA/TM—2007-215007 23
 From Equation (23), the gain of the PID controller is KI, with a double zero at frequency KI / KD
rad/sec with damping ζ = (1 / 2)( K P /( K I K D ) and a pole at the origin. Again, restricting the control
design to PID, with two zeros and one pole, will not permit us to satisfy the requirements. Let alone that
the PID controller transfer function is not proper or the control action would not be causal.
It is obvious comparing Equations (21) and (23) that using PID control instead of PI gives us a better
chance of matching the desirable CLG in Equation (14), because the PID allows the use of one more zero.
The fact that PID control is not causal can be overcome by the structure of the plant, which allows the
system as a whole to be causal.
Since we’ve learned from the previous sections how to best compensate and arrive at a desirable CLG
that has been derived based on requirements, let’s utilize PID and attempt to maximize it’s effectiveness
in this design example. The first and most important decision is at what frequency to place the double
zero of the PID. First, let’s examine what happens if we placed the double zero much above the first
double pole frequency of the plant, at 70 rad. If we do that, then with the pole at the origin and the double
pole at 70, the CLG will be decreasing with a rate of 60 dB/decade (18 dB/octave) above 70 rad and
below the double zero frequency. If the gain of the CLG at 70 rad was let’s say 20 dB, that means that
about one octave above the 70 rad, at slightly above 140 rad, the CLG will be crossing the 0 dB axis. For
one thing, this would substantially reduce the system bandwidth and disturbance attenuation at the mid-
frequency range, but also the phase of the CLG at cross-over, from the phase contribution of the poles at
the origin and at 70 rad, will be depleted and the system would be unstable.
Instead, if we were to place the zeros much below the 70 rad frequency, then the gain of the CLG will
be increasing at a rate of 20 dB/decade above the double zero frequency and decreasing at a rate of 20
dB/decade above the double pole frequency. Of course, we can’t be asking the gain of the CLG to be
much above 20 dB at 70 rad, or the cross-over will be approaching or exceed 1000 rad. Also, at this
frequency range, again, all the phase margin will be depleted due to the double pole phase contribution of
the plant at 1000 rad. So if the double zero is placed much below 70 rad, we will have a situation where at
this low frequency the gain of the CLG reaches a minimum and the system will not have good disturbance
attenuation in that frequency range. Therefore, the best design is realized when the double zero is placed
near or at the plant double pole, at 70 rad.
So let’s place the double zero at the double pole frequency, at 70 rad, with a 20 dB gain at this
frequency. We could go with 23 dB instead, but it seems that we would be increasing the frequency of the
cross-over and we will be running out of phase margin as mentioned before. Therefore, using Equation
(16), K C = ( K CLG / K P ) s which gives KC = 350. In order not to significantly vary the CLG at this
s = j 70
frequency we can choose the damping of the double zero to be the same as the damping of the double
pole ( ζ = 1 ) at 70 rad. Plugging these values in Equation (23) for the controller gain, the frequency and
the damping, the PID controller gains can be calculated (KP = 10, KI =350, KD = 0.0714). Now we can
simulate the CLG to examine the design. The CLG of this design is shown in Figure 29. From this design
it can be seen that the phase margin is only about 30°, with reduced bandwidth (~600 rad) and reduced
mid-frequency disturbance attenuation.
To be fair in terms of comparing the PID and the previous lead-lag compensation designs, we would
need to make the phase margin of the PID design to be approximately 60°. To accomplish this we need to
reduce the controller gain and leave the zeros at 70 rad, which we’ve determine that the PID design would
be more effective for this plant. Modifying the design accordantly, by reducing the controller gain (KI =
140), the new CLG resulting from the controller redesign is shown in Figure 30. The new PID controller
gains can be computed as before, KP = 4, KI = 140, KD = 0.0286. In this PID design the phase margin is the
same as the controller design based on lead-lag compensation, but we can only go as high as about 300
rad bandwidth (instead of about 1200 rad). As a result the mid-frequency disturbance attenuation has been
considerably reduced as seen in Figure 30.

NASA/TM—2007-215007 24
 Figure 29.—CLG With PID Design (~30° phase margin)

Figure 30.—CLG with PID Design (~60° phase margin)

Therefore, if we restrict ourselves to a PID design, we would not be able to meet the disturbance
attenuation requirement and still maintain a reasonable phase margin. Had the first plant double pole been
at a lower frequency, we would be forced to accept even lower mid-frequency disturbance attenuation
with a PID design. A step response of this PID design is shown in Figure 31. Granted, the response in
Figure 31 looks good compared to the response of the control design in Figure 28, but there is more to a
controller design than just the time response.

NASA/TM—2007-215007 25
 Figure 31.—Step Response of PID Design of 60° Phase Margin

Continuous Versus Discrete Domain Control Design

A large body of work is published describing discrete control design, and this paper will not attempt
to expand in this area. Rather some guidelines will be given here how to turn the continuous domain
design described in this paper into a discrete design for real time control applications. If the sampling time
of the control processor is about ten times faster than the highest frequency in the control loop (i.e., the
frequency of the highest pole or zero), the controller can be discretized (i.e., converted to difference
equations) without significantly compromising the control design, especially the stability margins of the
system. For instance, if the highest frequency time constant is only 1/7th the sampling time, the design
could still be fine, especially if the highest frequency pole or zero doesn’t significantly contribute to the
phase of the CLG at the cross-over frequency. However, it is always prudent to check the discrete
simulation. The conversion from continuous to discrete domain would have the most significant impact
on the phase margin. If there is significant compromise of the phase margin, this will show up as an
increase in the overshoot of the time response, compared to the overshoot of the continuous response. The
comparison of these overshoots can be used to estimate any compromise in the phase margin. If it turns
out that these guidelines are not met and the phase margin is significantly compromised, the answer is to
take the existing continuous control design and shift it down in frequency; estimate a new cross-over
frequency and come up with a new CLG design that maintains as much as possible stability margins and
disturbance rejection, but at lower frequencies.
When the design needs to be compromised because of limitations in the updating time of the real time
processor, then the time constant becomes the limiting factor of the control bandwidth, and not the
actuator speed. If the updating time is known ahead of time, than it can be estimated which of the two is
limiting the control bandwidth, and the proper cross-over frequency can be estimated in the beginning of
the control design process.

Conclusion
This paper demonstrated a methodical linear feedback control design process, namely loop shaping
with lead-lag compensation control design. This design process takes into account all the pertinent control
system design requirements, whether such requirements are explicitly stated or become implicit in the
design. In this design approach both the time domain response and the frequency domain are taken into
account. The design methodology starts with shaping the loop gain of the closed loop system to meet the

NASA/TM—2007-215007 26
control design requirements, and then given the plant transfer function, design the controller to match the
desired closed loop gain. This loop shaping control design which also employs lead-lag compensation, is
contrasted with examples and compared with traditional proportional integral derivative control design,
which shows the superiority of the approach. This loop shaping methodology takes into account the plant
design, more specifically the actuator speeds. Utilizing this approach can become even more beneficial in
linear system control designs where disturbance attenuation is an important driver in the control design
process, as is the case with aerospace type control systems.

NASA/TM—2007-215007 27
 Appendix I
This appendix lists the code for the script that produces the Nyquist plot design in the section that
covers the Loop Shaping Design Example.

num3=353.13*[1/25 1];
den31=[1/100 1 0];
den32=[1/1000 1];
den3=conv(den31,den32);
nyquist(num3,den3) %plot the Nyquist

% plot the circle of 0.5 radius

I=1;
xx=-0.5;
while xx <= 0.5
x2(I) = xx;
y2(I) = x2(I)^2;
xx = xx +0.001;
I=I+1;
end
x2(1001)=0.5;
y2=sqrt(0.25-y2); % circle y=sqrt((0.5)^2-x^2)
y2(1001)=0;
hold on;plot(x2,y2,'r')
hold on;plot(x2,-y2,'r')

% +/- 60 degree lines from negative real axis

x=-1:0.1:0;
y=1.732*x; %equation of straight line of 60 degree angle
hold on; plot(x,y,'r')
hold on; plot(x,-y,'r')
y=1.0*x; %equation of straight line of 45 degree angle
hold on; plot(x,y,'g--')
hold on; plot(x,-y,'g--')

NASA/TM—2007-215007 29
 References
1. Smith S.C.; Messner W.; “Loop Shaping with Closed-Loop Magnitude Contours on the Bode Plot”
Proceeding of the American Control Conference, Anchorage, AK, May 8-12, 2002.
2. Balley F.N.; Hul C.H.; “CACSD Tools for Loop Gain-Phase Shaping Design of SISO Robust
Controllers,” IEEE Controls Journal, 1989.
3. McFarlane D.; Glover K.; “An H ∞ Design Procedure Using Robust Stabilization of Normalized
Coprime Factors, Loop Shaping Design Procedure Using Synthesis,” IEEE Proceedings of the 27th
Conference on Decision and Control, Austin, TX, 1998.
4. D’Azzo J.J.; Houpis H.C.; “Linear Control System Analysis and Design,” McGraw-Hills, Inc, 1975.

NASA/TM—2007-215007 30
 Form Approved
REPORT DOCUMENTATION PAGE
OMB No. 0704-0188
The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the
data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this
burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302.
Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB
control number.
PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.
1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To)
01-09-2007 Technical Memorandum
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Feedback Control Systems Loop Shaping Design With Practical Considerations
5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

Kopasakis, George
5e. TASK NUMBER

5f. WORK UNIT NUMBER

WBS 984754.02.07.03.20.02
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
National Aeronautics and Space Administration REPORT NUMBER
John H. Glenn Research Center at Lewis Field E-16201
Cleveland, Ohio 44135-3191

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORS

National Aeronautics and Space Administration ACRONYM(S)
Washington, DC 20546-0001 NASA
11. SPONSORING/MONITORING
REPORT NUMBER
NASA/TM-2007-215007
12. DISTRIBUTION/AVAILABILITY STATEMENT
Unclassified-Unlimited
Subject Categories: 06 and 08
Available electronically at http://gltrs.grc.nasa.gov
This publication is available from the NASA Center for AeroSpace Information, 301-621-0390

13. SUPPLEMENTARY NOTES

14. ABSTRACT
This paper describes loop shaping control design in feedback control systems, primarily from a practical stand point that considers design
specifications. Classical feedback control design theory, for linear systems where the plant transfer function is known, has been around for a
long time. But it’s still a challenge of how to translate the theory into practical and methodical design techniques that simultaneously satisfy
a variety of performance requirements such as transient response, stability, and disturbance attenuation while taking into account the
capabilities of the plant and its actuation system. This paper briefly addresses some relevant theory, first in layman’s terms, so that it
becomes easily understood and then it embarks into a practical and systematic design approach incorporating loop shaping design coupled
with lead-lag control compensation design. The emphasis is in generating simple but rather powerful design techniques that will allow even
designers with a layman’s knowledge in controls to develop effective feedback control designs.
15. SUBJECT TERMS
Loop shaping control design; Loop shaping; Feedback control design; Control design; Disturbance rejection; Actuation rates control
design
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF RESPONSIBLE PERSON
ABSTRACT OF STI Help Desk (email:help@sti.nasa.gov)
a. REPORT b. ABSTRACT c. THIS PAGES 19b. TELEPHONE NUMBER (include area code)
U U PAGE UU 35 301-621-0390
U
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std. Z39-18