Getting ideas across takes time ...

! 1990 submission: “The L2 polynomial spline pyramid:

a discrete representation of continuous signals
in scale space”
Splines: on scale, differential ! Journal: IEEE Trans. Pattern Analysis and Mach. Intel.
operators and fast algorithms

Michael Unser
Biomedical Imaging Group
EPFL, Lausanne
Switzerland
Give in when necessary .... but persevere

Plenary talk, Scale Space 2005, Hofgeismar, April, 2005 2

At the beginning there was a continuum.

OUTLINE MOTIVATION Man made it discrete !

! Introduction p
Continuous domain:! L2 (R ) Discrete domain: l 2 (Z p )
p
! The basic atoms: B-splines f (x), x " R f (k), k " Z p

! Spline-based signal processing • real world

! objects • measurements
!
• signals, images • algorithms
! Interpolation ! • sensor input
! • signal and image processing

! Fast multi-scale algorithms ! Sampling and signal acquisition

! Applications ! Continuous/discrete algorithm design
! Edge detection, PDEs, image registration, …
! Splines and wavelet theory ! Multi-scale approaches
! Scale space
! Image pyramids, wavelets
! Coarse-to-fine and multigrid algorithms

3 4
Splines: a unifying framework Splines: definition
Splines: definition

Definition: A function s(x) is a polynomial spline of degree n with knots

Linking the discrete and the continuous …..
4 1 · · · < xk < xk+1 < · · · iff it satisfies the following two properties:
0.8
3

2
0.6
Piecewise polynomial:
0.4
1
0.2 s(x) is a polynomial of degree n within each interval [xk , xk+1 );
1 2 3 4 5 6 7 2 4 6 8
Splines Higher-order continuity:
s(x), s(1) (x), · · · , s(n−1) (x) are continuous at the knots xk .
4

3
Multiresolution Wavelets
" Effective degrees of freedom per segment: 2

n+1 !" n = 1 1
(polynomial coefficients)# (constraints)
1
1 2 3 4 5 6 7

" Cardinal splines = unit spacing and infinite number of knots

The right framework for signal processing

5 6

THE BASIC ATOMS: B-SPLINES PolynomialB-spline

B-splines
basis functions

Causal B-spline of degree n 1

" B-spline of degree n 
B-spline basis functions  1, x ∈ [0, 1)
! Polynomial B-splines n 0 0 B-spline
β+ (x) = β+ ∗ β+ ∗ · · · ∗ β+ (x) 0 basis functions
β+ (x) =
0
!
Causal B-spline "#of degree $ n  0, otherwise.
Causal B-spline (n +of1)degreeB-splinenbasis functions 
! B-spline representation times
n

 1, x ∈ [0, 1)
+ (x) = 1, x ∈ [0, 1) 2
Causal B-spline of degree
nβ+ (x) =" ∗ · · · ∗0β+ (x),
Key n
properties 0 0 0 0
0β+ ∗0β+ … 0β
β+ (x) = β+ ! ∗ β+ ∗ "# · · · ∗ β"+ (x), β+ (x) =  1 3 4 5
1)0, otherwise.
$  1, x otherwise.
! Differential properties Compact !
support:
β+ (x) = β+ (n
n 0
∗ β+
!(n + 1)
0"#
shortest
+"#∗1) 0 $
polynomial
∗ β+ (x)
· · · times spline
β+ (x) =
0 of degree ∈n[0,
0,
 0, otherwise.
Positivity times $
" Key
Keyproperties
properties (n + 1) times
! Dilation properties Key properties
Piecewise polynomial
Key properties
Compact support: shortest polynomial spline of degree n
Smoothness:
CompactCompact Hölder
support:
support: continuous
shortest
shortest of spline
polynomial
polynomial order ofndegree
spline of degree
n n
Positivity
Positivity
! Generalization: fractional B-splines Positivity
Symmetric
Piecewise B-spline
Piecewise polynomial
polynomial
of degree n
( Höldern+1
Piecewise polynomial )
β (x) = β+ x + 2 continuous
n Smoothness:n
Smoothness: H continuous
ölder of norder n
of order
1
Smoothness: Hölder continuous of order n
! Gaussian-like windows Symmetric B-spline of degree n
Symmetric B-spline
( )of degree n
" Symmetric
Symmetric B-splines
β n (x) =B-spline
β+n n+1 degree n
(x + 2of )
β (x) =nβ(+ x +
n n )
n+1 1
β n (x) = β+ x + n+1 2
2
1

1
7 1 !2 !1 1 2 8
 V = s(x) : D {s(x)} = a[k]δ(x − k) ∩ L2
k∈Z

Derivative operator: D= d
dx

Theorem: Every cardinal spline, s(x) ∈ V , has a unique and stable

representation in terms of its B-spline expansion Differential operators
"
B-spline
s(x) = representation
c[k] β (x − k) n
+
Spline-related differential operators
k∈Z
d F
Theorem (Schoenberg, 1946) B-spline representation D{·} = ←→ jω
!# Continuous operators
Differential operators !" Discretedxoperators
Every cardinal polynomial spline, s(x), has a unique and stable representation in F
! # Derivatives D {·}
Finite
m
←→
differences (jω) m
terms of its B-spline expansion "
" V = s(x) 1: D n+1
{s(x)} = a[k]δ(x − k) ∩ L2
s(x) = n
c[k] β+ (x − k) Basis functions d F F
0.8 k∈Z D{·} = ←→ jω ∆+ {·} ←→ 1 − e−jω
k∈Z dx
Derivative operator: D=0.6 d
F ∆m
F
(1 − e−jω )m
0.4
dx
Dm {·} ←→ (jω)m + {·} ←→
discrete signal
analog signal Theorem: 8 Every cardinal
(B-spline coefficients) 0.2spline, s(x) ∈ V , has a unique and stable Green function of Dn+1
representation in terms of its B-spline expansion F xn+
∆+ {·} = jω + O(|ω| )
←→ 1−e −jω D {ρ(x)} = δ(x) =⇒ ρ(x) = D
2 n+1
{δ(x)} = −(n+1)
" 2 4 6 8
Green function of Dn+1 n!
4 Cubic spline (n=3) s(x) = n
c[k] β+ (x − k) F Integrators (impulse response of (n + 1)-fold integ
n+1 ∆+ {·}
m
←→ (1 − e )
−jω m
x n
3 k∈Z GreenGreen
function of D of Dn+1
function Dn+1 {ρ(x)} = δ(x) =⇒ One-sided
One-sidedρ(x)
power
power D−(n+1) {δ(x)} = +
=function:
function
 n!
2 xn+ xn  xn , x ≥ 0
D n+1
{ρ(x)} = δ(x)= δ(x)
Dn+1 {ρ(x)} =⇒ =⇒ ρ(x) = D = D−(n+1)
ρ(x)
−(n+1)
=
{δ(x)}{δ(x)} = + xn+ = (impulse response of (n + 1)-fold integrator)
1 n! n!  0, x<0
of Dn+1
Green function(impulse response of (n + 1)-fold integrator)
1 2 3 4 5 6 7
(impulse response of (n + 1)-fold integrator)
xn
One-sided power {ρ(x)} = δ(x) =⇒ ρ(x) = D−(n+1) {δ(x)} = +
Dn+1function 2
One-sided
 power function n!
In modern terminology: {β+n
(x − k)}k∈Z forms a Riesz basis.
 xn (impulse response of (n + 1)-fold integrator)
x+9 = xn =
n
, x
 xn≥, 0x ≥ 0 3
10
+ 0,  x < 0
8 0, x<0 2

B-splines: differential interpretation B-splines: differential properties

4
4

0 function: x+ = D {δ(x)}
0 −1
!" Construction of the B-spline of degreeStep !# Fourier domain formula
Discrete derivatives
x0+ = D−1 {δ(x)}
0
β+ (x) = x0+ − (x − 1)0+ = ∆1+ Fourier
x0+ (1 − e−jω )n+1
Step function: domain formula β̂+ n
(ω) =
(jω) n+1
∆n+1 xn+
β+ 0
(x) = x0+ − (x − 1)0+ = ∆1+ D−1 {δ(x)} β+n
(x) = ∆n+1
+ D
−(n+1)
{δ(x)} = + (1 − e−jω )n+1
nExact derivatives
Linkingn!continuous and discreteFourierdifferential operatorsβ̂+ (ω) =
domain formula Step function: x0+ = D−1 {δ(x)}
x0+ = D−1 {δ(x)} (Step function) ! "n+1 (1 − (jω) n+1n+1
e−jω )
∆ n+1 n
x+ Fourier mdomain formula
n
β̂+ (ω) =
β+ n
0(x) = ∆+ D
n+1 −(n+1)
{δ(x)} = + 1 − e−jω (1∀f S " ,) ∆m
− e∈−jω n+1
f (x) = β!+
m−1
∗ D f
Link between
# Linking (x) “discrete” and exact (jω)derivatives
n+1 β+0
(x) = x0+ − (x − 1)0+ = ∆1+ x0+
β+ (x) = ∆1+ D−1 {δ(x)} = x0+ − (x −
!# Generalization n!1)+
0
β̂+1(ω) =
n
= +
continuous and discrete differential operators
! " jω (jω)n+1 ∆n+
−jω n+1 β+n
(x) = ∆n+1
+ D
−(n+1)
{δ(x)} = +
1 − e (1 − ∆
e )
n+1 n+1
−jω
xn B-spline differentiation formula
Linking "continuous
∆ m and
(x) = discrete
m−1
D differential
m
(x) operators
+ ==
0 −1 ∀f ∈ S , f β ∗ f n
xn(ω) D ∆n+1{δ(x)} (Step function) ∀f ∈ S , ∆+ f (x) = β+ ∗ D f (x)
" m−1
+ (x) = = = +n+1 +
n m m
+ D
β̂β+ −(n+1) + +
{δ(x)} ! "n+1
jω (jω) n! D β+ (x) = ∆+ β+ (x) Fourier , ∆formula m−1 (1 − )
−jω n+1 1 − e−jω (1 − e−
+ f (x) =β̂+ (ω) =∗ D f (x)
m n m n−m " m m e β̂+ (ω) =
n
=
β+ 0
(x) =!∆1+ D−1 {δ(x)} = x0+ − (x − 1)0+ ∀f ∈ Sdomain β
n+
(jω)
" m−1 Dm β+n
(x) = ∆m +3β+ 4 (x)
n−m
B-spline differentiation jω
∀fn ∈ S " , ∆ 1− m e−jω n+1
+ f (x) = β+ ∗(1D−m e−jω )n+1
f (x) 1 2 5 ! formula (jω)n+1
"n+1−m 1
(x)
1
2
β̂+ (ω) = = ∆ n+1 n
x ! " B-spline
B-spline differentiation
differentiation − e−jω
formulaformula β +
!# Fourier n =domain
mn (x) ∆∆ jωDn−m formula (jω)+n+1 + m n
D =
(jω) β̂+ Linking
(ω) m
β+(1
n
(x) − =
e ∆+)and
−jω m m n−m
(x)
β+· discrete
(x){δ(x)} =
n+1 −(n+1) Sketch of proof: continuous differential operators
Dβ + β+ (x) = ++β
m
+ n! jω
Discrete operator (finite difference)
D ∈βS+,(x) ∆ =+∆ ! "n+1−m
+ β+ (x)∗ D f (x)
m n" m n−m
∀f ∈ S , !∆+ f (x)
" m
="β m−1
∗ D f (x)
m ∀f m
f (x)
+ β=
m−1 m
1 − e−jω
+
n+1 1 2 3
1 − e−jω (1 − e−jω )n+1 Sketch of proof: (jω)m β̂+n
(ω) = (1 − e−jω )m · ! "
β̂ n
(ω) =
D +β+ (x) = ∆+jω
m n m
β+ (x)
n−m = B-spline differentiation formula nSpline degree reduction 1−ejω−jω n+1−m
(jω)n+1 Sketch
finite of proof:
difference (jω)m β̂+
operator (ω) = (1 − e−jω )m ·
4
D m n
β (x) = ∆ m n−m
β (x) jω!1 4

∀f ∈ S " , ∆m + f (x) = β+
m−1
∗ Dm f (x) Differentiation operator + + +
! " n+1−m
1 − e−jω
Dm β+
n
(x) = ∆m n−m
(x) 4 5
Sketch of proof: (jω)m β̂+
n
(ω) = (1 − e−jω )m ·
+ β+ 11 jω 12

5
4
5

5
 B-splines: dilation properties Dyadic case: wavelets
m-scale relation Dyadic case: wavelets
m-scale relation m-scale relation
Dilation by a factor m
"m−1 #n+1
2
Dilation by a factor ofDyadic case: wavelets
Dilation by a factor m Dilation by a factor m !
! 1 ! β+ (x/2)by=a factor
h2 [k]β
of 2+ (x − k)
! n n n n
β n
(x/m) = h [k]β n
(x − k) H n
(z) = z −k "m−1 #n+1 Dilation
+ (x/m) = m [k]β+ (x − k) with !
β+n
hmn n with m
mnn 1 ! !
k∈Z
k∈Z
k∈Z β n
+ (x/m) = h n
m [k]β (x − k)
k=0 with H n
m (z) = z −k
β+ (x/2) =
n
h2 [k]β+
n n
(x − k)
"m−1 #n+1 mn Binomial filter
! k∈Z k=0
Piecewise 1 case (n = 0)
k∈Z
" # n+1 " #
n
Hm (z) = nconstant z −k Binomial 1 + z −1
n+1
1 ! n+1
m Piecewise constant case (n = 0) H2n (z) =filter
2 = n z −k
H10 (z)
1 =
m 1 1 1+ k=0
z1−1 + · · · z −(m−1) (Moving sum filter) " 2 −1 #n+1 2 n+1 "
! n+1 k #
1+z 1 k=0
Piecewise constant case (nH= 0 0)
m (z) = 1 + z −1 + · · · z −(m−1) (Moving sum filter) H2n (z) = 2 = n z −k
Dyadic case (m = 2) 2 2 k
k=0
Hm (z) = 1 $
0
+z +−1
·%
· ·n+1
z −(m−1)
n+1 $ sum filter)
(Moving %
1 + z −1 1 !
Dyadic casen(m+ 1= 2−k ) Example: piecewise linear splines
H2n (z) = 2 = n $ k −1 z % (Binomial filter) $ %
Applications: fast2spline-based2 algorithms n+1 n+1
n k=0 1+z 1 ! n+1
Zooming H (z) = 2 = z −k (Binomial filter)
2
2 2n k 2
k=0
Smoothing 6

Multi-scale processing 1 1
5
Wavelet transform 6
5

5 13 14

Generalization: fractional B-splines Gaussian-like windows

Fractional B-splines
Fractional B-splines
Theorem: The (fractional) B-splines converge (in Lp -norm) to a Gaussian as the
Fractional B-splines F 1 − e−jω
0
β+ (x) = ∆+ x0+ ←→ F 1 − e−jω degree goes to infinity:
β 0 (x) = ∆ x0+F jω ←→
β 0 (x)+= ∆+ x0 + ←→
1 − e−jω
+ + jω # $
M M jω ! " 1 −(x − xα )2
lim α
β+ (x) = √ exp
! "α+1 α→∞ 2π · σα 2σα2
∆α+1 xα F α 1 − e−jω
! "α+1 %
α
β+ (x) = + α + ∆α+1 ←→ x 1 − e−jω! "α+1
1)= + ∆++ x←→
α+1
Γ(αβ++(x) αF
+ jω F 1 − e−jω with σα = α+1
α
β+ (x)Γ(α
= + 1) ←→ jω 12
Γ(α + 1) jω
 
 xα , x≥ 0α 
x , x≥0
+ =
α
One-sided power powerxfunction:
function:
One-sided
One-sidedpower x =
function:α  xα , x≥0
 0, + x<α 0,0
One-sided power function: x = x<0 +
Polynomial B-splines: α = n (integer)
 0, x<0
Properties Properties Compact support: [0, n + 1]
& α ' &β α (x − k)'
!" Properties is a valid Riez basis for α < − 12
β+ (x − k) Properties Riez basis for α < − 12
+ is a valid k∈Z
k∈Z
Fast convolution algorithms: recursive or multi-scale
&
Convolution '
α property:
α
α
β 1α
∗β α +α
2
=β 1 2

β α (x
Convolution property: 1
β+− β+2
∗k) =+βisα1a+α
+2 +
valid Riesz basis for α < − 12
+ k∈Z +
α1 α2
Convolution property: β+ ∗ β6 + α1 +α2 +1
= β+
(Unser & Blu, SIAM Rev, 2000)
6
14

7 15 16
SPLINE-BASED SIGNAL PROCESSING Spline fitting: overview
Spline fitting
! Spline fitting
s(x) = c[k]β n (x − k) !
!# B-spline representation: s(x) = c[k]β (x − k)
n
! Spline fitting: overview k∈Z
k∈Z
Spline fitting Spline
Splinefitting
fitting
Goal: Determine
! c[k] such that
! ! s(x)
Goal: is a ”good”
Determine representation
c[k] such that s(x) isof our signal
a ”good” representation of our signal
! B-spline interpolation s(x) = s(x)=(x
c[k]β
s(x) = − k)n
c[k]βnn(x(x−−k)k)
c[k]β
Discrete input f [k] c[k] f [k] that s(x)|
Discrete inputsuch x=k = fsuch
c[k] [k] that s(x)|x=k = f [k]
k∈Z k∈Z
k∈Z
! Fast multi-scale algorithms !# Interpolation (exact, reversible)
Goal: input
Analog (x)
Determine c[k]
fGoal:
Goal: c[k]
such that
Determine
Determine s(x)
Analog
c[k] input
c[k]such
is
such f (x)
a that s(x)
”good”
such
that s(x)isisamin c[k]
"frepresentation
representation
that a”good”
”good” −our
of such that
s"signal
representation
L2
min
ofofour
2
"f − s"L2
oursignal
signal
2
s∈Va s∈V a

[k] input
fDiscrete
Discrete inputDiscrete inputf f[k]
c[k] [k] Spline fitting
c[k]
c[k]
such s(x)|
Spline
that =
fitting
such
suchthat f [k]
Noisy discrete input f [k] = s(k) + n[k]
s(x)|
that s(x)| ==f [k]
f [k]
! Applications ! input f !
Noisy discrete [k] = s(k) +"n[k]
Interpolation
x=k
such that
2# +∞
x=k
x=k such that
$
s(x) "
Analog (x)
=input fs(x)
c[k]β=(x
Analogn
Analog −
input fk)
input f(x)
c[k]β(x)n (x −
c[k]
algorithm #! c[k]
k) min
c[k]that Spline
such such
suchthat$
s"min
"ffitting
−that Lmin
22
"f"f−−s"s"
LL
! minm +∞
|f [k]m
2 22
2 +λ
s∈V 2 s∈V
s∈V m 2
! 2s∈W − s(k)|
a |D s(x)| dx
aa
min
Noisy
k∈Z
[k]discrete
|finput
s(x)
m discreteNoisy
Noisyf−
k∈Z
s(k)|
[k]
= = input
discretes(k)+ nλ
inputf+
c[k]β 2
[k]
f [k]
(x
n[k]=
−=
k∈Zs(k)|D++such
s(k)
k)
s(x)|
n[k]
n[k]that
dx such−∞
suchthat
that
s∈W
Goal:
2
"k∈Z Goal:c[k]
Determine Determine
""# +∞
s(x)
such thatc[k] is−∞
such athat
”good”
s(x)representation
is a ”good” of our signal
$ representation
$$ of our signal
!# ! ! k∈Z
Spline approximation
!2(at scale a) ##+∞
+∞
m=s(x)|[k]
min inputDiscrete
Discrete [k] −Determine
|ffmin
min
Goal: s(k)||f
input |f+[k]
[k]
22 m
[k]
fc[k] λ 2
−−s(k)|
c[k] ++that
such
|D
c[k]
s(k)|
such that λ s(x)|
λs(x)|
s(x) dx
a|D
|D
issuch
m
x=k s(x)|
s(x)|
that
”good” dxx=k = f [k]
frepresentation
22
dx of our signal
s∈W2m s∈W
s∈Wmm
22 −∞ −∞
−∞
k∈Z
f (x)
Analog input Analog
k∈Z
k∈Z
(x)
f c[k]
f [k] such that"f,min
= "f −
− k)#
s"L2 2
x=k = f [k]
input
Discrete input c[k]c[k] ϕ̃(·/a
s∈Va such that s(x)|
such that
Sampling
min $f −algorithm
s$2L2 13

a f [k] = fs(k)
(x) + n[k] c[k] such that such that min "f − s"L2
2
Noisy discretes∈V
input
Analog input
13
" # +∞ $ a s∈V
!Noisy discrete input f [k] = s(k) + n[k] such that
min Noisy
|f [k] −discrete
" s(k)| + λ
2 input f [k] = s(k)
m + n[k]
2 such $that
17 13 |D s(x)|
# +∞13dx
s∈W2m "
! −∞ #
13 $
k∈Z
minm ! |f [k] − s(k)| + 2 +∞ m
|D s(x)| 2
2 λ dx
s∈W min
2 m |f [k] − s(k)| + λ−∞ m
|D s(x)| dx2
s∈W2 k∈Z −∞
k∈Z

Spline fitting
! 13
s(x) = c[k]β (x − k)
n 13

Spline fitting (Cont’d)

k∈Z

Goal: Determine c[k] such that s(x) is a ”good” representation of our signal
Spline fitting B-spline interpolation 13

!
Spline fitting
! s(x) = c[k]β n (x − k) B-spline interpolation 4/6
!# B-spline
Discrete input f [k]
representation: s(x) =
c[k] such(x
c[k]β n
k∈Z x=k = f [k]
−s(x)|
that k) " Discrete B-spline kernels
!n/2"
k∈Z !
Analog input f (x) c[k] = "f, ϕ̃(·/a zB-spline interpolation 1/6 1/6
− k)#
Goal: Determine c[k] bn1 [k] = β n (x)|x=k
that that s(x) is a ”good” representation of our signal
suchsuch ←→ B1n (z) = β n (k)z −k
!# Smoothing splines
min $f − s$2 Goal: Determine c[k] such that s(x) is a ”good” representation of our signal !n/2"
k=−!n/2"
L2 f [k] c[k] such that s(x)|x=k = f [k] n !
s∈Va Discrete, noisy input: Discrete input b1 [k] = β n (x)|!
zB-spline
B1n (z)interpolation " −k #
Discrete input f [k] Smoothing such that s(x)| ←→ n = β n (k)z
x=k = f [k]
x=k −1
c[k] f (x)| = c[l]β n
(k
2 " B-spline interpolation: inverse 1filterk=−!n/2"
− l) = (b ∗ c) [k] ⇒ c[k] = (bn1 ) ∗ f [k]
solution
Noisy discrete input f [k] = s(k) + n[k] Analog such
x=k
input f (x)
algorithm
that c[k] such that min "f − s"L2 k∈Z
!n/2"
!
" Analog input (x)
# f+∞ c[k] $ such that min "f − s"L2 s∈Va
2
bn1 [k] =! β n (x)|x=k ←→
z
B1nn(z) = β n (k)z −kn −1
! s∈Va f [k] = c[l] β n
(x − l)| 6 = (b ∗ c) [k] (1
B-spline⇒α)
− = (b1 ) ∗ f [k]
2c[k] interpolation
interpolation
B-spline
min
Theorem: The |f [k]
solution − s(k)|
(among
2
+λ
all functions) Noisy
of the
m discrete
2
|D smoothing
s(x)| dx input f [k]
splineproblem=
problem s(k) + n[k] such that n −1
(b1 ) k∈Z [k] ←→
z x=k 1
= k=−!n/2"
Theorem:
s∈WThe
2 Noisy discrete
−∞ input f [k] = s(k) + n[k]
m solution (among all functions) of the smoothing" spline such
# that $ ! z + 4 + z −1 (1 − αz)(1 − αz −1 )
!n/2" !n/2"
! −1
k∈Z! ! $ +∞
f [k]−1= n
(x=b− =(x)|
(bn1 ∗←→z [k] z ⇒ !
2 n = (bn ) f [k]
! " " ##+∞+∞
minm m |f2[k] # $+∞
− s(k)|2 + λ
$ m
|D s(x)|2 dx (bn1 )1 [k]
c[l]bnβz1[k] nβl)|
= 6βx=k
1 [k](x)|
n x=k n
=
c) ←→(1
B− =1 (z) =1 β n∗(k)z
n α)c[k]
1 (z) B β−k
n
(k)z −k
" 2! ←→1 x=k
min |f [k] − 2 + λ s∈W2|D
s(k)|
min [k] m s(x)|
+ 2 dx " Efficient
1 − αz recursive
k∈Z
1 − αz implementation
z + 4 + z −1 (1 − αz)(1 − αz −1 )
min m
s∈Wm2 |f [k] − s(k)| + λ|f − s(k)|
|D 2
s(x)|
k∈Z λdx |D m
s(x)|2 −∞
dx −1 k=−!n/2"k=−!n/2"
m s∈W2 −∞ ! ! 6n (1 − α)2 n
(bn1 1) [k] f←→
s∈W2 k∈Z −∞ −∞ −1 z1 (symmetric exponential)
(x−1−β=
−1
k∈Z k∈Z [k] =f [k] c[l]= β c[l] l)|(xx=k
n
− = x=k
l)| (bn1 ∗=c)(b[k]∗ c) [k]
⇒ ⇒ = c[k]
c[k] (bn1 )= −1
(b∗n1f)[k] ∗ f [k]
1 − αz −1 1 − αz k∈Z z+4+z (1 − αz)(1 − αz1 −1 )
isisaacardinal
cardinalspline
splineofofdegree
degree2m 2m−−11. .ItsItscoefficients
coefficients
13 c[k]==hhλ∗∗ff[k]
c[k] [k]can
canbe
be k∈Z
λ 1 1
obtained by suitable digital filtering of the input samples
obtained by suitable digital filtering of the input samples f [k]. f [k]. Cascade n −1of first order z recursive
z 6
filters 6 (1 − α)(12 − α)2
1 − αz −1 1−
(b 1 ) αz(b
[k]
1)
n −1
[k]
←→ 7←→ = −1 =
13 z+4+ z+4+z
z −1 (1 − (1 − −
αz)(1 αz −1−) αz −1 )
αz)(1
!#
Specialcase:
Special
Special case:the
case: thedraftman’s
the draftman’s
draftman’s spline
spline
spline 13 1 1 1 1
Theminimum
The minimumcurvature
curvatureinterpolant
interpolantisisobtained
obtainedbybysetting
settingmm==22and →00. .
andλλ→ 1 − αz1−1− αz −1 1 − αz17− αz
It is a cubic spline
It is a cubic spline ! !
causal anti-causal
7
19 20

7 7

14
 Generic C-code (splines of any degree n) Spline interpolation
" Main recursion " Equivalent forms of spline representation
! !" #
void ConvertToInterpolationCoefficients ( ! s(x) = c[k]β n! " k) = n −1 s(k)
(x − # ∗ (bn1 )−1 [k] β n (x − k)
double c[ ], long DataLength, double z[ ], long NbPoles, double Tolerance)
{double Lambda = 1.0; long n, k;
s(x) = c[k]β n (x − k) = s(k) ∗ (b1 ) [k] β n (x − k)
k∈Z k∈Z
if (DataLength == 1L) return; k∈Z k∈Z !
for (k = 0L; k < NbPoles; k++) Lambda = Lambda * (1.0 - z[k]) * (1.0 - 1.0 / z[k]); ! = s(k)ϕnint (x − k)
for (n = 0L; n < DataLength; n++) c[n] *= Lambda; = s(k)ϕnint (x − k)
for (k = 0L; k < NbPoles; k++) { k∈Z
c[0] = InitialCausalCoefficient(c, DataLength, z[k], Tolerance); k∈Z
for (n = 1L; n < DataLength; n++) c[n] += z[k] * c[n - 1L]; 1
c[DataLength - 1L] = (z[k] / (z[k] * z[k] - 1.0)) " Cardinal (or fundamental) spline
# * (z[k] * c[DataLength - 2L] + c[DataLength - 1L]);
!
for (n = DataLength - 2L; 0 <= n; n--) c[n] = z[k] * (c[n + 1L]- c[n]); } ! ϕn (x) = (bn )−1 [k] β n (x − k)
}
ϕnint (x) = (b1 ) [k] β n (x −1 k)
nint−1
k∈Z
k∈Z

" Initialization
double InitialCausalCoefficient (
double c[ ], long DataLength, double z, double Tolerance)
{ double Sum, zn, z2n, iz; long n, Horizon; -5 -4 -3 -2 -1 1 2 3 4 5
Horizon = (long)ceil(log(Tolerance) / log(fabs(z)));
if (DataLength < Horizon) Horizon = DataLength; 7
zn = z; Sum = c[0]; 7
Finite cost implementation of an infinite impulse response
for (n = 1L; n < Horizon; n++) {Sum += zn * c[n]; zn *= z;}
return(Sum); interpolator !
}
21 22

Limiting behavior Geometric transformation of images

Geometric transformation
Geometric transformation
Geometric transformation
" Spline interpolator 1 +!
" 2D separable model k1!+n+1 l1 +n+1
!
2
Impulse response Frequency response 1
k1!
f (x,! y) =
+n+1 l1 +n+1
c[k, l] β n (x − l) β n (y − l)
! "n+1
f (x, y) = l] 1β (xl=l
c[k,k=k n k1 +n+1 l1n
! !(y − l)
−1 l) β +n+1
F sin(ω/2) 1 0.5 k=k1 l=l1 f (x, y) = c[k, l] β n (x − l) β n (y − l)
ϕnint (x) ←→ H n (ω) =
ω/2 B1n (ejω ) k=k1 l=l1
! "n+1 f [k, l] c[k, l] (x, y)
F sin(ω/2) 1 f [k, l] c[k, l] (x, y)
int (x) (ω) =
n
The cardinal splineϕinterpolators converge H nsinc-interpolator
←→ to the (ideal filter) as then jω f [k, l] c[k, l] (x, y)
ω/2 B1 (e )0.5 1 1.5 2
" goes
degree Asymptotic
to infinity: property #ω$ 2D filtering 2D re-sampling
lim ϕnint (x) = sinc(x), lim H n (ω) = rect (separable)
n→∞ The cardinal spline interpolators
n→∞ converge to2π
the sinc-interpolator (ideal filter) as the
(in all Lpdegree
-norms)goes to infinity:
#ω$
lim ϕnint (x) = sinc(x), lim H n (ω) = rect (in all Lp-norms )
n→∞ n→∞ 2π 5

(in all Lp -norms) " Applications 5

5
(Aldroubi et al., Sig. Proc., 1992)
zooming, rotation, re-sizing, re-formatting, warping
7
Includes Shannon’s theory as a particular case !

23 24
7
Cubic spline coefficients in 2D High-quality image interpolation
! Splines: best cost-performance tradeoff

35

Geometric transformation

Lena 256 x 256, rotation 15 x 24°, central 128 x 128 SNR (dB)
Bspline(6)
Geometric transformation
Digital filter k1!
+n+1 l1 +n+1 30 Bspline(5)

(recursive, ! Bspline(4)

separable) f (x, y) = c[k, l] β n (x − l) β n (y − l) Bspline(3)

k1!
+n+1 l1 +n+1
! k=k1 l=l1
f (x, y) = c[k, l] β n (x − l) β n (y − l) 25
Bspline(2)
Schaum(2) [1993] Meijering(7) [1999]
Keys [1981] Meijering(5) [1999]
k=k1 l=l1
f [k, l] c[k, l] (x, y) Schaum(3) [1993]

Dodgson [1997]
Demo
f [k, l] c[k, l] (x, y) Pixel values f [k, l] 20
Linear

Pixel values f [k, l] B-spline coefficients c[k, l] Sinc Hamming(4)

Nearest-neighbor
German [1997]
B-spline coefficients c[k, l] 15

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

Execution time (s rot-1)

Thévenaz et al., Handbook of Medical Image Processing, 2000

5
25 26

Interpolation benchmark Fast multi-scale filtering

Cumulative rotation experiment: the best algorithm wins ! f (x) ∗ β (x/a) n
Three alternative
Three alternative methods formethods
methods
Three alternative the fast computation
for the of:computation
fastfastevaluation
for the ofof: f (x) ∗ β n (x/a)
n
O(NThree alternative
) complexity
Three alternative
n
methods
Three
for
methods the
alternative
Three fastfast
for the computation
methods
alternative for fof:(x)
of: the
computation
methods for fast
the ∗fast
f (x) (x/a)
β n (x/a)of: fof:(x)
β∗ computation
computation ∗ β∗ (x/a)
f (x) β n (x/a)
O(N ) complexity
O(N
1) Pyramid) complexity
O(N
Pyramid ) complexity
or
and tree O(N ) complexity
treealgorithms
O(N
algorithms ) complexity
Pyramid and tree algorithms
a = 2iPyramid Pyramid andand treetree
algorithms
Pyramid
algorithms
Pyramid andand treetreealgorithms
algorithms
1 1 1 1 1 1 1 1
Three alternative methods for the fast computation of: f (x) ∗aβ= n
(x/a)
a i =i 2i
a 2= 2 methods =a 2fasti i
H2n (z)
Three alternative ↓2 fora the = 2computation of: f (x) ∗ β (x/a) n

O(N ) complexity
2 (z) 2(z)
n nn
O(N H
Recursive(z)
H
H) 2complexity
2 (z) filtering ↓ 2(iterative
n n
↓ 2H↓2 H 2 (z)
moving average) ↓ 2↓ 2 a = m ∈ Z+
Pyramid and tree algorithms =moving Z∈average) =a+m mZ∈ Z+
+ + +
smPyramid Recursive
smRecursive
[k]Binomial and
[k −tree
= Recursive
filter
filtering
1] (iterative
algorithms
+filtering
f [k] Recursive
filtering
− (iterative
fRecursive
[k − moving
m]filtering
(iterative moving average)
filtering (iterative
moving
average) aaverage)
(iterative
(Unser
m
a et ∈
=moving
mIEEE
al., =
aZTrans. ∈aZ1993)
mPAMI,
average) =∈
a=2 i
i [k] = sm [k − 1] +sfm[k] [k] =
a= s2m sm sm
Differential:[k][k] =integration
sms[km−
= [k1]−+and 1] −
sfm[k]+[k] fs[k
f− =mf−
[k]
weighted [k −
m]
s[km− 1]
[k+1]−f+[k]
[kfdifferences
− −
m] m] − f−[kf−
f [k] [km]− m]
Bilinear Windowed-sinc Cubic spline
H2n (z) ↓2 2) Recursive filtering (iterated moving average)
a 2∈(z) Differential: ↓2 integration Differential:
and weightedintegration
differences and weighted differences
n
H R Differential: integration Differential:
and weighted
andintegration differences and weighted
differences differences
+
Differential: integration weighted
Recursive filtering (iterative moving average) a =am∈∈RZ +
+ + !
a ∈aR∈xmoving
+
Recursive Rfiltering (iterative
a(or∈primitive): R+ average) =m
a (x)} 1 ∈ 1Z+ 1
a ∈ R+ F (x) = −∞! fx (t)dt = D−1 {f 1 1
Integral
sm [k] = sm [k − 1] + f [k] − f [k − m] !x !
−1 −1 x ! x
m [k] =Integral[k primitive):
1] + f [k]aF (x) =(or
[k
fa(x) = (t)dt
fprimitive):
f!(t)dt=
F (x)DF = {fa) (x)} 1f (t)dt =
1 D =1 D{f1 (x)}
−1 −1
x − f= (x(x)
D ={f (x)} f1(t)dt {f (x)}
Integralsm(or − Integral primitive):
sFinite difference (or primitive):
with step : − ∆FIntegral
{f −
(x} m]
(or=
−∞ −∞ f (x) − −∞ −∞
Truncated
Truncatedsinc
sinc Cubic
Cubic
spline
spline Differential: integration and weighted differences FiniteIntegral (or primitive): F (x) = −∞ f (t)dt = D−1 {f (x)}
f (x) 0difference
Differential: (x/a) =with
integration
F (x)step aFFinite
Finite
and : a ∆ ∆ a)a(x}
{f
difference
(x weighted = (x}
{f =
with
∆ =(x)
fstep
differences (x)
fstep fa∆
−:(xaf∆
a−: (x)} (xa(x}
{f
− a)
− (x}
{f a)= f=(x) − f−(xf−
f (x) (xa)− a)
aD
1with−1
∗ Finite
β+ difference with −step :−adifference {f (Unser et al., IEEE Trans. Sig. Proc, 1994)
a ∈ R+ Finite 0 difference with step 0 a:0 ∆a {f (x} = f (x) − f (x − a)
a ∈ fR(x) ∗ β∗+ (x/a) = F=(x) (x) β(x
∗+F(x/a) −= a) ∆ =a(x)D F−(x(x)} −= a) ∆ Da D{f (x)}
1 −1 1 −1
27 !x
+
f (x) β+ (x/a)
0 fF f−(x)
(x) ∗F− β− (x a)
+ (x/a)
F ∆ −
D{f
F a(x)
1 −1
F−
{f (x
(x)}a) =a∆ 1 −1
{f (x)}
28
Integral (or primitive): F (x) = −∞ f (t)dt = D−1 {f (x)} f (x) ∗ β 0 (x/a) =! F x (x) 17 − F (x −1a) = ∆1 D−1 {f (x)}
−
Integral (or primitive):+ F (x) = −∞ f (t)dt = D {f (x)} a
Finite difference with step a: ∆a {f (x} = f (x) − f (x − a)
17 17 17 17
Finite difference with step a: ∆a {f (x} = f (x) − f (x − a)
f (x) ∗ β+ 0
(x/a) = F (x) − F (x − a) = ∆1a D−1 {f (x)} 0 17
f (x) ∗ β+ (x/a) = F (x) − F (x − a) = ∆1a D−1 {f (x)}
 a = 2i O(N ) complexity
alternative methods for the fast computation of: f (x) ∗ β n
n(x/a)
n alternative methods for the fast computation of: f (x) ∗ β (x/a)
Three
Three
H2 (z) ↓2 Pyramid and tree algorithms
O(N )) complexity
O(N complexity
a = 2i average) a = m ∈ Z+
Recursive filtering (iterative moving
Pyramid
Pyramid and and tree
tree algorithms
algorithms
[k2−(z) ↓2
n
sm [k]i = sm [k − 1] + f [k] − f H m]
Fast multi-scale filtering (Cont’d)
aa == 22i
Differential: integration and weighted Recursive filtering (iterative moving average) a = m ∈ Z+
differences
Splines: more applications
H n
(z)
H22 (z)+
n ↓
↓22
a∈R sm [k] = sm [k − n1] + f [k] − f [k − m]
Challenge:
Three alternative methods
Recursive forO(N)
the fast evaluation
computation of: f (x)
of ∗ β (x/a)a=m∈Z + ! Sampling and interpolation
Recursive filtering (iterative
filtering (iterative! xmoving
moving average)
average)
Differential: −1 a = m ∈ Z+
Integral (or primitive): F (x) = −∞ f (t)dt = Dintegration
{f (x)} and weighted differences ! Interpolation, re-sampling, grid conversion
O(N ) complexity sm [k] = s m[k
m [k] = sm [k −− 1]1] +
+ ff [k] − ff [k
[k] − [k −
−R m]
3) sDifferential
Finite difference approach
with step a: ∆a {f a∈ m]+
(x} = f (x) − f (x − a)
! Image reconstruction
Pyramid and tree Differential:
algorithms
Differential: integration and weighted
integration and weighted differences
differences !x ! Geometric correction
f (x) ∗+β+ 0
(x/a) = F (x) − F (xIntegral − a) = (or∆
aD
1 −1 F (x) = −∞ f (t)dt = D−1 {f (x)}
{f (x)}
primitive):
a = 2i aa ∈∈R R+ step a: ∆a {f (x} = f (x) − f (x − a)
! Feature extraction
!! xFinite difference with
primitive): F (x)
(x) =
=
x f (t)dt = D−1
17 0 = D−1{f (x)} ! Contours, ridges
f (x) ∗ β+ (x/a) = F(x)}
(t)dt
Integral
Integral (or
(or primitive): F f {f
H2 (z)
n
↓2 −∞
−∞ (x) − F (x − a) = ∆1a D−1 {f (x)} ! Differential geometry
Finite difference with step : ∆ {f (x} = f (x)
Finite difference with step a: ∆aa {f (x} = f (x) − f (x −+a)
a − f (x − a) ! Image pyramids
Recursive filtering (iterative moving average) a 1= m ∈Z
ff (x) ∗ β 0
(x/a) = F (x) − F (x − a) = ∆ D −1
{f (x)} ! Shape and active contour models
(x) ∗ β+ (x/a) = F (x) − F (x − a) = ∆a D {f (x)}
0
+ a
1 −1 17

sm [k] = sm [k − 1] + f [k] − f [k − m]
!" Generalization multiple weighted ! Image matching
17 integration differences
Differential: integration and weighted differences
17 ! Stereo
f (x) ∗ (x/a) =
Running sums Interpolation + filtering ! Image registration (multi-modal, rigid body or elastic)
an ∆a D {f (x)}
n 1 n+1 −(n+1)
a ∈ R+ β+
! x of a spline of degree
Principle: The integral n is a spline of degree n + 1. ! Motion analysis
Integral (or primitive): Fn(x) = −∞ 1 f (t)dt = D−1 {f (x)}
f (x) ∗ β+ (x/a) = an ∆a
n+1 −(n+1)
(Munoz
D
et al.,
{f (x)}
IEEE Trans. Imag. Proc, 2001) ! Optical flow
Finite difference with step a: ∆a {f (x}
Principle: The integral
= f (x) − f (x − a)
of a spline of degree n is a spline of degree n + 1.
29 30
f (x) ∗ β+
0
(x/a) = F (x) − F (x − a) = ∆1a D−1 {f (x)}

17

SPLINES AND WAVELET THEORY

18
Scaling function Scaling function
18
! Scaling functions Definition: ϕ(x) is an admissible scaling function of L2 iff:
Scaling function
! !
! Order of approximation Riesz
! Definition: basis
Scaling ancondition
ϕ(x) isfunction !"
admissible scaling function of L2 iif: !
Definition: ϕ(x) is an∀c
! of L iif: ! ! !
∈ "2 ,scalingA
admissible ·!"
#c#!22 ≤ ! !! c[k]ϕ(x − k)! ≤ B · #c#!2
function
!
! ! ! !
! B-spline factorization theorem ∀c ∈ "2 , A · #c#! ≤ !
!"
Scaling function
!
2
! !
c[k]ϕ(x − k)! ≤ B · #c#!
! k∈Z
! 2
L2
∀c ∈ "2 , A · #c#!2 ≤ ! c[k]ϕ(x − k∈Zk)! ≤ B · #c#!2 L2
! !
! Splines: the key to wavelet theory Definition: ϕ(x) is an admissible scaling
!
k∈Z function of L2 iif:
!
L2

! Two-scale
!" relation
! "
! !
! Fractional B-spline wavelets ∀c ∈ "2 , A · #c#!2 ≤ !
!
c[k]ϕ(x
"
ϕ(x/2)
− k)!= ≤ B
! k∈Z
h[k]ϕ(x "1 1
· #c#!2 − k)
k∈Z
ϕ(x/2) = h[k]ϕ(x
" −
L2ϕ(x/2)
k) = h[k]ϕ(x − k)
k∈Z ϕ(x − k) = 1 k∈Z
"
k∈Z
" ϕ(x − k) = 1 "
ϕ(x/2) = k∈Z h[k]ϕ(x − k)
! Partition k∈Z of unity ϕ(x − k) = 1
" 1
k∈Z
ϕ(x − k) 1= 1
k∈Z

31
32
1
1
 From scaling functions to wavelets Order of approximation
Order of approximation
! From of
Wavelet bases scaling functions to1989]
L2 [Mallat-Meyer, wavelets ! General “shift-invariant” space at scale a a=1
Order of approximation
! %
Theorem: For any given admissible scaling function of L2 , ϕ(x), " #x $
! Va (ϕ) = !
sa (x) = c[k]ϕ − k : c ∈# "2
there exists a wavelet ψ(x/2) = g[k]ϕ(x − k) such that the " a 1 2 3 4 5
k∈Z k∈Z
Va (ϕ) = sa (x) = Order
c[k]ϕ(t k) :approximation
− of c ∈ "2
family of functions ! # $ %
" # i $% 1 -1 "
k∈Z a=2
Va (ϕ) = sa (x) = k∈Z c[k]ϕ at − k : c ∈ "2
2−i/2 ψ x−2
2i
k
Projection
! ∀f ∈ L2 , Poperator
a f = arg min $f − sa $L ∈ V2
i∈Z,k∈Z 2
sa ∈Va 2 4
forms a Riesz basis of L2 . ∀f ∈ L2 , Pa f = arg min $f − sa $L2 ∈ V2
∀f ∈ L2 ,sa ∈VaPa f = arg min $f − sa $L2 ∈ V2
sa ∈Va
Definition
! Orderfunction
A scaling/generating
Definition of approximation
ϕ has order of approximation γ iff
! Constructive approach: perfect reconstruction filterbank γ γ (γ)
∀f ∈ W $f
Definition
2,
A scaling/generating − Pa ϕ
function f $has ≤ C of
L2 order · aapproximation
· $f $L2L iff
si+1 (k) B-splines of ∀f Wα2Lhave
A∈scaling/generating
degree , $f L2 ≤ C ϕ
− Pofa fapproximation
order $function L · $f
· ahas (L)
α +$Lof
γ =order 12 approximation γ iff
2 H˜ (z"1 ) !22 2H(z)
si (k)
!2
si (k) ∀f ∈ W2γ , $f − Pa f $L2 ≤ C · a · $f γ (γ)
$L2
di +1(k) B-splines of degree α have
3 order of approximation γ =α+1
! 2G˜ (z"1 ) !2 !! 2 2G(z) 16

B-splines of degree # have order of approximation $=#+1

33 34
! !

16

Spline reconstruction of a CAT-scan B-spline factorization

f (x) ∗ β+
n
(x/a) = an ∆a
1
D
n+1 −(n+1)
{f (x)}
Principle: The integral of a spline of degree n is a spline of degree n + 1.
Piecewise constant
!" Factorization theorem
γ=1 A valid scaling function ϕ(x) has order of approximation γ iff
! α "
γ=4 ϕ(x) = β+ ∗ ϕ0 (x)
α
where β+ with α = γ − 1: regular, B-spline part

f (x) ∗ β+
n
(x/a) = an ∆a
1
D
n+1 −(n+1)
{f (x)} ϕ0 ∈ S ! : irregular. distributional part
Principle: The integral of a spline of degree n is a spline of degree n + 1. A valid scaling function ϕ(x) has order of approximation γ iff
γ=1 ! α "
Cubic spline ϕ(x) = β+ ∗ ϕ0 (x)
γ=4 21
α
where β+ with α
=
= γ − 1: regular, B-spline part
"

ϕ0 ∈ S ! : irregular. distributional part

!" Refinement filter: general case
# $γ 21
1 + z −1
H(z) = · Q(z)
2 % &' (
% &' ( distributional part
spline part
21
36
35
Splines: the key to wavelet theory CONCLUSION
Sobolev smoothness ! Distinctive features of splines
! s" # L2 ! Simple to manipulate
! Smooth and well-behaved
!"s
! ! Excellent approximation properties
! Multiresolution properties
B-spline factorization: Approximation order: Multi-scale differentiator
# $1
! ! ˜ˆ (") # ($ j") % , " & 0
! Fundamental nature (Green functions of derivative operator)
! = "+ % ! 0 f ! Pa f L2
= O(a " ) !
! Splines and image processing
general case: n < ! " n + 1
! ! ! A story of avoidance and, more recently, love….
compact support: ! = n + 1 (Strang-Fix, 1971)
! Best cost/performance tradeoff
Polynomial reproduction
! Many applications …..
degree: n = # ! " 1$
! Unifying signal processing formulation
c ! Tools: digital filters, convolution operators
Vanishing moments:
p
! Efficient recursive filtering solutions
!x ˜ (x)dx = 0, p = 0,…, n
" (Unser and Blu, IEEE-SP, 2003)
! Flexibility: piecewise constant to bandlimited
37 38

Scale space vs. splines Acknowledgments

!" Linear scale space !# Smoothing splines Many thanks to
(redundant) !# Multi-resolution analysis " Dr. Thierry Blu
(non-redundant) " Prof. Akram Aldroubi
!# Finite difference methods !" Hilbert space methods " Prof. Murray Eden
" Dr. Philippe Thévenaz
" Annette Unser, Artist
!# Non-linear diffusion !# Non-linear smoothing splines
!# Wavelet denoising
+ many other researchers,
No need to be dogmatic: you can also use splines to
and graduate students
improve “scale-space” algorithms
" ! Hilbert space framework: Think analog, act discrete !
" ! “Optimal” discretization of differential operators
" ! Fast multi-scale, multi-grid algorithms
" ! ... 39 40
The end: Thank you!

! Spline tutorial
! M. Unser, "Splines: A Perfect Fit for Signal and Image
Processing," IEEE Signal Processing Magazine, vol. 16, no. 6, pp.
22-38, 1999.
! Spline and wavelets
! M. Unser, T. Blu, "Wavelet Theory Demystified," IEEE Trans. on
Signal Processing, vol. 51, no. 2, pp. 470-483, 2003.
! Smoothing splines and stochastic formulation
! M. Unser, T. Blu, "Generalized Smoothing Splines and the Optimal
Discretization of the Wiener Filter," IEEE Trans. Signal
Processing, in press.

!
Preprints and demos: http://bigwww.epfl.ch/

41