Medical Image Analysis 10 (2006) 215–233

www.elsevier.com/locate/media

United Snakes
a,* b,c d,c
Jianming Liang , Tim McInerney , Demetri Terzopoulos
a
Computer Aided Diagnosis and Therapy, Siemens Medical Solutions USA, Inc., Malvern, PA 19355, USA
b
Department of Computer Science, Ryerson University, Toronto, Ont., Canada M5B 2K3
c
Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 3H5
d
Department of Computer Science, University of California at Los Angeles, Los Angeles, CA 90095, USA

Received 21 September 2005; accepted 27 September 2005

Available online 28 November 2005

Abstract

Since their debut in 1987, snakes (active contour models) have become a standard image analysis technique with several variants now
in common use. We present a framework called ‘‘United Snakes’’, which has two key features. First, it unifies the most popular snake
variants, including finite difference, B-spline, and Hermite polynomial snakes in a consistent finite element formulation, thus expanding
the range of object modeling capabilities within a uniform snake construction process. Second, it embodies the idea that the heretofore
presumed competing technique known as ‘‘live wire’’ or ‘‘intelligent scissors’’ is in fact complementary to snakes and that the two tech-
niques can advantageously be combined by introducing an effective hard constraint mechanism. The United Snakes framework amplifies
the efficiency and reproducibility of the component techniques, and it offers more flexible interactive control while further minimizing
user interactions. We apply United Snakes to several different medical image analysis tasks, including the segmentation of neuronal den-
drites in EM images, dynamic chest image analysis, the quantification of growth plates in MR images and the isolation of the breast
region in mammograms, demonstrating the generality, accuracy and robustness of the tool.

2005 Elsevier B.V. All rights reserved.

Keywords: Snakes; Active contours; Live wire; Intelligent scissors; Finite elements; Interactive image analysis; Neuronal dendrite segmentation; Dynamic
chest image analysis; Growth plate quantification; Breast region isolation

1. Introduction against noise, improve selectivity for certain classes of

objects, etc.). No formulation has yet emerged as the ‘‘gold
Snakes (active contour models) quickly gained popular- standard’’. Rather, the primary variants seem well-suited to
ity following their debut in 1987 (Kass et al., 1988). They different applications with particular image modalities and
have proven to be especially useful in medical image anal- processing scenarios.
ysis (McInerney and Terzopoulos, 1996; Singh et al., 1998) Given the broad array of choices for the user, there is a
and for tracking moving objects in video (Terzopoulos and need for a portable and reusable snakes implementation
Szeliski, 1992; Blake and Isard, 1998), among other appli- which unites the best features of the variants while main-
cations. Variants such as finite element snakes (Cohen and taining the simplicity and elegance of the original formula-
Cohen, 1993), B-snakes (Menet et al., 1990; Blake and tion. To this end, our first contribution in this paper is to
Isard, 1998), and Fourier snakes (Staib and Duncan, unify the most important snakes variants, including finite
1992) have been proposed in an effort to improve aspects difference, B-spline, and Hermite polynomial snakes, in a
of the original finite difference implementation (e.g., to comprehensive finite element formulation, where a particu-
decrease initialization sensitivity, increase robustness lar type of snake can be derived by simply changing the
finite element shape functions at the user level.
*
Corresponding author. Tel.: +1 610 448 1460. Subsequent to snakes, a related technique, known as
E-mail address: jianming.liang@computer.org (J. Liang). ‘‘live wire’’ or ‘‘intelligent scissors’’ (Mortensen et al.,

1361-8415/$ - see front matter
2005 Elsevier B.V. All rights reserved.
doi:10.1016/j.media.2005.09.002
216 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

1995; Falcão et al., 1996; Barrett and Mortensen, 1997; which couples the snake to the image via a scalar potential
Falcão and Udupa, 1997; Mortensen and Barrett, 1998; function PI(x, y) typically computed from I(x, y) through
Falcão et al., 1998, 2000) emerged as an effective interactive image processing. The Euler–Lagrange equations of mo-
boundary tracing tool. Based on dynamic programming tion for a dynamic snake are
(Falcão et al., 1998) or Dijkstra
s graph search algorithm    2 
o2 v ov o ov o2 ov
(Mortensen and Barrett, 1998), it was originally developed l 2 þc
a þ 2 b 2 ¼ qðvÞ. ð3Þ
as an interactive 2D extension to earlier optimal boundary ot ot os os os os
tracking methods. Live wire features several similarities The first two terms represent inertial forces due to the mass
with snakes, but it is generally considered in the literature density l(s) and damping forces due to the dissipation den-
as a competing technique. Our second contribution in this sity c(s). The next two terms represent the internal stretch-
paper is the idea that live wire and snakes are in fact com- ing and bending deformation forces. On the right-hand side
plementary techniques that can be advantageously com- are the external forces q(v) =
$PI(v) + f(s, t), where the
bined via a simple yet effective method for imposing hard image forces are the negative gradient of the image poten-
constraints on snakes. An advantage of this combination tial function. The user may guide the dynamic snake via
is the efficient handling of large images – a potential obsta- time-varying interaction forces f(s, t) (usually applied
cle for live wire alone. through an input device such as a mouse), driving the
We call our software implementation United Snakes snake out of one energy minimizing equilibrium and into
(Liang et al., 1999a,b), because it unites several snake vari- another. Viewed as a dynamical system, the snake may also
ants with live wire to offer a general purpose tool for inter- be used to track moving objects in a time-varying (video)
active image segmentation that provides more flexible image I(x, y, t).
control while reducing user interaction. United Snakes is
implemented in the highly portable Java programming lan- 2.1. Finite element formulation
guage. We have applied United Snakes to several different
medical image analysis tasks including the segmentation of In a finite element formulation (Zienkiewicz and Taylor,
neuronal dendrites in EM images, dynamic chest image 1989), the parametric domain is partitioned into finite sub-
analysis, the quantification of growth plates in MR images domains, so that the snake contour is divided into ‘‘snake
and the isolation of the breast region in mammograms, elements’’. Each element e is represented geometrically
demonstrating the generality, accuracy, robustness, and using shape functions N(s) and nodal variables ue(t). The
ease of use of the tool. nodal variables of all the elements are assembled into the
In the remainder of this paper, we first describe our snake nodal variable vector u(t). This leads to a discrete
finite element framework in Section 2 and show how sev- form of the equations of motion (3) as a system of sec-
eral snake variants can be integrated within it. Section 3 ond-order ordinary differential equations in u(t):
describes the live wire technique. We justify the idea of
M€u þ Cu_ þ Ku ¼ g; ð4Þ
combining snakes with live wire in Section 4 and develop
a hard constraint mechanism in Section 5 that makes this where M is the mass matrix, C is the damping matrix, K is
combination possible. Section 6 presents results utilizing the stiffness matrix, and g is the external force vector, which
the United Snakes system in medical image segmentation are assembled from corresponding element sub-matrices
scenarios. We conclude in Section 7 and propose future that depend on the shape functions N (Appendix A details
extensions of United Snakes. the finite element formulation).
By using different shape functions and thereby generat-
2. Finite element unification of snakes ing different stiffness matrices, the behavior of the resulting
snake can be adapted to specific tasks. For example, snakes
A snake is a time-varying parametric contour v(s, t) = that use B-spline shape functions are typically character-
(x(s, t), y(s, t))T in the image plane ðx; yÞ 2 R2 , where x ized by a low number of degrees of freedom, typically use
and y are coordinate functions of parameter s and time t. polynomial basis functions of degree 2 or higher, and are
The shape of the contour subject to an image I(x, y) is dic- inherently very smooth. Therefore, these ‘‘B-snakes’’
tated by an energy functional EðvÞ ¼ SðvÞ þ PðvÞ. The (Menet et al., 1990; Blake and Isard, 1998) can be effective
first term is the internal deformation energy defined as in segmentation or tracking tasks involving noisy images
Z

2
2
2 where the target object boundaries may exhibit significant
1 L
ov

o v

SðvÞ ¼


aðsÞ

þ bðsÞ

2

ds; ð1Þ gaps in the images. On the other hand, object boundaries
2 0 os os with many fine details or rapid curvature variations may
where a(s) controls the ‘‘tension’’ of the contour and b(s) best be segmented by a snake that uses simpler shape func-
regulates its ‘‘rigidity’’. The second term is an external im- tions and more degrees of freedom, such as a finite differ-
age energy ence snake (Kass et al., 1988). Various contour
Z L representations are reviewed in Gavrila (1996). The unifica-
PðvÞ ¼ P I ðvÞ ds; ð2Þ tion of these different shape functions in a single frame-
0 work expands the range of object modeling capabilities,
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 217

and the range of segmentation and tracking scenarios that (Kwon and Bang, 1997) may be employed to approximate
can be handled by a single tool. the value of the integral for the element external force vec-
The following sections address Hermitian shape func- tor Fe. For element ei we have
tions, B-spline shape functions, and ‘‘shape functions’’ Z l X
for finite difference snakes. Since the two coordinate func- Fexi ¼ NTh qx ðvðsÞÞ ds ¼ qj Nh ðnj ÞT qx ðvðnj ÞÞ; ð10Þ
tions x(s) and y(s) of the snake v(s) are independent, we 0 j
shall discuss the shape functions in terms of only one com- where the subscript x indicates the association with coordi-
ponent x(s); the shape functions for y(s) assume an identi- nate function x(s), and where nj and qj are the jth Gaussian
cal form. integration point and its corresponding weighting coeffi-
cient, respectively. Feyi is derived in a similar fashion.
2.2. Hermitian shape functions To make the global matrix assembly process identical
for all shape functions, we introduce assembling matrices.
In the case of Hermitian snakes, x(s) (0 6 s 6 l, where l Suppose that we have a snake with n elements and N nodes
is the element parametric length) is approximated with a (N = n if the snake is closed and N = n + 1 if it is open).
cubic polynomial function, parameterized by position x For the ith element ei of the snake (0 6 i 6 n
1), the
and slope h at the endpoints s = 0 and s = l of an element. assembling matrices are GeMi ¼ GeCi ¼ Geai ¼ Gebi ¼ GeFi ¼
We can show that xðsÞ ¼ Nh uei , where uei ¼ ½xi ; hi ; xiþ1 ; Gei , where
T
hiþ1  are the nodal variables of element ei and Nh = sH 
1 if ðj þ diÞ mod ðdN Þ ¼ k;
are the Hermitian shape functions, with s = [1, s, s2, s3] ei
ðG Þjk ¼ ð11Þ
and the Hermitian shape matrix is 0 otherwise
2 3 are (2d) · (dN) matrices, with d the number of degrees of
1 0 0 0
6 0 1 0 0 7 freedom of each node in an element (here d = 2). Hence,
6 7 Ka, Kb and F may be assembled as follows:
H¼6 7. ð5Þ
4
3=l2
2=l 3=l2
1=l 5
X
n
1
2=l3 1=l2
2=l3 1=l2 M¼
T
ðGeMi Þ Mei ðGeMi Þ; ð12Þ
i¼0
It is reasonable to assume that the mass density l(s), the
dissipation density c(s), the tension function a(s) and rigid- X
n
1
T
C¼ ðGeCi Þ Cei ðGeCi Þ; ð13Þ
ity function b(s) are constant within the element. Hence,
i¼0
for element ei, the mass matrix is
X
n
1
2 3 Ka ¼
T
ðGeai Þ Keai ðGeai Þ; ð14Þ
156 22l 54
13l
i¼0
6 22l 4l2 13l
3l2 7
6 7 X
n
1
Mei ¼ li 420l6 7 ð6Þ
4 54 13l 156
22l 5 Kb ¼ ðGebi ÞT Kebi ðGebi Þ; ð15Þ
2 2 i¼0

13l
3l
22l 4l
X
n
1
T
the damping matrix is F¼ ðGeFi Þ Fei . ð16Þ
i¼0
2 3
156 22l 54
13l
In our implementation, we set the element parametric
6 22l 4l2 13l
3l2 7
6 7 length l to 1. Only the shape matrix and the assembling
Cei ¼ ci 420l6 7 ð7Þ
4 54 13l 156
22l 5 matrices are determined by specific shape functions. There-

13l
3l2
22l 4l2 fore, in the following section we shall focus only on the der-
ivation of the shape matrix and the assembling matrices for
and the stiffness matrices associated with the tension and B-spline shape functions, and briefly mention other kinds
rigidity components are, respectively, of shape functions which are suitable for snakes.
2 3
36 3l
36 3l
ai 6
6 3l 4l2
3l
l2 7 7 2.3. B-spline shape functions
Keai ¼ 6 7; ð8Þ
30l 4
36
3l 36
3l 5
For B-spline shape functions, the x(s) coordinate func-
3l
l2
3l 4l2 tion of v(s) is constructed as a weighted sum of NB basis
2 3
12 6l
12 6l functions Bn(s), for n = 0, . . . , NB
1, as follows:
b 6 6 6l 4l2
6l 2l2 7
7 x(s) = B(s)TQx, where BðsÞ ¼ ½B0 ðsÞ; . . . ; BN B
1 ðsÞT ; Qx ¼
Kebi ¼ 3i 6 7. ð9Þ ½x0 ; . . . ; xN B
1 T and xi are the weights applied to the respec-
l 4
12
6l 12
6l 5
tive basis functions Bn(s).
6l 2l2
6l 4l2
A B-spline span serves as an element in our finite ele-
An analytic form of the external forces q(v) in (3) is gener- ment formulation (hence ‘‘span’’ and ‘‘element’’ are inter-
ally not available. Therefore, Gauss–Legendre quadrature changeable terms). Consequently, we shall determine the
218 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

nodal variables, the shape matrix, and the assembling The assembling matrix Gei can be defined as
matrix associated with a span. When all spans are of unit 
ei 1 if ðj þ br Þ mod N B ¼ k;
length, the knot multiplicities at the breakpoints are ðG Þjk ¼ ð24Þ
m0, . . . , mL (L isPthe number of spans and the total number 0 otherwise.
L
of knots N B ¼ i¼0 mi ), the P knot values ki are determined In a similar fashion as above, we may construct other kinds
l
by ki = l, such that 0 6 ði
j¼0 mj Þ < mlþ1 . Furthermore, of shape functions; for instance, NURBS shape functions
the nth polynomial Brn;d in span r can be computed as (Terzopoulos and Qin, 1994), Catmull-Rom shape func-
follows: tions, Bézier shape functions, and Fourier shape functions
 (Staib and Duncan, 1992). The latter are global shape func-
r
1 if k n 6 r < k nþ1 ;
Bn;1 ðsÞ ¼ ð17Þ tions over the whole snake, thus the associated assembling
0 otherwise;
matrix becomes an identity matrix.
ðs þ r
k n ÞBrn;d
1 ðsÞ ðk nþd
s
rÞBrnþ1;d
1 ðsÞ
Brn;d ðsÞ ¼ þ .
k nþd
1
k n k nþd
k nþ1 2.4. Finite difference snakes in element form
ð18Þ
Despite the differences between finite element snakes
For span r, the index br for the first basis function whose and finite difference snakes, the finite difference snakes
supportP includes the span can be determined as can also be constructed in the finite element fashion, using
br ¼ ½ð ri¼0 mi Þ
d mod N B . Therefore, the Dirac delta function d(s) as the shape function. The
I ¼ ½br ; ðbr þ 1Þ mod N B ; . . . ; ðbr þ d
1Þ mod N B  construction primitives are as follows. For a snake with n
nodes, Mei is a 1 · 1 matrix and its corresponding assem-
are the indices of the nodal variables and also those of the d
bling matrix GeMi is a 1 · n matrix:
polynomials Brn;d .1 Now, the shape matrix for span r can be
T
constructed by collecting the coefficients of each of the d Mei ¼ li ½ 1  ½ 1  ¼ li ½ 1 ; ð25Þ
polynomials Brn;d as its columns. For example, the shape 
 ei  1 if i ¼ k;
matrix of a regular cubic B-spline is GM 0;k ¼ ð26Þ
2 3 0 otherwise;
1=6 2=3 1=6 0
6
1=2 where 0 6 i 6 n
1 for both open and closed snakes. Cei is
6 0 1=2 0 77 also a 1 · 1 matrix with a 1 · n assembling matrix GeCi :
H¼6 7 ð19Þ
4 1=2
1 1=2 0 5 T
Cei ¼ ci ½ 1  ½ 1  ¼ ci ½ 1 ; ð27Þ

1=6 1=2
1=2 1=6 
 ei  1 if i ¼ k;
GC 0;k ¼ ð28Þ
and the element matrices for element ei are 0 otherwise;
2 3
20 129 60 1 where 0 6 i 6 n
1 for both open and closed snakes. Keai is
l 6 6 129 1188 933 60 7
7 a 2 · 2 matrix and its corresponding assembling matrix Geai
Mei ¼ i 6 7; ð20Þ is a 2 · n matrix:
5040 4 60 933 1188 129 5  
1 60 129 20 T 1
1
2 3 Keai ¼ ai ½
1 1  ½
1 1  ¼ ai ; ð29Þ
20 129 60 1
1 1

 ei 
c 66 129 1188 933 60 77 Ga jk ¼
1 if ðj þ iÞ mod n ¼ k;
ð30Þ
Cei ¼ i 6 7; ð21Þ 0 otherwise;
5040 4 60 933 1188 129 5
1 60 129 20 where 0 6 i 6 n
2 for an open snake and 0 6 i 6 n
1
2 3 for a closed snake. Kebi is a 3 · 3 matrix and with it is asso-
6 7
12
1
ciated a 3 · n assembling matrix Gebi :
ai 6
6 7 34
29
12 7 7 2 3
Keai ¼ 6 7; ð22Þ 1
2 1
120 4
12
29 34 7 5 T 6 7
Kebi ¼ bi ½ 1
2 1  ½ 1
2 1  ¼ bi 4
2 4
2 5;

1
12 7 6
2 3 1
2 1
2
3 0 1
ð31Þ
bi 6
6
3 6
3 0 7
7 
Kebi ¼ 6 7. ð23Þ 1 if ðj þ iÞ mod n ¼ k;
64 0
3 6
3 5 Gebi ¼ ð32Þ
jk 0 otherwise;
1 0
3 2
where 0 6 i 6 n
3 for an open snake and 0 6 i 6 n
1 for
1
a closed snake. The 1 · n assembling matrix GeFi is defined as
In an open B-spline snake, d knots are introduced at the two ends. As a 
result, the index for the first basis function in the first span is zero (i.e.,  ei  1 if i ¼ k;
GF 0;k ¼ ð33Þ
b0 = 0) and the index of the last basis function in the last span is NB
1. 0 otherwise;
For a closed B-spline snake, the index needs to be wrapped properly
(Blake and Isard, 1998). where 0 6 i 6 n
1 for both open and closed snakes.
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 219

With the above formulation of finite difference snakes, toboggan-based intelligent scissors (Mortensen and Bar-
we have a uniform finite element construction for a variety rett, 1998; Mortensen, 2000), respectively.
of snake representations, which leads to a relatively Live-wire-like user interaction techniques have been
straightforward United Snakes implementation in an proposed in the snakes literature. In (Cohen and Kimmel,
object-oriented programming language, such as Java. 1997), Cohen and Kimmel compute the global minimal
path between two points using Sethian
s fast marching
3. Live wire algorithm (Sethian, 1997), which has sub-pixel accuracy2
and may eliminate metrication errors of graph search algo-
Live wire (or intelligent scissors) is a recently proposed rithms. A minimal path between two points is also
interactive boundary tracing technique (Mortensen et al., obtained in (Grzeszczuk and Levin, 1994) based on simu-
1995; Falcão et al., 1996, 1997; Falcão and Udupa, 1997; lated annealing. In a technique called ‘‘static’’ snakes, pro-
Mortensen and Barrett, 1998; Falcão et al., 1998; Falcão posed in (Neuenschwander et al., 1994), the user initially
et al., 2000). Although it shares some similarities with specifies two end snake points and then the snake takes
snakes – it was originally developed as an interactive 2-D image information into account progressively from the
extension to previous stage-wise optimal boundary track- two end points to its center, resulting in a minimal path
ing methods – it is generally considered in the literature between the two points. A similar technique has also been
as a competing technique to snakes. Like snakes, the idea proposed in (Hyche et al., 1992). Dubuisson-Jolly and
behind the live wire technique is to allow image segmenta- Gupta have formulated tracking an active contour with
tion to occur with minimal user interaction, while at the shape constraints in an image sequence as a shortest path
same time allowing the user to exercise control over the problem (Dubuisson-Jolly and Gupta, 2001).
segmentation process. However, live wire realizes the idea
differently from snakes. 3.1. Trace formation
Live wire is very easy to use. The user begins by placing
an initial seed point near the boundary of the object of Boundary finding in live wire can be formulated as a
interest. As the cursor, or free point is moved around the directed graph search for an optimal (minimum cost) path
image, the current calculated boundary, called the live wire using Dijkstra
s algorithm in the underlying graph model.
or trace, from the seed point to the free point is dynami- First, the graph is initialized with the local costs as
cally displayed. If the displayed trace is acceptable and described in the next section. Once the user selects a seed
the user clicks the mouse, the free point is collected as an point (node), it will be used as the starting point for a
additional seed point, and this trace will be frozen and will recursive expansion process. In the expansion process, the
become part of the extracted object boundary. The result- local cost at the seed point is summed into its neighboring
ing live wire boundaries are piecewise optimal (i.e., optimal nodes. The neighboring node with the minimum cumula-
between seed points), while the snake gives an optimal tive cost is then further expanded and the process produces
solution over the entire contour. a dynamic ‘‘wavefront’’. The wavefront expands in the
The genesis of live wire has its origin in the early col- order of minimum cumulative cost. Consequently, it prop-
laboration between Udupa (University of Pennsylvania) agates preferentially in directions of highest interest (i.e.,
and Barrett (Brigham Young University) (Mortensen along image edges).
and Barrett, 1998; Falcão et al., 1998). Their two research For any dynamically selected goal node (i.e., the free
groups have since independently developed different live point) within the wavefront, the optimal path back to the
wire systems. They share two essential components: a seed point which forms a live wire trace can be displayed
local cost function that assigns lower cost to image fea- in real time. When the cursor (the free point) moves, the
tures of interest, such as edges, and an expansion process old live wire trace is erased and a new one computed and
that forms optimal boundaries for objects of interest displayed in real time. The expansion process aims to com-
based on the cost function and seed points provided inter- pute an optimal path from a selected seed point to every
actively by the user. However, they employ different other point in the image and lets the user choose among
underlying graph models with different local cost func- paths interactively, based on the current cursor position.
tions. In (Mortensen and Barrett, 1998), each pixel repre- Live wire may be implemented very efficiently in multi-
sents a graph node, and directed, weighted edges are threaded programming languages, such as Java, because
created between each pixel and its eight adjacent neigh- the expansion process and the user interface can execute
bors. In (Falcão et al., 1998), the graph nodes are pixel in separate, parallel threads. Since the free point is gener-
corners and they are connected by oriented, weighted edge ally near the target object boundary, the expansion process
cracks, called boundary elements (bels for short). In both will most likely have already advanced beyond that point
cases, when the image is large, a corresponding large and the live wire trace can be displayed immediately. That
underlying graph may have to be maintained and live wire
performance will be compromised. To improve the effi- 2
Toboggan-based live wire (Mortensen and Barrett, 1999) obtains sub-
ciency of live wire, the two groups have developed exten- pixel localization by fitting an edge model to the tobogganed region
sions known as live lane (Falcão et al., 1998) and boundaries.
220 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

is, the live wire trace can typically be displayed before the is the normalized bidirectional link or unit edge vector be-
expansion process has finished sweeping over the entire tween pixels p and q.
image. Therefore, our implementation (Liang et al., Along with the gradient magnitude fG, pixel value fea-
1999a,b) is equivalent to the interleaved computation pro- tures (fP, fI and fO) are used in on-the-fly training to
posed in (Mortensen et al., 1995; Mortensen and Barrett, increase the live wire dynamic adaptation (Mortensen
1998) or live wire-on-the-fly introduced in (Falcão et al., and Barrett, 1998). With the typical gray-scale image pixel
2000) in terms of computation cost, and the multi-threaded value range [0, 255], they are defined as
Java implementation is more elegant in software design
1
and in supporting user interactions. fP ðqÞ ¼ IðpÞ; ð39Þ
255
1
3.2. Local cost functions fI ðqÞ ¼ Iðp þ k  DðpÞÞ; ð40Þ
255
1
Many local cost functions can be defined. In (Mortensen fO ðqÞ ¼ Iðp
k  DðpÞÞ; ð41Þ
et al., 1995), the local cost l(p, q) on the directed link from p 255
to a neighboring pixel q is defined as a weighted sum of six where D(p) is the unit vector of the gradient direction as de-
local component costs created from various edge features: fined above, and k is a constant distance value for deter-
mining the inside and outside features.
lðp; qÞ ¼ xZ fZ ðqÞ þ xG fG ðqÞ þ xD fD ðp; qÞ þ xP fP ðqÞ In (Falcão et al., 1998), the local cost assigned to each
þ xI fI ðqÞ þ xO fO ðqÞ; ð34Þ boundary element (bel) b is a linear combination of the
costs with its eight possible features fi:
where fZ(q) is the Laplacian zero-crossing function at
q, fG(q) is the gradient magnitude at q, fD(p,q) is the gradi- P8
wi cf ðfi ðbÞÞ
ent direction from p to q, fP(q) is the edge pixel value at lðbÞ ¼ i¼1P8 i ; ð42Þ
i¼1 wi
q, fI(q) and fO(q) are the ‘‘inside’’ and ‘‘outside’’ pixel values
at q, respectively, while xZ, xG, xD, xP, xI and xO are where wi is the associated weight with feature fi, and where
their corresponding weights. cfi , called the feature transform function of feature fi, con-
The Laplacian zero-crossing function fZ(q) is a binary verts feature value fi(b) into a cost value. The eight features
function defined as of a bel b include the intensity values on positive and neg-
 ative sides of b (f1 and f2), four different gradient magnitude
0 if I L ðqÞ ¼ 0;
fZ ðqÞ ¼ ð35Þ approximations (f3, f4, f5, f6), orientation-sensitive gradient
1 otherwise; magnitude (f7) and boundary distance (f8). Each feature va-
where IL(q) is the Laplacian of the image I at pixel q. The lue (fi, 1 6 i 6 8) may be converted into a cost value with
gradient magnitude serves to establish a direct connection any of the following six feature transforms: linear (c1), in-
between edge strength and cost. The function fG is defined verted linear (c2), Gaussian (c3), inverted Gaussian (c4),
as an inverse linear ramp function of the gradient magni- modified hyperbolic (c5), and inverted modified hyperbolic
tude G (c6). Training methods have been developed for optimum
selection of the bel features and automatic selection of
maxðG0 Þ
G0 G0
fG ¼ 0 ¼1
; ð36Þ the parameters with their feature transforms, based on
maxðG Þ maxðG0 Þ the typical segments painted by the user along the desired
where G 0 = G
min(G). When calculating l(p,q), the object boundary.
function fG(q) is furtherpscaled
ffiffiffi by 1 if q is a diagonal
neighbor to p and by 1= 2 if q is a horizontal or vertical 4. Combining snakes and live wire
neighbor.
The gradient direction fD(p, q) adds a smoothness con- Excluding user interaction, an accurate initialization is
straint to the boundary by associating a higher cost for generally needed in order for a snake to lock onto image
sharp changes in boundary direction. With D 0 (p) defined features of interest in all but the simplest images. There-
as the unit vector normal to the gradient direction D(p) fore, researchers have been actively investigating tech-
at pixel p (i.e., D(p) = [Ix(p), Iy(p)] and D 0 (p) = [Iy(p), niques to mitigate the sensitivity of snakes to their

Ix(p)]), the formulation of the gradient direction cost is initialization. Among these techniques are the use of an
2 inflation force (Terzopoulos et al., 1988; Cohen and Cohen,
fD ðp; qÞ ¼ farccos½d p ðp; qÞ þ arccos½d q ðp; qÞg; ð37Þ 1993), a chamfer distance map (Cohen and Cohen, 1993)
3p
and gradient vector flow (Xu and Prince, 1998). These tech-
where dp(p, q) = D 0 (p) Æ L(p, q) and dq(p, q) = L(p, q) Æ D 0 (q) niques can work well if the image feature map is relatively
are vector dot products and clean. However, most clinical images are noisy, contain
 many uninteresting edges, or texture is present. Hence,
1 q
p if D0 ðpÞ  ðq
pÞ P 0; these more automatic techniques can fail. For this reason,
Lðp; qÞ ¼ ð38Þ
kp
qk p
q if D0 ðpÞ  ðq
pÞ < 0 we explore an alternative direction – instead of attempting
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 221

to mitigate initialization sensitivity, we seek to increase the

efficiency of interactive initialization. In particular, we
enable the user to instantiate (i.e., construct, initialize,
and activate) snakes quickly and with minimal effort by
exploiting the strengths of the live wire technique.
In this section, we first justify the complementarity of
snakes and live wire, and then we show that the combina-
tion of snakes and live wire also provides an efficient mech-
anism for handling large images.
Fig. 1. (a) Delineation of the lung in X-ray fluoroscopy images using live
wire (seed points are shown). (b) A Hermite snake instantiated from live
4.1. Snakes and live wire are complementary wire traces with the first seed point imposed as a hard constraint. It is
interactively pulled out of the strong edge with spring forces and then
There are numerous ways to define the local cost func- locks onto the lung boundary.
tions in live wire, as long as sufficiently low cost values
are assigned to the desired object boundaries. Therefore, 1998). Basically, these techniques (dynamically) update
various techniques developed for computing snake poten- the cost map to filter out the image features which do
tials (Kass et al., 1988; Blake and Isard, 1998) can be used not have edge characteristics similar to the sample bound-
for the generation of local cost maps in live wire. For ary specified by the user. In other words, these methods
instance, the chamfer distance map (Cohen and Cohen, rely on the assumption that the edge property is relatively
1993) and gradient vector flow (Xu and Prince, 1998) are consistent along the object boundary. Training is most
readily usable. Similarly, the local cost map computed for effective for those objects with relatively consistent bound-
live wire may be treated as an image potential in snakes. ary properties and may be counter-productive for objects
Therefore, in United Snakes, snakes and live wire may with sudden and/or dramatic changes in their boundary
share the same image potential (local cost map). properties (Mortensen and Barrett, 1998). For example,
In general, live wire provides user-friendly control dur- in the lung image of Fig. 1(a), the live wire snaps to the
ing the segmentation process. The user may freely move strong edges of the elliptical viewport rather than the
the free point on the image plane, and the corresponding desired lung boundary. In this case, training is ineffective
live wire trace is interactively displayed in real time. How- since the edge property of the lung boundary varies consid-
ever, once the free point is collected as an additional seed erably over its extent and is also disturbed by the ribs (not
point, the trace is frozen and it becomes a part of the obvious to the eye). Consequently, it is difficult to specify a
extracted object boundary. At this point, the user has no typical segment of the lung boundary. Nevertheless, in situ-
further control over the trace between seed points other ations where training can be effective, snakes can also take
than backtracking. Therefore, when the shape of the object advantage of it by utilizing the trained cost map. More-
boundary is complex or when object boundaries are noisy over, in United Snakes, the user has more control, using
and unclear, the user may need to backtrack to produce spring forces to pull the snake out of one minimum into
acceptable traces. Consequently, many seed points may another without training, as shown in Fig. 1(b).
be needed to guide the live wire to an accurate result. Fur- The underlying graph search makes it possible for live
thermore, it is not uncommon for the user to make small wire to bridge boundary gaps and pass through noisy
errors when placing seed points using a mouse or other areas. For instance, in MR wrist images from Falcão
input device, forcing the user to repeat the placement. By et al. (2000) (Fig. 2), live wire bridges the gap along the ves-
contrast, a snake may be dynamically adjusted or refined sel boundary in Fig. 2(a) and passes through a noisy region
as it deforms at any time and at any point on the snake in Fig. 2(b). Even if there are no image features at all
via intuitive mouse-controlled forces. However, the best between two points, live wire can still provide a minimal
performance of the snake is often achieved when user-spec- path – a straight line. However, live wire is inherently
ified constraint points are utilized. The constraint points do
not ‘‘lock in’’ the solution – they too may be changed
dynamically, allowing further refinement of the extracted
object boundary.
Live wire seeks a global minimal path between two
points. Therefore, when a section of the desired object
boundary has a weak edge relative to a nearby strong but
uninteresting edge, the live wire snaps to the strong edge
rather than the desired weaker boundary. In order to mit-
igate this problem, Falcão et al. (1998) have developed
training techniques for optimum feature selection and
automatic parameter selection, and Mortensen and Barrett Fig. 2. Live wire bridges the gap along the vessel boundary (a) and passes
have proposed on-the-fly training (Mortensen and Barrett, through a noisy region (b) in an MR wrist image.
222 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

impose smoothness on live wire traces, and bridge compli-

cated gaps along object boundaries. This is what snakes are
good at doing. Snakes adhere to edges with sub-pixel accu-
racy and they may also be adjusted interactively as para-
metric curves with intuitively familiar physical behaviors.
Furthermore, snakes have the power to track moving
objects, while live wire does not.
However, the efficient performance of interactive snakes
is linked to fast, reasonably accurate initialization and
user-specified constraints. Even with a few seed points, live
wire can quickly give much better results than casual man-
ual tracing. Hence, the resulting live wire boundary can
serve well to instantiate a snake. The live wire seed points
reflect the user
s prior knowledge of the object boundary.
They can therefore serve as either hard or soft point con-
straints for the snake, depending on the user
s confidence
in the accuracies of these seed points.
Fig. 3. (a) Delineation of the heart in X-ray fluoroscopy images using live
Because a live wire-traced initial object boundary is
wire (seed points are shown). (b) The unacceptable segment replaced by
manual drawing. Alternately, the user may place multiple seed points and more accurate than a hand-drawn boundary, and with
let live wire generate a piecewise linear path between adjacent seed points the further incorporation of the seed points as snake con-
to approximate the missing cardiac boundary. (c) Initial B-spline snake straints, the snake will very quickly lock onto the desired
and control polygon instantiated from live wire traces in (b). (d) Resulting object boundary. If necessary, the user may then correct
segmentation after a few iterations with control point 3 as a hard
mistakes inherited from the live wire-generated boundary
constraint, which effectively bridges the gaps along the cardiac boundary.
by applying mouse-controlled spring forces to the snake.
Because the user still has the opportunity to correct the
image-based, rather than model-based. Fundamentally, it deficiencies of the trace as the snake is evolving, the num-
is not designed to bridge gaps in a manner that is consistent ber of seed points needed by live wire to generate the object
with the image features bordering the gaps and the smooth- boundary can be further reduced. Consequently, a satisfac-
ness of the traces cannot be guaranteed. For instance, in tory initial object boundary can be generated very quickly
Fig. 3(a), part of the live wire trace from seed point 1 to using live wire. Other hard or soft constraints may be
seed point 2 is a straight line where the cardiac boundary added during the snake deformation process as well.
is missing, and the live wire technique does not generate Because constrained values may be changed dynamically,
an acceptable cardiac boundary from seed point 3 to seed the user may adjust the seed points to further refine the
point 1. The user may place multiple seed points and let live object boundary as the snake deforms.
wire generate a piecewise linear path between adjacent seed To illustrate their performance, we apply United Snakes
points to approximate the missing cardiac boundary. In to an angiogram (Fig. 4) and a vertebra image (Fig. 5), to
this case, we have found that it is convenient to draw a which Mortensen and Barrett applied their live wire algo-
rough curve manually between the points (Fig. 3(b)). rithm in (Mortensen and Barrett, 1998). With only a few
Snakes, on the other hand, are model-based and were seed points, United Snakes generate the boundaries shown
designed to adhere to image edges and interpolate between in Figs. 4 and 5(c), which are comparable to the ideal
edge features in regions of sparse and noisy data (i.e., fill in boundaries used as references in (Mortensen and Barrett,
the gaps). For example, a B-snake instantiated from the 1998).
live wire traces is more effective in bridging the gaps along As further evidence that the United Snakes framework
the cardiac boundary, as shown in Fig. 3(d). improves upon the robustness and accuracy of its compo-
In summary, it is desirable to enable the user to exercise nent techniques, Fig. 6 shows a synthetic image of a known
more control over the live wire traces between seed points, curve degraded by strong Gaussian white noise (variance

Fig. 4. Segmenting a vessel in an angiogram. (a) The image used in (Mortensen and Barrett, 1998). (b) Live wire segmentation. (c) United Snakes generate
boundaries comparable to ideal boundaries in (Mortensen and Barrett, 1998).
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 223

Fig. 5. Segmenting the outer boundary of a vertebra. (a) The image used in (Mortensen and Barrett, 1998). In United Snakes, we only expect a coarse
object boundary from live wire. To illustrate this point, referring to Eq. (35), we have set xG = 0.50, xZ = 0.5, and turned all the other parameters off (i.e.,
xD = xP = xI = xO = 0), resulting in the live wire segmentation (b). From it, United Snakes generate a boundary (c) which is comparable to the ideal
boundary in (Mortensen and Barrett, 1998).

Fig. 6. Performance of United Snakes demonstrated using a noisy synthetic image. This image was designed to challenge the snakes and live wire with the
high curvature points as well as the small wave details. (a) A live wire is sensitive to noise (the required seed points are shown). (b) United Snakes are
robust against noise. (c) The segmented boundary accurately conforms to the ideal boundary.

0.25). Given its image-based nature, the live wire is sensi- For effective user interaction, the thread responsible for
tive to noise as shown in Fig. 6(a). A snake instantiated computing an optimal path from the seed point to every
by the live wire gives a better result (Fig. 6(b)). Fig. 6(c) other point in the image should not stop until the user
shows that the United Snakes result is very close to the has selected the current free point as a seed. This ensures
boundary in the ideal image, despite the strong noise. This the user may move with more freedom in the image plane
performance is a consequence of the imposed hard con- to select optimal paths and quickly generate an acceptable
straints, without which the snake would slip away from object boundary with a minimal number of seed points.
high curvature points. That is, the user should be able to place two neighboring
seed points as far apart as desired. However, when the
4.2. Handling large images desired object boundary is not clear/sharp (e.g., chest
images, mammograms, etc.) or has many branches (e.g.,
Large images are now common in clinical settings a retinal angiogram), the wavefront will spread too widely
because high resolution is often needed to make accurate and include many pixels for any path of reasonable length.
diagnoses. For instance, in the mammogram analysis task As a result, the memory required to maintain the auxiliary
(see Section 6.4), we need to handle images with a typical information in Dijkstra
s algorithm will increase dramati-
resolution of 3500 · 6500 pixels. However, due to the nat- cally for large images. Furthermore, as demonstrated also
ure of its underlying graph-based algorithm, the basic live in (Falcão et al., 2000), when the image size changes from
wire algorithm is unable to handle large images efficiently. 128 · 128 to 1024 · 1024, the live wire performance will be
To support user interaction, live wire aims to compute an reduced by a factor of 400, and the ultra fast live wire on
optimal path from the last seed to every other point in the fly may still be 40 times slower.
the image. Even with our efficient multi-threaded Java The combination of live wire and snakes in United
implementation of Dijkstra
s algorithm (e.g., with bucket Snakes provides a new mechanism for handling large
sort (Mortensen and Barrett, 1998) or Dia
s method (Fal- images. The computational complexity of snakes is O(n)
cão et al., 2000)), the performance of live wire will be sig- in each iteration, where n is the number of snake nodal
nificantly compromised when working with large images. variables. In United Snakes, we typically require live wire
The reason is that the lower bound of its computational to generate only a coarse boundary with a few seed points.
complexity is O(m), where m is the number of image pixels Therefore, we can construct a truncated pyramid of images,
involved in the computation of an optimal path from the and let the live wire work at the top of the pyramid with a
seed to the free point; that is, all the pixels within the wave- small image size (for example, 128 · 128 or 256 · 256), thus
front (i.e., the expansion process). In the worst case, m is efficiently supporting user interaction. The snake ‘‘des-
the total number of pixels in the image. cends’’ the image pyramid from coarse to fine levels of
224 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

resolution, tolerating any live wire errors introduced at the As a demonstration, Fig. 7(a) shows a retinal angio-
top of the pyramid, and accurately locks onto the desired gram with pixel resolution of 256 · 256 obtained by
object boundary. The original large image is still displayed down-sampling the original large image with resolution
to the user and thus the seed points can generally be accu- 1024 · 1024. Suppose we would like to trace the vessel
rately specified or dynamically adjusted if necessary. The starting from point S to one of the target points 0–9
extra memory needed to maintain the pyramid is offset (see Fig. 7(b)). Once point S is selected as the first seed,
by the reduced memory necessary for the auxiliary infor- the corresponding branch should ideally be instantly
mation in Dijkstra
s algorithm. In practice, we do not have available once the user points to any of the 10 branch
to maintain a pyramid for the original image, but only for end-points. This real-time user interaction is achievable
the image potential. Assuming the pyramid has n levels and by live wire when the image size is under 300 · 300 on
the original image occupies M amount of memory, the modest PCs (Fig. 7(c)). For larger images, however,
extra memory required for the pyramid then is real-time user interaction becomes increasingly difficult
    to achieve using live wire alone. Table 1 shows the time
1 1 1 1 1 M
þ þ þ    þ ðn
1Þ M ¼ 1
ðn
1Þ ; ð43Þ needed for the wavefront to reach the 10 targets as well
4 42 43 4 4 3
as the time needed to sweep over the entire image at dif-
while the reduced auxiliary memory (e.g., only for the ferent resolutions on an 866 MHz Pentium PC with SUN
cumulative cost map) in the live wire implementation is JDK1.3. From the table, we can see that the time
1
ð1
4ðn
1Þ ÞM. required at 256 · 256 resolution is approximately 1/4 of

Fig. 7. (a) A retinal angiogram with pixel resolution 256 · 256 obtained by down-sampling the original 1024 · 1024 image. (b) The seed point S and 10
marked target points. (c) The superimposed live wire traces shown only for the first and last target points. (d) The superimposed snakes dynamically
instantiated from the live wire traces in (c) descend the truncated pyramid reaching the intermediate level with resolution 512 · 512 (e) and the original
large image (f). This mechanism supports real-time user interaction: once point ‘‘S’’ is selected as a seed, the corresponding vessel branch is instantly
available when the user points to a new position (such as, the 10 targets) on the original large image.
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 225

Table 1
The time (in milliseconds) needed for the wavefront to reach the 10 targets shown in Fig. 7, as well as the time needed to sweep over the entire image at
different resolutions on an 866 MHz Pentium PC with SUN JDK1.3
Targets 0 1 2 3 4 5 6 7 8 9 Entire image
256 · 256 65 109 118 131 131 170 205 211 225 240 405
512 · 512 374 526 545 604 805 875 979 1025 1081 1248 2178
1024 · 1024 1562 2293 2393 2703 3479 3683 4425 4543 4688 5481 9802
The time required at 256 · 256 resolution is approximately 1/4 of that at 512 · 512 resolution, which is roughly 1/4 of that at 1024 · 1024 resolution. The
time needed for a snake sliding down is O(n) in each iteration, where n is the number of snake nodal variables. We use five iterations at each level of the
pyramid. The longest snake (from point S to point 9 in Fig. 7) in this experiment has 100 nodal variables. When the live wire trace is available at the top
level (256 · 256), it takes 15 iterations or 77 ms for the snake to descend the pyramid. So, the total time needed is 317 (77 + 240) ms, which is much less
than 5481 ms when working directly at the resolution of 1024 · 1024.


that at 512 · 512 resolution, which is roughly 1/4 of that 1 if 2i ¼ j;
K2i;j ¼ ð44Þ
at 1024 · 1024 resolution. This can be justified by the 0 otherwise;
observation that reducing an image by a factor of 2 in
linear dimension while maintaining its aspect ratio where 0 6 j 6 2(N
1) and N is the number of snake
reduces its area in pixels to 1/4, and that the complexity nodes. The system force vector F is updated as
of the wavefront computation is proportional to the lat- Fx2i ¼ xci ; Fy2i ¼ y ci ; ð45Þ
ter. Thus, with a three-level pyramid, we can make the
algorithm approximately 16 times faster and, with four where x and y indicate coordinate function x(s) and y(s),
levels, it becomes approximately 64 times faster. respectively. It is then guaranteed that the snake node i is
In United Snakes, snakes that are dynamically instanti- always at position ðxci ; y ci Þ.
ated from live wire traces at the top of the truncated image A drawback of this simple technique, however, is that
pyramid can easily descend the pyramid, reaching the ori- the updated system stiffness matrix is no longer symmetric.
ginal large image (Fig. 7(f)) via intermediate level(s) Consequently, we are unable to store the stiffness matrix
(Fig. 7(e)), resulting in real-time user interaction on the ori- economically using skyline storage, nor factorize it into
ginal large image. Thus, United Snakes with the image LDLT form (see Appendix A). Nevertheless, since the posi-
pyramid scheme yields real-time response – a critical factor tion of node i is given, a constant force may be derived
in any interactive segmentation scheme – with sub-pixel from the stiffness matrix for each degree of freedom and
accuracy in original large images. subtracted from its corresponding position in the system
force vector so that we can restore the symmetry of the
5. Hard constraints stiffness matrix while keeping the system in balance. In
our implementation, we store column 2i of K into a vector
Our combination of snakes and live wire relies on an k2i; i.e., k2i
j ¼ Kj;2i , for 0 6 j 6 2(N
1), before K is made
efficient constraint mechanism. A constraint on a snake symmetric with

may be either soft or hard. Hard constraints generally com- 1 if 2i ¼ j;
pel the snake to pass through certain positions or take cer- Kj;2i ¼ ð46Þ
0 otherwise.
tain shapes, whereas soft constraints merely encourage a
snake to do so. Two kinds of soft constraints, springs To keep the system in balance, the system force vector F is
and volcanos, were described in the original snakes paper updated with
(Kass et al., 1988) and they are incorporated into our finite Fxj ¼ Fxj
xci k2i Fyj ¼ Fyj
y ci k2i ð47Þ
j ; j
element formulation. Hard constraints have been used to
prevent snake nodes from clustering in dynamic program- for 0 6 j 6¼ 2i 6 2(N
1). We can constrain the slope in the
ming snakes (Amini et al., 1990). Generic hard constraints same way. If we constrain two node variables of an element
are discussed in (Fua and Brechbühler, 1997,). In this sec- in both position and slope, this element will be frozen. Its
tion, we propose a convenient mechanism, called pins, as a two neighboring elements will also be influenced by the
simple yet effective way to impose hard constraints on constraint. The constraints on a B-snake are imposed on
snakes for the integration of snakes and live wire. the nodes of its control polygon. Imposing hard constraints
Suppose that we wish to guarantee that the snake node i in this manner also lessens computational cost, in terms of
sticks at position ðxci ; y ci Þ in the Hermitian parameteriza- both memory and time, since the number of entries in the
tion. Recall that in the Hermitian parameterization, the skyline storage of the stiffness matrix is reduced. Conse-
polynomial shape of each element is parameterized by the quently, the LDLT factorization and forward/backward
position and slope of x(s) and y(s) at the two nodes (posi- substitutions can be performed more efficiently (see Appen-
tion and slope variables occupy alternating positions in the dix A). It is also possible to apply more general constraints
nodal variable vector u). Therefore, the snake stiffness to any point on the snake as is described in (Terzopoulos
matrix K may be updated with and Qin, 1994).
226 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

In the formulation above, the updated stiffness matrix

only indicates which degrees of freedom of the snake are
constrained, it does not contain any constraint values.
These are recorded in the system force vector. As a result,
the constraint values may be updated dynamically during
snake deformation. Hence, the user can move the con-
straint points around the image plane to refine the object
boundary as the snake is deforming. This property is very
useful when integrating snakes with live wire. While a
snake is deforming, additional hard constraints may be
imposed on the snake to restrict its deformation. Because
these constraints are unknown before the snake is instanti-
ated, they may be incorporated on-the-fly using reaction
Fig. 9. Segmenting the corpus callosum in an MR head image of a human
forces on the system force vector without changing the stiff-
volunteer. (a) The live wire snaps to the nearby strong edge. (b) An
ness matrix. However, small time steps are required to additional seed bridges the missing boundary. (c) The corresponding snake
ensure the stability of the snake. In our implementation, with the seed points naturally imposed as hard constraints nicely captures
we create a new snake from the current snake plus the hard the desired object. (d) When releasing the hard constraints, the strong
constraints, since the LDLT factorization is fast. image force in the region of region A will gradually drive the snake away
from the desired boundary, while, in region B, the snake will become
Hard constraints play very important roles in capturing
insufficiently peaked due to its internal energy.
the intricate details and bridging gaps along object bound-
aries in image segmentation. For instance, to segment the
bladder from an MR image of a female abdomen shown any further user intervention as shown in Fig. 8(d). Hard
in Fig. 8, neither the live wire (Fig. 8(a)) nor its correspond- constraint points may be adjusted to refine the object
ing, instantiated dynamic snake (Fig. 8(b)) would be able to boundary. For illustration purposes, hard constraint point
capture the intricate details indicated by the rectangle, 1 has been deliberately moved far away from the desired
which require additional seed points (Fig. 8(c)). With all bladder boundary in Fig. 8(e). Releasing the hard con-
the seed points imposed as hard constraints, the corre- straints will lose the details as shown in Fig. 8(f).
sponding snake accurately captures the details without The desired object boundary might be unclear or even
missing in many clinical images. For example, in segment-
ing the corpus callosum in an MR head image of a human
volunteer, the live wire snaps to the nearby strong edge
9(a), and additional seed points are required to bridge the
missing boundary 9(b). These seed points can be naturally
imposed as hard constraints on the corresponding snake,
which nicely captures the desired object in Fig. 9(c). When
releasing the hard constraints, the strong image force in
region A (see Fig. 9(d)) will gradually drive the snake away
from the desired location, while the snake will become
insufficiently peaked in region B due to its internal energy.

5.1. Static vs. dynamic constraint integration

We have argued that a hard constraint mechanism is

crucial in practical image segmentation. Live wire generally
requires seed points at the critical, complicated locations
where the desired boundary is twisted, unclear, weak or
even missing. These seed points, interactively provided by
the user to guide the live wire, capture the user
s expert
prior knowledge about the desired object boundary, and
Fig. 8. Segmenting the bladder in an MR image of a female abdomen. they can naturally be imposed as hard constraints on the
Neither the live wire (a) nor its corresponding dynamic snake (b) would be snake that is then instantiated from the complete live wire
able to capture the intricate details indicated by the rectangle without trace. We refer to this form of livewire-snake integration as
additional live wire seed points (c). (d) With all the seed points naturally static integration – once the live wire result is used to
imposed as hard constraints, the corresponding snake accurately captures
instantiate a snake, the segmentation process continues
the fine, intricate details without any further user intervention. (e) Hard
constraint point 1 is deliberately moved far away from the desired bladder using only the constrained, user-controlled snake.
boundary to illustrate the adjustment capability. (f) Releasing the hard A more dynamic constraint integration ‘‘mode’’ is often
constraints will lose the details. useful – once the live wire trace between the last seed point
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 227

Fig. 10. Using United Snakes in dynamic mode to segment neuronal EM images. (a) Live wire boundary showing three seed points and free point
(rectangle indicates a problem area). (b) Open snakes dynamically generated from the live wire traces and constrained by seed and free points. (c) Third
snake corrected in the problem area using the mouse.

and the free point is formed, a corresponding open snake

with constraints at the seed point and the free point is
instantiated and set in motion. When the free point is cho-
sen and collected as a seed point, this open snake is merged
with the snake (if any) instantiated from previous live wire
traces. All seed points are automatically applied as con-
straints. Fig. 10 illustrates this process where ‘‘+’’ indicates
the current free point. The live wire and snake results are
shown separately in the neuronal EM images in Figs.
10(a) and (b), respectively. Since the snake is automatically
set in motion, the user may use the mouse-controlled
springs to adjust it in any problematic areas along the
snake trace (Fig. 10(c)).

6. Applying United Snakes

In this section, we apply United Snakes to several differ-

ent medical image analysis tasks, demonstrating the gener-
ality, accuracy, robustness, and ease of use of the tool.

6.1. Segmenting neuronal dendrites in EM images

A neuronal dendrite is the receiving unit of a nerve cell.

The area of contact between the dendrites of different cells
is called a synapse and is located on the dendritic spines. In
humans, morphological changes in dendritic spines are Fig. 11. Using United Snakes in static mode to segment neuronal EM
seen with aging and with diseases that affect the nervous images. (a) Approximate live wire boundary using just three seed points
(rectangle indicates a problem area). (b) Additional seed points can
system, such as dementia, brain tumors and epilepsy (Carl-
improve live wire
s accuracy. (c) Instantiated from the live wire traces in
bom et al., 1994). Detailed anatomical models of dendritic (b), the snake tolerates live wire errors and locks on cell boundary without
spines and their synapses will provide new insights into further user interaction. (d) Instantiated from the live wire traces in (a), the
their function, thus providing better opportunities to snake ‘‘sticks’’ in the problem area, but it is easily adjusted (e) using the
understand the underlying causes and effects of these dis- mouse. (f) Snake adjustment capability illustrated by moving constraint
point 2.
eases. To build such models, the dendrite must be seg-
mented from the surrounding tissue in positive electron
micrography (see Carlbom et al., 1994 for a detailed the free point. If the displayed trace is acceptable, the free
description of how snakes are used in reconstruction of point is collected as an additional seed point. For example,
3D nerve cell models from serial microscopy). Here, we we can capture an approximate cell boundary in Fig. 11(a)
are interested in localizing nerve cell membranes, which with just three seed points.
appear dark in positive micrography. The live wire tends to stick to the object boundary using
In the United Snakes system, the user begins an image the seed points as a guide. The trace between the two adja-
segmentation task using a live wire. An initial seed point cent seed points is frozen. The user has no further control
is placed near the boundary of the object of interest. As over these traces other than backtracking. In order to gen-
the cursor, or free point, is moved around, the live wire, erate a more accurate result in the area indicated by a rect-
or trace, is interactively displayed from the seed point to angle, more seed points may be placed as in Fig. 11(b).
228 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

Although the live wire boundary is somewhat jagged and

exhibits some small errors, it is in general as accurate as
manual tracing, but more efficient and reproducible.
Next, we instantiate a snake from the live wire-gener-
ated boundary and use the seed points to constrain it.
The user may select a shape function for the snake which
is suitable for the object boundary. In our cell segmenta-
tion example, if the live wire result with five seed points
is used to instantiate a finite difference snake, it is able to
tolerate the live wire errors and very quickly and accurately
lock onto the cell boundary without any further user inter-
action (Fig. 11(c)). Using the live wire result with three seed
points, the snake becomes ‘‘stuck’’ in the problematic area
(Fig. 11(d)) due to the live wire-generated boundary errors.
However, this situation can be easily remedied using the
mouse spring (Fig. 11(e)). Furthermore, as the snake is
deforming, the hard constraints may be adjusted to refine Fig. 12. Dynamic lung delineation with United Snakes. (a) An automat-
ically instantiated Hermite snake. (b) United Snakes result for the left lung
the snake boundary. In Fig. 11(f), for example, constraint with three seed points. (c) The final result for both lungs. (d) Releasing
point 2 is moved to illustrate this snake boundary adjust- hard constraints except those at both lung apices for lung motion tracking.
ment capability. By contrast, it is not nearly as easy to
adjust a seed point in the live wire algorithm.
In summary, the information from live wire including interactively for the first image in the sequence. Fig. 12(a)
the user guidance and expert prior knowledge is fully uti- shows the first dynamically instantiated Hermite snake
lized by the snake; the snake very quickly locks onto the with two end points (seeds) applied as hard constraints.
image features of interest with reasonable tolerance to mis- Three seed points are sufficient for delineating the left lung
takes in the live wire traces. (Fig. 12(b)), similarly, for the right lung shown in
Fig. 12(c).
6.2. Dynamic chest image analysis In the dynamic integration method, all seed points are
automatically applied as hard constraints. Although hard
The aim of the dynamic chest image analysis task is to constraints can be dynamically adjusted, for motion
show focal and general abnormalities of lung ventilation tracking it is not convenient to perform hard constraint
and perfusion based on a sequence of digital chest fluoros- adjustments in each frame. Therefore, the United Snakes
copy frames collected over a short time period (typically system allows the user to add or release hard constraints
about 4 s) (Liang, 2000; Liang et al., 1997a,b, 1998, 2001, dynamically. The edge information at the lung apex is
2003). The project uses only plain X-ray fluoroscopy for very weak and there is no observable motion in quiet
the ventilation and perfusion studies; the radiation dose breath. Consequently, it is desirable to maintain a hard
to patients is low and, unlike a nuclear medicine scan, no constraint there. All other hard constraints are released
preparation is required before the examination and radio- for lung motion tracking. Fig. 12(d) shows a Hermite
active isotopes are unnecessary. The information gleaned snake with the first seed imposed as hard constraint for
from these images is helpful in several aspects of cardiotho- each lung.
racic radiology. Diseases directly related to the parameters We apply the snake motion tracking mechanism on the
being measured include pulmonary embolism, pulmonary entire image sequence, resulting in the registration of the
emphysema, cardiac failure, congenital heart disease and lung from one frame to another. Since the first seed is
other diseases (tumors, obstructive lesions or infections) applied as hard constraint, the snake can firmly stick at
which may change pulmonary ventilation and/or perfu- the apex, although the edge information there is rather
sion. The shapes and motions of the lung and heart give weak. Fig. 13 illustrates the tracking result for every fifth
an indispensable clue to ventilation and perfusion examin- image.
ations. Therefore, an essential first step for ventilation and In the case of the heart, we first employ the static inte-
perfusion analysis is the delineation of the lungs and the gration method for heart boundary tracing with the B-
heart from each frame in a chest image sequence. The Uni- spline shape function in the first image. A least squares
ted Snakes system has been used for this purpose. Typically approximation to the initial curve shown in Fig. 3(b) with
most of the user interactions to initialize and edit the snake a cubic B-spline with 5 knots can be used as an initializa-
are applied to the first image of the sequence only. The tion to a B-snake in Fig. 3(c). A hard constraint may be
resulting snake is then propagated and deformed through further imposed on control polygon node 3 to effectively
the remaining frames of the image sequence. bridge the gap along the heart boundary. The resulting
We employ the dynamic integration mode, which was B-snake for the first frame (Fig. 3(d)) is then used to track
described in Section 5.1, to delineate the lung boundaries the heart motion through the whole image sequence (see
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 229

Fig. 13. Lung motion tracking result for every fifth frame.

Fig. 14). Since, in this patient orientation, there is no signif- node 3. In the case of cardiac motion tracking, the hard
icant motion with the missing cardiac boundary, it is desir- constraint is not only effective for single images but also
able to apply a hard constraint on the control polygon for the entire image sequence.

Fig. 14. Cardiac motion tracking result for every fifth frame.
230 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

Fig. 15. Quantifying growth plates in MR images. (a) An MR growth plate image. (b) The live wire results. (c) The United Snakes results.

6.3. Quantifying growth plates in MR images of the lines and the small distance between each pair of
lines. However, live wire can be used to generate quickly
The aim of the growth plate image analysis task is to an acceptable snake initialization with just two or three
determine the right time for surgery for patients with seed points as shown in Fig. 15(b). In the final results
abnormal growth of the legs. To this end, the four tiny shown in Fig. 15(c), two hard boundary conditions are
(essentially horizontal) lines in the image (Fig. 15(a)) must applied on each of four finite difference snakes.
be detected to quantify the growth plate.
In this scenario, it is difficult for the user to trace an ini- 6.4. Isolating the breast region in mammograms
tial contour for a snake manually because of the small size
The goal of the mammogram project is to use pattern
recognition techniques to detect abnormalities in the breast
tissue. The mammograms we are handling are very large
with a typical resolution of 3500 · 6500 pixels, requiring
about 30 MB of disk space. For effective and efficient
abnormality detection, it is essential to isolate the breast
region from the background (Ojala et al., 2000, 2001).
For instance, the original mammogram in Fig. 16 has a res-
olution of 3691 · 6466. In United Snakes, we can achieve
the real-time interactive segmentation of the breast region
on the original mammogram with only two or three seed
points using the truncated image pyramid technique pro-
posed in Section 4.2.

7. Discussion and conclusion

It is concluded in (Falcão et al., 1998) that the main

goals of research in interactive segmentation methods
are (i) to provide as complete control as possible to
the user of the segmentation process while it is being exe-
cuted and (ii) to minimize the user
s intervention and the
total user time required for segmentation. The entire seg-
mentation process may be thought of as consisting of
two tasks: recognition and delineation. Recognition deter-
mines roughly where the object (boundary) is, while
delineation defines precisely the spatial extent of the
object region/boundary in the image. For practical appli-
cations, we have found that an additional task – refine-
ment – is essential. The errors in reproducibility occur
mostly in the vicinity of seed points (Mortensen and Bar-
rett, 1999). In United Snakes, both live wire traces and
hard constraint points can be interactively adjusted for
refinement. Furthermore, dynamically instantiated snakes
can tolerate the live wire errors and thus reduce the
number of the seed points which are interactively given
Fig. 16. Real-time isolation of the breast region in mammograms from a by the user. In other words, United Snakes provides
3691 · 6466 pixel image using only three seed points. more complete control to the user while further
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 231

minimizing the user
s intervention in the interactive seg- provided courtesy of Eric Mortensen of Brigham Young
mentation process. University. The bladder image and the corpus callosum
In summary, our United Snakes framework unites sev- images were provided courtesy of Johannes Hug of Swiss
eral snake variants with live wire to provide a general pur- Federal Institute of Technology, Switzerland. The MR
pose tool for interactive medical image segmentation and wrist images were provided courtesy of Dr. Alexandre X.
tracking. The union of these techniques amplifies the effi- Falcão of State University of Campinas, Brazil.
ciency, flexibility and reproducibility of the component
techniques. The United Snakes technique offers more con- Appendix A. Finite element Snakes formulation
trol for relatively less user interaction. As it quickly locks
onto the image features of interest with reasonable toler- The two coordinate functions x(s, t) and y(s, t) of the
ance to errors in live wire, the snake fully exploits the user snake v(s, t) are independent, we shall develop the finite ele-
guidance and expert prior knowledge captured by the initial ment formulation and the corresponding matrix equations
live wire trace and the seed points. We have demonstrated in terms of only one component x(s, t). An identical form
the generality, accuracy and robustness of United Snakes will be assumed for component y(s, t). We apply Galerkin
s
in applications ranging from the segmentation of neuronal method to the Euler–Lagrange equation for x(s, t):
dendrites in EM images, to the analysis of dynamic chest    2 
o2 x ox o ox o2 ox
images, to the quantification of growth plates, to the isola- l 2 þc
a þ 2 b 2
qðxÞ ¼ 0; ðA:1Þ
tion of the breast region in mammograms, among other ot ot os os os os
examples. We believe that United Snakes are in several ways which expresses the necessary condition for the snake at
superior to live wire or snakes alone. equilibrium. The average weighted residual is
We the creators of the United Snakes, in order to form a Z L 2    2  
more perfect union of snake technologies, plan to incorpo- ox ox o ox o2 ox
I¼ l 2 þc
a þ 2 b 2
qð x Þ
rate within our framework, affine cell image decomposition 0 ot ot os os os os
methods for snake topological adaptability (McInerney  wðsÞ ds ¼ 0; ðA:2Þ
and Terzopoulos, 2000), advanced snake motion tracking
where w(s) is an arbitrary test function. By performing inte-
mechanisms (Terzopoulos and Szeliski, 1992; Blake and
grations by parts once for the third term and twice for the
Isard, 1998), generic hard constraint mechanisms (Fua
fourth term of (A.2), we arrive at the weak formulation of
and Brechbühler, 1997; Fua, 1997), automatic learning
the snake model:
and adaptation of shape functions to specific images, and
Z L Z L Z L 
other snake techniques. We anticipate that such efforts will o2 x ox ow ox
further enhance the effectiveness of this image segmenta- wl 2 ds þ wc ds þ a ds
0 ot 0 ot 0 os os
tion tool. Z L 2  Z L
o w o2 x
þ b ds
wq ds þ b ¼ 0; ðA:3Þ
Acknowledgments 0 os2 os2 0

where
This work was carried out at the Turku Centre for Com-   2  L
puter Science, Turku, Finland, and in the Department of ox o ox ow o2 x
b ¼
wa þ w b 2
b 2 ðA:4Þ
Computer Science, University of Toronto, Toronto, Can- os os os os os 0
ada. J. Liang thanks the Turku Centre for Computer Sci- are the boundary conditions at the two boundary points,
ence and the Science Foundation of Instrumentarium s = 0 and s = L. We approximate x(s, t) as
Corporation (now GE Healthcare) for their financial sup-
port, as well as the Faculty of Mathematics and Natural xðs; tÞ ¼ NðsÞuðtÞ; ðA:5Þ
Sciences of the University of Turku for a faculty research where N(s) = [N1(s), N2(s), . . ., Nn(s)] are the shape func-
award that supported the dynamic chest image analysis pro- tions and u(t) = [u1(t), u2(t), . . ., un(t)]T are the n nodal vari-
ject. The authors gratefully acknowledge the insightful com- ables (degrees of freedom) of the snake model, implying the
ments and suggestions of Prof. Alexandre X. Falcão, Prof. derivatives of x(s, t) are
Pascal Fua, Prof. Dariu M. Gavrila, Prof. Timo Järvi, Prof.
Ron Kimmel, Dr. Aaro Kiuru, Prof. Martti Kormano, o2 x ox ox oN o2 x o2 N
¼ N€u; ¼ Nu;
_ ¼ u; ¼ 2 u. ðA:6Þ
Prof. Eric Mortensen, Dr. Erkki Svedström, Prof. Jayaram ot2 ot os os os2 os
K. Udupa, and the anonymous referees. The chest images In Galerkin
s method, the arbitrary test function w takes
were acquired by Dr. Raimo Virkki and provided by Dr. the form
Aaro Kiuru. The growth plate image was provided by Prof.
Martti Kormano and Dr. Matti Sauna-aho. The mammo- w ¼ Nc; ðA:7Þ
gram was provided by Prof. Olli Navalainen. The cell image where N are the same shape functions as in (A.5), and c is
was obtained from Dr. Kristen Harris of the Harvard Med- an arbitrary vector. As w is a scalar, we have
ical School. The retinal angiogram was obtained from Dr.
Piotr Jasiobedzki. The angiogram and spine images were w ¼ w T ¼ cT N T . ðA:8Þ
232 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233

Substituting (A.5)–(A.8) into (A.3) yields the snake equa- matrix (Bathe and Wilson, 1976). The solution u(t + Dt) to
tions of motion (A.17) is obtained by first solving Ls = bu(t) + cu(t
Dt)
u þ Cu_ þ Ku
F þ P ¼ 0;
M€ ðA:9Þ with forward substitution, then LTu = D
1s with back-
ward substitution. Since A is constant, only a single fac-
where M is the mass matrix, C is the damping matrix, K is torization is necessary. Therefore, at each time step only
the stiffness matrix, F is the force vector, and P is the the forward/backward substitutions are performed to
boundary forces, defined as follows: integrate the snake equations of motion forward through
Z L time.
M¼ NT lN ds; ðA:10Þ
0
Z L References
C¼ NT cN ds; ðA:11Þ
0 Amini, A., Weymouth, T., Jain, R., 1990. Using dynamic programming
K ¼ Ka þ Kb ; ðA:12Þ for solving variational problems in vision. IEEE Transactions on
Z L  T   Pattern Analysis and Machine Intelligence 12 (9), 855–867.
oN oN Barrett, W., Mortensen, E., 1997. Interactive live-wire boundary extrac-
Ka ¼ a ds; ðA:13Þ tion. Medical Image Analysis 1 (4), 331–341.
0 os os
Z L  2 T  2  Bathe, K.-J., Wilson, E.L., 1976. Numerical Methods in Finite Element
oN oN Analysis. Prentice-Hall, Englewood Cliffs, NJ.
Kb ¼ 2
b ds; ðA:14Þ Blake, A., Isard, M., 1998. Active Contours. Springer, Berlin.
0 os os2
Z L Carlbom, I., Terzopoulos, D., Harris, K., 1994. Computer-assisted
registration, segmentation, and 3D reconstruction from images of
F¼ NT q ds; ðA:15Þ neuronal tissue sections. IEEE Transactions on Medical Imaging 13
0
"  2   T 2 #L (2), 351–362.
oN TT o oN oN oN Cohen, L., Cohen, I., 1993. Finite element methods for active contour
P ¼
N a þN b 2
b 2 u. models and balloons for 2D and 3D images. IEEE Transactions on
os os os os os Pattern Analysis and Machine Intelligence 15 (11), 1131–1147.
0
ðA:16Þ Cohen, L., Kimmel, R., 1997. Global minimum for active contour models:
a minimal path approach. International Journal of Computer Vision
Eq. (A.9) gives the finite element formulation for the whole 24 (1), 57–78.
Dubuisson-Jolly, M.P., Gupta, A., 2001. Tracking deformable templates
snake. To achieve acceptable accuracy in the finite element
using a shortest path algorithm. Computer Vision and Image
approximation, the integration domain should be discret- Understanding 81 (1), 26–45.
ized into a number of small subdomains, resulting in the fi- Falcão, A.X., Udupa, J.K., 1997. Segmentation of 3D objects using live
nite element mesh. That is, the snake contour is divided wire. In: SPIE on Medical Imaging 1997, vol. 3034, Newport Beach,
into small segments (elements), each of which can still be CA, pp. 228–239.
Falcão, A.X., Udupa, J.K., Miyazawa, F.K., 2000. An ultra-fast user-
considered a snake. Applying (A.9) to an element e, we
steered segmentation paradigm: live-wire-on-the-fly. IEEE Transac-
have Me € ue þ Cu_ e þ Ke ue
Fe þ Pe ¼ 0, where Me is the tions on Medical Imaging 19 (1), 55–62.
element mass matrix, Ce is the element damping matrix, Falcão, A.X., Udupa, J.K., Samarasekera, S., Hirsch, B.E., 1996. User-
Ke is the element stiffness matrix, Fe the element force vec- steered image boundary segmentation. In: Proceedings of SPIE on
tor, and Pe the element boundary forces applied to the Medical Imaging, vol. 2710, Newport Beach, CA, pp. 278–288.
Falcão, A.X., Udupa, J.K., Samarasekera, S., Sharma, S., 1998. User-
boundary points of the element. Assembling the element
steered image segmentation paradigms: live wire and live lane.
matrices results in the system matrix motion equation (4). Graphical Models and Image Processing 60, 233–260.
In a closed snake, the boundary forces will cancel each Fua, P., 1997. Model-based optimization: an approach to fast, accurate,
other. In an open snake, the boundary conditions may be and consistent site modeling from imagery. In: Firschein, O., Strat,
assumed to be zero at the two ends. However, for general- T.M. (Eds.), RADIUS: Image Understanding for Intelligence Imagery.
Morgan Kaufmann, Los Altos, CA.
ity and clarity, we introduce g for the external force vector.
Fua, P., Brechbühler, C., 1997. Imposing hard constraints on deformable
To solve the motion equation (4), we replace the time models through optimization in orthogonal subspaces. Computer
derivatives of u with the backward finite differences Vision and Image Understanding 65, 148–162.
Gavrila, D.M., 1996. Hermite deformable contours. In: Proceedings of the
€u ¼ ðuðtþDtÞ
2uðtÞ þ uðt
DtÞ Þ=ðDtÞ2 ; u_ ¼ ðuðtþDtÞ
uðtÞ Þ=Dt; International Conference on Pattern Recognition, Vienna, Austria, pp.
130–135.
where the superscripts denote the quantity evaluated at the Grzeszczuk, R., Levin, D., 1994. Brownian strings: segmenting images
time given in the parentheses and the time step is Dt. This with stochastically deformable contours. In: Robb, R. (Ed.), Proceed-
yields the update formula ings of the Third Conference on Visualization in Biomedical Com-
puting (VBC
94), SPIE Proceedings, vol. 2359. SPIE, pp. 72–89.
Hyche, M.E., Ezquerra, N.F., Mullick, R., 1992. Spatiotemporal detection
AuðtþDtÞ ¼ buðtÞ þ cuðt
DtÞ þ g; ðA:17Þ
of arterial structure using active contours. In: Proceedings of the
2 2 Second Conference on Visualization in Biomedical Computing (SPIE
where A = M/(Dt) + C/Dt + K and b = 2M/(Dt) + C/Dt
and c =
M/(Dt)2. Because A is symmetric and banded, vol. 1808), Chapel Hill, NC, October, pp. 52–62.
Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes: active contour
it can be economically saved in skyline storage, and effi- models. International Journal of Computer Vision 1 (4), 321–331.
ciently factorized uniquely into the form A = LDLT, Kwon, Y.W., Bang, H., 1997. The Finite Element Method Using Mat-
where L is a lower triangular matrix and D is a diagonal labCRC Mechanical Engineering Series. CRC Press, Boca Raton, FL.
 J. Liang et al. / Medical Image Analysis 10 (2006) 215–233 233

Liang, J., 2000. Dynamic chest image analysis: new model-based methods Mortensen, E.N., 2000. Simultaneous multi-frame subpixel boundary
for dynamic pulmonary imaging and other applications. Turku Centre definition using toboggan-based intelligent scissors for image and
for Computer Science, Turku, Finland, December [TUCS Dissertation movie editing. Ph.D. Thesis, Department of Computer Science,
No. 31] (available at <http://www.cs.toronto.edu/~liang/ Brigham Young University, Provo, UT.
phddissertation.pdf>). Mortensen, E.N., Barrett, W.A., 1995. Intelligent scissors for image
Liang, J., Haapanen, A., Järvi, T., Kiuru, A., Kormano, M., Svedström, composition. In: Proceedings of Computer Graphics (SIG-
E., Virkki, R., 1998. Dynamic chest image analysis: model-based GRAPH
95), Los Angeles, CA, August, pp. 191–198.
pulmonary perfusion analysis with pyramid images. In: Hoffman, Mortensen, E.N., Barrett, W.A., 1998. Interactive segmentation with
E.A., (Ed.), Medical Imaging 1998: Physiology and Function from intelligent scissors. Graphical Models and Image Processing 60, 349–
Multidimensional Images, San Diego, CA, pp. 63–72. 384.
Liang, J., Järvi, T., Kiuru, A., Kormano, M., Svedström, E., 2001. Mortensen, E.N., Barrett, W.A., 1999. Toboggan-based intelligent scissors
Dynamic chest image analysis: evaluation of model-based perfusion with a four parameter edge model. In: Proceedings of IEEE Confer-
analysis with pyramid images. In: Proceedings of the 23rd Annual ence on Computer Vision and Pattern Recognition, Fort Collins, CO,
International Conference of the IEEE Engineering in Medicine and June, pp. 452–458.
Biology Society, Istanbul, Turkey, October, pp. 415–420 (invited Neuenschwander, W., Fua, P., Székely, G., Kübler, O., 1994. Initializing
paper). snakes. In: Proceedings of the Conference on Computer Vision and
Liang, J., Järvi, T., Kiuru, A., Kormano, M., Svedström, E., 2003. Pattern Recognition (CVPR
94). IEEE Computer Society Press, Silver
Dynamic chest image analysis: model-based perfusion analysis in Spring, MD, pp. 658–663.
dynamic pulmonary imaging. EURASIP Journal on Applied Signal Ojala, T., Liang, J., Näppi, J., Nevalainen, O., 2000. Interactive
Processing (5), 437–448 (special issue on Advances in Modality- segmentation of the breast region from digitized mammograms. In:
Oriented Medical Image Processing). Proceedings of the IASTED International Conference on Signal
Liang, J., Järvi, T., Kiuru, A., Kormano, M., Svedström, E., Virkki, R., Processing and Communications (SPC 2000), Marbella, Spain, Sep-
1997. Dynamic chest image analysis: model-based ventilation study tember, pp. 132–136.
with pyramid images. In: Hoffman, E.A. (Ed.), Medical Imaging 1997: Ojala, T., Näppi, J., Nevalainen, O., 2001. Accurate segmentation of the
Physiology and Function from Multidimensional Images, Newport breast region from digitized mammograms. Computerized Medical
Beach, CA, pp. 81–92. Imaging and Graphics 25 (1).
Liang, J., McInerney, T., Terzopoulos, D., 1999a. United snakes. In: Sethian, J.A., 1997. A fast marching level set method for monotonically
Proceedings of the Seventh International Conference on Computer advancing fronts. Proceedings of the National Academy of Sciences of
Vision (ICCV
99), Kerkyra (Corfu), Greece, September. IEEE Com- the United States of America 93 (4), 1591–1595.
puter Society Press, Silver Spring, MD, pp. 933–940. Singh, A., Goldgof, D., Terzopoulos, D., 1998. Deformable Models in
Liang, J., McInerney, T., Terzopoulos, D., 1999b. Interactive medical Medical Image Analysis. IEEE Computer Society Press, Silver Spring,
image segmentation with united snakes. In: Proceedings of the Second MD.
International Conference on Medical Image Computing and Com- Staib, L., Duncan, J., 1992. Boundary finding with parametrically
puter-Assisted Intervention (MICCAI 99), Cambridge, England, deformable models. IEEE Transactions on Pattern Analysis and
September. Springer, Berlin, pp. 116–127. Machine Intelligence 14 (11), 1061–1075.
Liang, J., Virkki, R., Järvi, T., Kiuru, A., Kormano, M., Svedström, E., Terzopoulos, D., Qin, H., 1994. Dynamic NURBS with geometric
1997. Dynamic chest image analysis: evaluation of model-based constraints for interactive sculpting. ACM Transactions on Graphics
ventilation study with pyramid images. In: Zurawski, R., Liu, Z.-Q. 13 (2), 103–136.
(Eds.), IEEE First International Conference on Intelligent Processing Terzopoulos, D., Szeliski, R., 1992. Tracking with Kalman snakes. In:
Systems, Beijing, China, pp. 989–993. Blake, A., Yuille, A. (Eds.), Active Vision. MIT Press, Cambridge,
McInerney, T., Terzopoulos, D., 2000. Topology adaptive snakes. MA, pp. 3–20.
Medical Image Analysis 4, 73–91. Terzopoulos, D., Witkin, A., Kass, M., 1988. Constraints on deformable
McInerney, T., Terzopoulos, D., 1996. Deformable models in models: recovering 3D shape and nonrigid motion. Artificial Intelli-
medical image analysis: a survey. Medical Image Analysis 1 (2), gence 36 (1), 91–123.
91–108. Xu, C., Prince, J.L., 1998. Snakes, shapes, and gradient vector flow. IEEE
Menet, S., Saint-Marc, P., Medioni, G., 1990. B-snakes: implementa- Transactions on Image Processing 7 (3), 359–369.
tion and application to stereo. In: Proceedings DARPA, pp. 720– Zienkiewicz, O., Taylor, R., 1989. The Finite Element Method. McGraw-
726. Hill, New York, NY.