Image and Vision Computing 17 (1999) 583–588

Adaptive segmentation of ultrasound images

Barbara Levienaise-Obadia, Andrew Gee*
Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK
Received 15 October 1997; received in revised form 28 September 1998; accepted 30 October 1998

Abstract
This article describes a novel approach to the semi-automatic segmentation of ultrasound images. Assisted segmentation is particularly
attractive when processing many slices through a 3D data set, and even though fully automatic segmentation would be ideal, this is currently
not feasible given the quality of ultrasound images. The algorithm developed in this article is based on the active contour paradigm, with
several important modifications. The contour is attracted to boundaries described locally by statistical models: this allows for the fact that the
definition of what constitutes a boundary may vary around the boundary’s length. The statistical models are trained on-the-fly by observing
boundaries accepted by the operator. In this way, operator intervention in a particular slice is sensibly exploited to reduce the need for
intervention in subsequent slices. The resulting algorithm provides fast, reliable and verifiable segmentation of in vivo ultrasound images.
q 1999 Elsevier Science B.V. All rights reserved.
Keywords: Adaptive segmentation; Ultrasound image; B-spline snake

1. Introduction However, automatic interpretation of ultrasound data is

confounded by noise, especially speckle, an artefact
Given the current proliferation of 3D medical imaging common to all ultrasound images. Given these difficulties,
devices, the need to segment the data they gather is of para- fully-automatic segmentation of ultrasound images is not
mount importance. Little quantitative information can be currently feasible. Instead the focus has been on semi-auto-
gleaned from a raw 3D data set: the data can be displayed matic techniques, in particular live-wires and snakes. An
on a slice by slice basis, or by volume rendering, but further important feature of semi-automatic segmentation is that
processing generally requires some sort of segmentation. the clinician views the boundary as it is delineated in
For example, a surface rendering of an organ requires every slice. Thus the clinician has confidence in the final
segmentation of its boundary; area and volume measure- segmentation, something which cannot be said for fully-
ments require segmentation of the region of interest; land- automatic techniques.
mark-based registration requires segmentation of the Live-wire [3] is currently one of the best semi-automatic
landmarks. segmentation techniques, and is beginning to be supplied
While manual segmentation of a 2D image is merely with 3D visualisation packages. The user clicks on one
tedious, manual segmentation of every slice through a 3D point on the boundary and then drags the cursor roughly
data set is often prohibitively time-consuming. For this around the outline. The computer displays in real time the
reason there has been considerable research into fully- and “best” path from the seed point to the current cursor posi-
semi-automatic medical image segmentation techniques, tion: one path is better than another if it follows more
including grey-level thresholding [1], region growing [1], boundaries (as defined by boundary models) in the image.
active contours (“snakes”) [2] and “live-wire” algorithms [3]. If the “best” path deviates from the desired boundary, the
Particularly challenging is the segmentation of 3D ultra- user simply deposits another seed point at the location
sound data. 3D ultrasound has been attracting considerable where the boundary was lost. Accurate segmentation can
research and commercial interest as a cheap, portable and be achieved in a fraction of the time it would take to trace
non-invasive method for acquiring 3D medical images [4]. the boundary by hand. Further, the live-wire can be trained
on-the-fly to favour different types of boundaries: the algo-
* Corresponding author. Tel.: 1 44 1223 332754; fax: 1 44 1223
rithm simply examines the boundary accepted by the opera-
332662. tor and uses this information to update its boundary models.
E-mail address: ahg@eng.cam.ac.uk (A. Gee) Snakes are active contours which deform in the image
0262-8856/99/$ - see front matter q 1999 Elsevier Science B.V. All rights reserved.
PII: S0262-885 6(98)00177-2
584 B. Levienaise-Obadia, A. Gee / Image and Vision Computing 17 (1999) 583–588

The internal forces usually take the form

Z1 2r 22 r 2 
Eint ˆ aj 2
1b ds
0 2s 2s2

The a term controls the “tension” of the snake while the b

term controls its “rigidity”. The external forces take a simi-
lar form:
Z1
Eext ˆ gP r s††ds
0

where P(x,y) is a scalar potential function designed so that

local minima of the function coincide with image features of
interest. For instance, if we are trying to detect boundaries
Fig. 1. B-spline snakes. The snake evolves under a set of “forces” applied at which are characterised by high intensity gradients, then a
a set of sample points. Each force is proportional to the distance between
the sample point and the target point. The target point is found by a one-
suitable potential would be P x; y† ˆ 2u7Gs *I x; y†u, where
dimensional search normal to the spline for an image feature of interest. At Gs is a 2D Gaussian kernel with standard deviation s
equilibrium the snake provides a compact representation of the boundary. pixels.
Starting from an initial position, the snake evolves under
under a set of internal and external forces [5]. The internal the influence of the combined field E ˆ Eint 1 Eext . In
forces keep the snake smooth, while the external forces practical implementations, the snake is split into piecewise
attract the snake to features of interest in the image. If linear segments and local forces are calculated at each
initialised by the operator near a boundary, the snake vertex. The motion of the snake is computed using
deforms to “hug” the boundary along its length. Particularly gradient descent [5] or Lagrangian dynamics [7]. For
useful is the ability to use the snake’s equilibrium position segmentation applications, the idea is that the equilibrium
in one slice as its initial position in the next slice. If the position of the snake provides the desired segmentation. If
slices are closely spaced then the snake should snap on to I(x,y) changes, the snake can also be used to track a
the new boundary with no operator assistance. Moreover, boundary over time, providing motion as well as shape
given prior knowledge of the likely shape of the boundary, information [8].
the snake can be constrained to give only physically plau- The main reason for using snakes in segmentation is that
sible segmentations [6], providing further resilience to they allow the incorporation of geometric constraints. In
noise. particular, they exploit (in fact they rely on) prior boundary
In this article we develop a snake-based approach to the estimates obtained manually or from the previous image in a
semi-automatic segmentation of ultrasound images. We sequence. Such an approach sensibly combines the strengths
incorporate many of the attractive features of live-wire, in of human operators (initial boundary recognition) and
particular on-the fly learning of statistical boundary models computer algorithms (accurate boundary delineation).
following operator intervention, while preserving the Snakes have proved very successful for medical image
straightforward inter-frame propagation of snakes. We segmentation [2]. In particular, snakes have been shown to
demonstrate how the resulting algorithm provides fast, reli- give state-of-the art performance for segmenting intra-
able and verifiable segmentation of in vivo ultrasound vascular ultrasound images [1].
images. An alternative way to enforce the smoothness constraint
is to use cubic B-spline snakes [9]. A cubic B-spline is
specified by m 1 1 control points p0, p1…pm and comprises
2. B-spline and piecewise linear snakes m-2 cubic polynomial curve segments r3,r4…rm. The join-
ing points between each curve segment are known as knots.
Active contours, or snakes [5] have found many applica- The equation of each segment is
tions in image analysis and computer vision. A snake is a 2 32 3
21 3 23 1 pi23
parametric contour r s† ˆ x s†; y s††, s [ ‰0; 1Š, in the 6 76 7
6 3 26 3 0 76 p 7
image plane (x,y). The snake evolves under internal and 1 3 2 6 76 i22 7
external “forces”, usually described by energy terms Eint ri si † ˆ ‰si si si 1Š6
6
76
76
7
7
6 6 23 0 3 0 76 pi21 7
and Eext. The internal forces keep the snake smooth and 4 54 5
the external forces couple the snake to the image I(x,y), 1 4 1 0 pi
attracting the snake to features of interest (for segmentation
applications, these features would be object boundaries). for 0 # si , 1 and 3 # i # m. For notational convenience
 B. Levienaise-Obadia, A. Gee / Image and Vision Computing 17 (1999) 583–588 585

target points, i.e.:

0 12
X
n X
m
Eext ˆ @y si † 2 fj si †pj A 1†
iˆ1 jˆ1

The positions of the control points are then updated to mini-

mise Eext using linear least squares [9]. The one-dimensional
search and re-estimation of the control points is repeated
Fig. 2. Organ boundaries in ultrasound images. The gall blagger (a) is a
fluid filled cavity which is fairly difficult to segment. Note how the bound-
until the snake attains equilibrium.
ary is not characterised by a high gradient everywhere: the boundary prop- The main drawback of both piecewise linear and B-spline
erties are not stationary. The kidney (b) is even more challenging to snakes is the difficulty in specifying a priori the correct
segment as tissue–tissue boundaries are relatively difficult to localise in smoothness parameters (i.e. setting a and b , or specifying
ultrasound images. the correct number of control points). Techniques have been
proposed to automatically introduce the “best” number of
we can re-parameterise the spline over a single parameter s: control points [11], though such techniques tend to be
X
m computationally expensive. Here we propose an extremely
r s† ˆ fi s†pi ; straightforward, novel work-around. We use a B-spline
iˆ0 formulation with a fairly small number of control points.
While this tends to give overly smoothed contours, we base
where fi are the spline basis functions and 0 # s # 1. The
our segmentation not on the B-spline r(s) but on a piecewise
representation p0…pm is more compact than the long list of
linear contour with vertices y(s1)…y(sn). The target points
vertices in piecewise linear formulations. Moreover, the
provide a good segmentation, while the B-spline formula-
internal forces are no longer required: the smoothness of
tion constrains the search for the target points and therefore
the snake is implicit in the B-spline formulation. B-splines
stops the snake becoming too “bent” and intersecting itself.
may be open or closed as required, and are defined with
continuity properties at each knot. The flexibility of the
curve increases as more control points are added: each addi-
3. Properties of boundaries
tional control point allows one more inflection. It is also
possible to use multiple knots to reduce the continuity at We turn now to the particular problem of ultrasound
knots [10]. image segmentation. Compared with other medical imaging
However, what makes B-splines especially useful for modalities (e.g. CT and MRI) ultrasound is particularly
segmentation is that they exhibit local control: modifying difficult to segment as the quality of the images is relatively
the position of one control point causes only a small part of low. In particular, organ boundaries are not always promi-
the curve to change. This leads to an effective and simple nent (see Fig. 2). Fully automatic techniques for ultrasound
algorithm for computing the motion of the snake under the image segmentation are not likely to be robust. Instead we
influence of image forces (see Fig. 1). Local forces are make sensible use of operator assistance, through snakes, to
computed at n sample points r(s1)…r(sn) distributed evenly produce fast and reliable segmentations with the minimal
around the spline. At each sample point, a one-dimensional amount of manual intervention.
search is performed normal to the spline for a local mini- To guide the snakes, we need to define potential functions
mum y(si) of P(x, y): we shall refer to the points y(s1)…y(sn) based on image properties. In addition to the intensity gradi-
as target points. A force is applied at each sample point ent, which will be of limited use, it is reasonable to attempt
proportional to the distance between the sample and the some sort of texture segmentation [12,13], though the noise
properties of the ultrasound images suggest that looking at
anything beyond second order grey level statistics is point-
less. We therefore attempt segmentation based on two prop-
erties:
1. 1D intensity gradients along the B-spline snake’s search
lines.
2. First and second order grey level statistics.
Both properties are measured after smoothing with a Gaus-
sian kernel of standard deviation s ˆ 3 pixels. This reduces
the speckle but preserves meaningful image structure.
Fig. 3. Texture classification. A texture classifier is trained for each spline
segment by sampling the mean and variance of 7 × 7 patches within the The texture segmentation is performed as follows. Given
search window and finding the optimal discriminant between “in” and “out” an initial segmentation, we gather texture statistics
patches. from both sides of the boundary and derive an optimal
586 B. Levienaise-Obadia, A. Gee / Image and Vision Computing 17 (1999) 583–588

Fig. 4. Segmenting a human gall bladder. The figure shows a typical run of the adaptive segmentaion system, demonstrating clearly the benefits of statistical
boundary models with on-the-fly training. See text for details.

discriminator between inside and outside (or equivalently, 1 S

Cˆ ln i 4†
for open splines, left and right (see Fig. 3)). We represent the 2 So
texture statistics of a 7 × 7 patch centered on (x,y) as the
vector x x; y† ˆ ‰x1 ; x2 ŠT , where x1 is the mean of the inten- As the texture statistics are generally not stationary around
sities in the patch and x2 is their variance. We then calculate the boundary, we compute mi, mo, Si and So locally for each
the mean mi of the x’s sampled from the “inside” class Vi, of the spline segments r3…rm.
and also their covariance matrix Si. Likewise, we calculate We can now define potential functions for both intensity
the corresponding quatities m0 and S0 for samples taken and texture-based segmentation. The potential functions are
from the “outside” class V0. Assuming the class-conditional calculated along each of the B-spline snake’s search lines
density functions p(xuVj) are independent normal distribu- and local minima provide the target points y(s1)…y(sn):
tions, the optimal discriminator is [14]: P7 x; y† ˆ 2j 7 Gs *I x; y††:n si †j 5†
if 2 di 1 d0 . C then x [ Vi else x [ V0 ;
PV x; y† ˆ j C 2 do 2 di †j 6†
where where n(si) is a unit vector along the search line. It is not
immediately apparent which potential function will perform
1
di ˆ x 2 mi †Si21 x 2 mi †T 2† best, and indeed this will vary for different images, or even
2 at different portions of the boundary in a single image. We
therefore propose to use a linear combination of the two,
1 with on-the-fly training to adaptively select the appropriate
do ˆ x 2 mo †So21 x 2 mo †T 3† weights.
2
 B. Levienaise-Obadia, A. Gee / Image and Vision Computing 17 (1999) 583–588 587

4. On-the-fly training snake’s current position p0…pm as a starting point in the

new slice.
In the spirit of [13] we propose on-the-fly training for the
most appropriate local potential of the form
5. Results
P x; y† ˆ aP7 x; y† 1 bPV x; y† 1 g 7†
A typical run of the segmentation system is illustrated in
While g has no effect on the location of any extrema of P Fig. 4. The task is to identify the boundary of a human gall
(x,y), it allows us to consider Eq. (7) as a single layer bladder in a series of slices through a 3D ultrasound data set.
network with weights a , b and g . We can then train the In the first slice the user specifies B-spline control points
network using a pseudo-inverse [14] and training samples near the boundary (Fig. 4a) and the system searches normal
(xt,yt) gathered from around the boundary. We define target to the spline for local minima of P7. As the boundary is not
values for the training samples as follows: always well characterised by high gradients, the target
points give a fairly poor segmentation (Fig. 4b). The user
P xt ; yt † ˆ 2 exp 2d2 =s2 † manually corrects the target points to give the desired
segmentation (Fig. 4c). The system now re-fits the B-spline
where d is the distance in pixels of (xt, yt) from the boundary. to the target points and trains local boundary models for
Thus the target values are 21 on the boundary and approach each spline segment, so that more appropriate potentials P
zero asymptotically away from the boundary. When trained, can be used in the next slice. The B-spline is propagated into
P(x,y) should have local minima on the segmentation the next slice and the ID searches are repeated (Fig. 4d):
boundary, as required. As we are training local boundary target points are located at local minima of P along the
models, a , b and g are calculated for each spline segment. search lines (Fig. 4e). This gives an acceptable segmentation
We found that the performance of the system was not sensi- without the need for any user intervention. For comparison,
tive to the parameter s , which we conveniently set to unity. Fig. 4f shows the segmentation obtained using the intensity
We also found it advantageous to discard the moduli signs in gradient alone.
Eq. (5): this allows the algorithm to exploit the polarity of The segmentation system is not computationally expen-
the boundary. Note how on-the-fly training will automati- sive and runs fast enough for comfortable interaction. Little
cally find the right sign for a . user intervention is required after slice 1, as the adaptive
While such training is of no benefit for single-frame models are able to track slow changes in the boundary statis-
segmentation, it can greatly speed-up the process of tics. Even when user intervention is required, the segmenta-
segmenting many slices through a 3D data set. As the tion is usually considerably better than a segmentation based
boundary statistics generally change slowly from one slice on gradient information alone.
to the next, optimal segmentation potentials Eq. (7) learned
in one slice will also work well on the next slice. What
emerges is a segmentation paradigm with the following 6. Conclusions
structure:
While the goal of fully-automatic ultrasound image
(1) The user specifies an initial B-spline by selecting segmentation remains elusive, we have shown how operator
control points p0…pm in slice 1. Initially, in the absence assistance can be exploited to produce fast, reliable and
of any texture models, the snake’s potential function Eq. verifiable semi-automatic segmentation. Key features of
(7) is calculated with a ˆ 1 and b ˆ 0. our approach include high resolution segmentation from
(2) The spline is sampled at n locations r(s1)…r(sn) target points constrained by a B-spline snake, local statisti-
distributed evenly around its length. Corresponding target cal boundary models and on-the-fly training of the boundary
points y(s1)…y(sn) are located using a ID search normal models. 3D data sets can be segmented in a fraction of the
to the spline for a local minimum of the potential Eq. (7). time it would take to manually trace the boundaries in each
(3) The user corrects any mis-positioned target points. frame. Further work could look into exploiting prior knowl-
Segmentation (based on the target points) is now edge of an organ’s shape [6] to further improve resilience to
complete for this slice. noise and reduce the amount of operator intervention. It
(4) The B-spline control points p0…pm are adjusted to would also be interesting to investigate other boundary indi-
minimise Eext in Eq. (1). cators besides gradients and low order texture statistics.
(5) Local texture potentials PV (one per spline segment)
are calculated by sampling patches around the boundary
(Fig. 3) and using Eq. (2)–(4) and (6).
References
(6) Local a ,b and g are calculated using training samples
and a pseudo-inverse [14] such that P (x,y) in Eq. (7) is [1] C.J. Bouma, W.J. Niessen, K.J. Zuiderveld, E.J. Gussenhoven, M.A.
minimised at boundaries. Viergever, Evaluation of segmentation algorithms for intravascular
(7) Move on to the next slice and repeat from 2, using the ultrasound images, in: K.H. Höhne, R. Kikinis (Eds.), Visualization in
588 B. Levienaise-Obadia, A. Gee / Image and Vision Computing 17 (1999) 583–588

Biomedical Computing, Lecture Notes in Computer Science 1131, [8] A. Blake, M. Isard, D. Reynard, Learning to track the visual motion of
Springer, Berlin, 1996, pp. 203–212. contours, Artificial Intelligence 78 (1995) 101–133.
[2] T. McInerney, D. Terzopoulos, Deformable models in medical image [9] R. Cipolla, A. Blake, The dynamic analysis of apparent contours,
analysis: a survey, Medical Image Analysis 1 (2) (1996) 91–108. Proceedings of the Third International Conference on Computer
[3] W.A. Barrett, E.N. Mortensen, Fast, accurate, and reproducable live- Vision, Osaka, Japan, December 1990, pp. 616–623.
wire boundary extraction, in: K.H. Höhne, R. Kikinis(Eds.), Visuali- [10] A. Watt, 3D Computer Graphics, 2nd ed., Addison-Wesley, Reading,
zation in Biomedical Computing, Lecture Notes in Computer Science MA, 1993.
1131, Springer, Berlin, 1996, pp. 183–192. [11] T.-J. Cham, R. Cipolla, Automated B-spline curve representation with
[4] F. Hottier, A. Collet Billon, 3D echography: status and perspective, MDL-based active contours, Proceedings of the British Machine
in: K.H. Hohne, H. Fuchs, S.M. Pizer(Eds.), 3D Imaging in Medicine: Vision Conference, Edinburgh, 1996, pp. 363–372.
Algorithms, systems, applications, Springer, Berlin, 1990, pp. 21–41. [12] J. Ivins, J. Porrill, Active region models for segmenting textures and
[5] M. Kass, A. Witkins, D. Terzopoulos, Snakes - active contour models, colours, Image and Vision Computing 13 (5) (1995) 431–438.
International Journal of Computer Vision 1 (4) (1987) 321–330. [13] R. Ronfard, Region-based strategies for active contour models, Inter-
[6] T. Cootes, A. Hill, C. Taylor, J. Haslam, The use of active shape national Journal of Computer Vision 13 (2) (1994) 229–251.
models for locating structures in medical images, Image and Vision [14] C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon
Computing 12 (6) (1994) 355–366. Press, Oxford, 1995.
[7] A. Lobregt, M.A. Viergever, A discrete dynamic contour model,
IEEE Transactions on Medical Imaging 14 (1) (1995) 12–24.