Image Anal Stereol 2003;22:35-42

Original Research Paper

DYNAMIC PARTICLE SYSTEMS FOR OBJECT STRUCTURE

EXTRACTION

OLIVIER LAVIALLE1,2, FRANCK ANGELLA1, CHRISTIAN GERMAIN1,2 AND PIERRE BAYLOU1

1
Equipe Signal/Image ENSEIRB - University of Bordeaux I BP 99, 33402 Talence Cedex, 2ENITA de
Bordeaux, BP 201, 33175 Gradignan cedex - France
e-mail: lavialle@tsi.u-bordeaux.fr
(Accepted February 2, 2003)

ABSTRACT

A new deformable model based on the use of a particle system is introduced. By defining the local behavior
of each particle, the system behaves as an active contour model showing a variable topology and
regularization properties. The efficiency of the particle system is illustrated by two applications: the first one
concerns the use of the system as a skeleton extractor based on the propagation of particles inside a tree-
shaped object. Using this method, it is possible to generate a cartography of structures such as veins or
channels. In a second illustration, the system avoids the problem of initialization of a piecewise cubic B-
spline network used to straighten curved text lines.

Keywords: deformable models, particle system, snakes, structure extraction.

INTRODUCTION to change its topology and unlike the classical models

does not require an initialization close to the solution.
Deformable models have been widely used since Then, we present two applications of this model.
the original definition of snakes or active contours First, a classical application consists in propagating
(Kass et al., 1988). Most developments of deformable the particles inside a grayscale tree-shaped object. The
models deal with boundary detection and feature algorithm simulates the evolution of a system subjected
extraction in 2-D or 3-D images (Cohen, 1991). to internal, and external forces. The simultaneous use of
Basically, the method consists in deforming and particle systems and tree generation allows to extract
moving an initial curve toward a region of interest. The a linear piecewise, and connected skeleton of the object.
solution is obtained by minimizing the model energy. The second application concerns the straightening of
The inability to change from a given topology to text lines (Lavialle et al., 2001): the approach considers
another is a major drawback of classical deformable a piecewise cubic B-spline system where each curve is
models. For this reason, several approaches have been defined as a particle. The system propagates in the
proposed. Topologically adaptable snakes analyse the vertical direction from a central position. Internal forces
curve-topology (Mc Inerney and Terzopoulos, 1995) control the expansion of the curve system, whereas
or the surface-topology (Mc Inerney and Terzopoulos, external forces drive each curve toward a text line to
1997), and include cutting or fusion steps during the obtain the global structure of the distorted text.
evolution. Level set methods consider the curve or
the surface as zero values of a higher dimension PARTICLE SYSTEM
object (Sethian, 1996). Implicit models act the same
way but are generated from predefined primitives
Model definition
(Yahia et al., 1998). A special class of deformable Let us denote by S a particle system. This system is
models is represented by particle systems (Szeliski a set of nodes (or particles) Mi. Each node can evolve
and Tonnesen, 1992) allowing topology changes during in the x-y image plane in response to different acting
the evolution, and providing results involving forces. These forces are divided in two groups: internal,
predefined primitives, as opposed to level set methods. and external forces. Internal forces control, and regulate
The next section presents a deformable model the expansion of the system; external forces drive
based on the definition of a particle system (Angella et evolution of S according to the current image of
al., 1998). This model overcomes classical deformable interest. The behavior of a particule is the consequence
model drawbacks. The particle system shows the ability of the resulting force exerted on this particle.

35
 LAVIALE O ET AL: Dynamic particle system for object structure extraction

Our goal is to encourage the particle system to evolution follows the classical Newtonian mechanical
explore the image. Towards that end, interaction rules relation:
between particles have to be defined. A repulsion
behavior allows the system to expand; an attractive dvi (t )
mi ⋅ = Fi , (4)
behavior forces particles not to move away from other dt
particles. Thus, internal forces are defined through the
introduction of a Lennard-Jones interaction potential mi stands for the mass of the particle. For our
function gathering a long-range attraction term and a purpose, mi is a constant and can be included in
short-range repulsion term (Heyes, 1998). Classically, Fi. The numerical integration of this differential
the Lennard-Jones potential describes the interaction equation is obtained using the Euler integration
between pairs of atoms in a solid or a liquid. It has method. This leads to the choice of a time interval
the following form: parameter, resulting from a precision/computation
time compromise. Then, (4) becomes:
  σ 12  σ  6 
φ (r ) = ε    −    , (1) vit = vit −1 + δ t ⋅ ( Fi − α f vit −1 )
 r   r    t , (5)
  xi = xit −1 + δ t ⋅ vit
where r is the distance from an atom to an any other
where x it and v it are respectively the position and the
atom. The constants ε and σ are important in that they
determine the strength and shape of the interaction, velocity of a particle Mi at step t. α f is a friction
hence the properties of the solid or liquid. The force coefficient added to ensure the stability of the system.
between individual atoms is the negative of the Our approach consider the evolution of the
potential gradient. system as a solution of a Newtonian mechanical Eq.
Herein, we consider the following function: 4. Another equivalent approach can be proposed by
generating all forces from a unique potential function,
B A
ρ (d ) = − n, (2) so that the evolution of the particle system S will be
d m
d similar to the process involved when minimizing a
global energy, as it is performed when dealing with
where d is the distance between two particles. A, B, m classical active contour (Kass et al., 1988).
and n are positive coefficients used to set the
equilibrium distance. In our application, we chose APPLICATIONS OF THE MODEL
m = 2, n = 1 yielding an equilibrium distance equal to
2B/A. Object structure extraction
Using ρ, we can compute a global interaction The problem of skeleton extraction has found
( t ,i ) many solutions based on mathematical morphology
force f int : algorithms (Serra, 1982). Here, we propose to extract
r structures in grayscale tree-like objects by using the
f int(t ,i ) = − ∑ ρ (d ij ) ,
∇ (3) particle system described above (Angella et al.,1998).
M j ∈υi( t )
The evolution of the system is the consequence of a
(t ) Lennard-Jones interaction force (Eq. 3) complemented
where υ i is the neighborhood of Mi at step t (the with two other internal forces presented in Angella et
notion of neighborhood will be defined in the next al. (1998) within the framework of object structure
section). extraction.
External forces being application-dependent, they The first one provides the system with learning
will be described (along with additional internal abilities: this force, inspired by the so-called
forces) in the next section where two applications are evolutionary computation domain, acts as ”insect
considered. pheromones”, driving particles preferentially to the
paths already used by prior particles (Dorigo and
System evolution Gambardella, 1997). To do so, we build a velocity
One can control the behavior of each particle Mi map Iph using the velocity vit of each particle Mi at
by computing Fi, the total resulting force. Then, if we each step t of the algorithm. This map saves the
consider that each particle is a mechanical system, its average of the velocity observed in each pixel p of the

36
Image Anal Stereol 2003;22:35-42

image. Then, we define an elementary pheromonal

the computation of f int(t ,i ) and ( t ,i )
f reg . These
force averaged around xit as expressed by Eq. 6:
neighborhoods are actually necessary to list particles
( )
1 r influencing a given particle. In this respect, the
f ph(t ,i ) = ⋅ ∑ ∇I ph , (6) algorithm does not rely on the shape of objects but on
card Γi(t ) p∈Γi(t ) p
their structure as expressed by L.
where xit is the position of Mi at step t and Γi( t ) is the We define the set of links L with the binary
t function:
given area around x . i
1 if M i are linked at step t
The second additional internal force is a l (t ) : (i , j ) a  , (9)
regularization force between particles that tends to 0 if not
align particles within the same branch of a given
with l (i, i ) = 0, ∀i .
(t )
object by attracting a particle to the middle point of
its two neighbors. Let us consider a particle M i with
Eq. 9 defines the neighborhood of Mi at step t:
{
a neighboring, at step t, ν it = M j , M k . The force } ν it = {M j / l (t ) (i, j ) = 1}. (10)
is defined by:
The search for a neighbor of Mi is performed with
 x (jt ) − xk(t )  respect to the trajectory of Mi and includes time an
f ( t ,i )
= − xi( t )  , (7)
reg  2  space constraints: the neighbor shall be the closest to
  a previous location of Mi. The entire trajectory of Mi
is accordingly scanned starting at step t down to step
x tj , x kt and xit are respectively the position of M j , 1 using a restricted area around the trajectory. The
M k and M i at step t. selected neighbor Mj is obtained using the following
algorithm:
As we want our system to lie inside objects, we
use an external force depending directly on the for m=t,…,1:

( )
gradient of the image I, defined as follows:
r j = arg min x kt − x im
f ext = −k ext ⋅ ∇( ∇G ∗ I ), (8)
k / M k ∈S

∇G ∗ I is the modulus of a filtered gradient of the

if (x t
j )
− x im ≤ Dε

image. G is a low pass filter used to extend the then M j ∈ν it break

influence area of objects boundaries in I. kext is used
to control this influence. else m=m-1
The particle system will evolve inside objects end if
from an initial location xinit where particles are
generated with initial velocity vinit. The generation is
T-periodic (prior to generating a particle, a test is run Dε restricts the search process around the
to avoid overpopulation within the generating area). trajectory of Mi and m denotes the current iteration
The process described above simulates the varying from t down to 1. The process is stopped at
evolution of a deformable expansible tree created and iteration m if a neighbor satisfying the proximity
modified by linking neighboring particles and cutting condition is found.
branches according to different rules defined below. Updating the tree modifies the structure of the
The evolution of the tree T may be viewed as the joint links at every step of the evolution of S. Links
evolution of the particle system S and a set of links L between particles are dynamically changed according
between the elements of S. to the geometric relations between them and taking
Updating the tree carries with it the ability to advantage of the previous trajectories of particles. We
change the topology. In addition, L is involved in the analyze different cases corresponding to different
determination of the neighborhoods ν it , and then in configurations for each particle: addition and
convergence, deletion, divergence and shift:

37
 LAVIALE O ET AL: Dynamic particle system for object structure extraction

− Addition and convergence: when a particle has an are removed and new links have to be found
empty neighborhood, a neighbor must be found leading to a topology change.
among the other particles using the algorithm This simultaneous evolution of S and L allows a
above. This situation happens for a newly hierarchical exploration of the object and overcomes
generated particle or for a particle whose links are the topology changes problem. Finally, the approach
cut to remain consistent with the object structure. returns a hierarchical, connected, and piecewise linear
In addition, the system can hold optional “skeleton” of the object without any post-processing.
convergence cases, leading to cycle generations,
so that the tree becomes a graph including loops. Fig. 2 illustrates the evolution of the system on a
synthetic tree-like object. In this case, all particles are
− Deletion: if a particle Mi moves too far away from generated at the same location with the same initial
its neighbors, its links must be cut. The deletion velocity vector. The tree update process consists in
process is just based on a deletion distance. adding, deleting or shifting links according to the
− Divergence and shift: the divergence, or shift geometric relations between the particles and taking
behavior, occurs around a junction in the object the trajectories into account (Angella et al., 1998). In
structure. It is a local reorganization of the set of this example, the evolution stops because of the
links. To understand how the tree is updated, let initialization vicinity test.
us analyze Fig. 1. In the vicinity of a junction Fig. 3 shows a result obtained on a more complex
(step a), the particle M2 does not follow the image. The simultaneous use of a particle system and
particle M1, so that at step b the structure of the tree generation yields the skeleton of the river in an
links in L does not reflect the structure of the aerial image. The particles are generated from the
object. At step c, we decide to cut all links to M2 top-left of the structure and are then allowed to explore
and to suitably reconnect the tree at step d. This the object. Fig. 3a and 3b show the trajectories of the
topology change is tractable if one finds a particles and the resulting structure, respectively.
criterion to decide when cutting the skeleton at Note that some ramifications are not explored: some
step c. The rule adopted is based on angle image areas do not provide the particles with relevant
comparisons considering that the angle between external forces due to a low contrast between the
two links associated to a given particle has to be object and its environment. In addition, the object
greater than a fixed minimal angle θjunction. If it is structure may become too thin at the end of a
not the case, a junction is detected, the two links ramification to allow the extension of the system.

Fig. 1. Links update in the divergence-shift case. Particles are moving from right to left.

38
Image Anal Stereol 2003;22:35-42

a) b) c)

d) e) f)
Fig. 2. Synthetic tree-like object. 2a, 2b, 2c and 2d: Evolution of the particle system. 2e: Particle trajectories.
2f: final tree

a) b)
Fig. 3. Real tree-like object structure extraction. 2a) particle trajectories. 2b) main ramifications of the
skeleton for an aerial image.

ACTIVE CONTOURS NETWORK TO splines (B-splines) (Lehmann et al., 1999). Here, we

STRAIGHTEN DISTORTED TEXT LINES rather use piecewise cubic B-splines (i.e. each curve
is a set of connected cubic B-splines). Let us note Ci a
We proposed a 2D extension of the system to
straighten distorted text (Fig. 4a) (Lavialle et al., B-Spline. aik , s is then the kth control point of the sth
2001). For such a purpose, the system of particles is connected spline of Ci. The model continuity is
transformed into a system of curves. A curve is enforced by constraining the control points in order to
associated to each line of text. Noting a ik the kth obtain smoothed curves (Lavialle et al., 2001).
parameter of a ith curve Ci, our goal is to define an The horizontal coordinate of each parameter aik , s
energy function depending on a ik . Using a classical
is fixed, and the value of aik , s yields the vertical
approach, each curve may be modeled by Basis

39
 LAVIALE O ET AL: Dynamic particle system for object structure extraction

position of the sth spline of Ci. The internal energy is (Deriche, 1987). The smoothing resulting from the
defined by assuming that the text is locally uniform use of this operator is necessary to extend the
(i.e. line spacing is constant). Thus, our purpose is to influence of the lines, and then to attract the curves
move each curve toward the centre of its two neighbors: toward the solution
2 Fig. 4c shows the value of the vertical gradient
E k ,s
int
 1
2
( ) 
(i ) = aik , s − aik−,1s + aik+,1s  . (11) ∇I y . Finding the bottom of the lines is done with a
 
very simple energy functional: E ext = −∇I y leading
The system evolves along the vertical direction
from an initial location. In addition to the internal to the external force:
r
f ext = k ext ⋅ ∇(∇I y ) .
energy defined above, we introduce an interaction
force based on a Lennard-Jones potential between the (12)
control points of neighboring curves to force them to
All curves are generated at an initial location yinit
move from their initial location.
with an initial velocity vinit. The evolution of the
The minimization of an appropriate external system follows the classical mechanical relation
energy will allow to attract the curves toward the text. described in the previous section.
The computation of this energy is based on a
Fig. 5 shows an example of distorted text. Fig. 6
mathematical morphology transformation: we first
illustrates the evolution of the curve system on a
use an opening on the original image to fill in spaces
detail of Fig. 5. The curves are initialized at the
between letters. Fig. 4b shows the result obtained
middle of the page. After 150 iterations, each curve
with a structuring element of size 11 × 5. Then,
system has converged toward a text line and the
considering the bottom of the lines, the external
straightening is possible (Fig. 7).
energy is computed using a vertical Deriche gradient

a) b) c)
Fig. 4. a) curved text, b) opening of 4a, c) vertical gradient computed from 4a.

Fig. 5. Example of distorted text.

40
Image Anal Stereol 2003;22:35-42

a b

c d
Fig. 6. Evolution of a curve system.

Fig. 7. Straightened text.

41
 LAVIALE O ET AL: Dynamic particle system for object structure extraction

DISCUSSION cooperative learning approach to the traveling

salesman problem. IEEE Trans on Evol Comput 1:53-66.
In this paper, we presented a particle system Heyes DM (1998). The liquid state: application of
acting as a deformable model with flexible topology molecular simulations. John Wiley and Sons Ltd.
and regularization properties. When applied to the Kass M, Witkin A, Terzopoulos D (1988). Snakes: Active
extraction of an object structure, the system returns a contour models. Int J Compr Vision 9:321-31.
continuous and piecewise connected skeleton. As the Lavialle O, Molines X, Angella F, Baylou P (2001). Active
particles first move through large ramifications, the contour network to straighten distorted text lines.
method can give additional information on the Proceedings IEEE-ICIP 01, Thessaloniki, 3:748-51
hierarchy of structures. We can notice that the 3-D Lehmann TM, Gönner C, Spitzer K (1999). Survey:
extension of this method is obvious: it has been Interpolation Methods in Medical Image Processing.
already successfully tested on synthesized 3-D images. IEEE Trans Med Imag 18:1049-75.
Mc Inerney T, Terzopoulos D (1995). Topologically
In the second application, we demonstrated the Adaptable Snakes. Proceedings of the 5th International
ability of our method to simplify the initialization Conference on Computer Vision, Cambridge, 840-5.
step in the case of a system of cubic B-splines used to Mc Inerney T, Terzopoulos D (1997). Medical image
straighten text lines. segmentation using topologically adaptable surfaces.
Grenoble: CVRM, 23-32.
REFERENCES Serra J (1982). Image analysis and mathematical morphology.
Vol 1. Academic Press.
Angella F, Lavialle O, Baylou P (1998). A deformable and Sethian JA (1996). Level set methods. Evolving interfaces
expansible tree for structure recovery. Proccedings in geometry, fluid mechanics. Computer Vision and
IEEE-ICIP 98, Chicago, 1:241-5. Material Science. Cambridge: University Press.
Cohen LD (1991). On active contour models and ballons. Szeliski R, Tonnesen D (1992). Surface modeling with
CVGIP: Image understanding 53:211-8.
oriented particle system. Proceedings SIGGRAPH,
Deriche R (1987). Using canny's criteria to derive a 26:185-94.
recursively implemented optimal edge detector. Int J Yahia HM, Berroir JP, Mazars G (1998). Fast and robust
on Comput Vision, 167-87.
level-set segmentation of deformable structures.
Dorigo M, Gambardella LM (1997). Ant colony system: a Proceedings IEEE-ICASSP, Seattle, 2765-8.

42