The British Journal of Radiology, 77 (2004), S126–S132 E 2004 The British Institute of Radiology

DOI: 10.1259/bjr/17464219

Image filtering techniques for medical image post-processing:

an overview
1,2
C P BEHRENBRUCH, PhD, 1S PETROUDI, MSc, 1S BOND, MA, 2J D DECLERCK, PhD,
2,3
F J LEONG, MD, PhD, ARPS and 1J M BRADY, PhD, FRS, FREng
1
Medical Vision Laboratory, Engineering Science, Oxford University, Parks Road, Oxford OX1 3PJ, 2Mirada Solutions
Ltd., 23–38 Hyth Bridge Street, Oxford OX1 2EP, UK and 3Department of Medical and Molecular Pharmacology,
University of California Los Angeles, Los Angeles CA, USA

Images from an ordinary consumer digital camera Noise equalization

convey information at a wide range of spatial (and
All imaging modalities, but especially those that are
temporal) scales and enable the viewer to decompose the
relevant for medical imaging, generate image noise,
image into regions that are uniform in some way (colour,
whether due to stability of a low-flip angle MRI acqui-
texture, …), recognize familiar objects, determine spatial
sition [1], ultrasound speckle [2], quantum noise in an
relationships between objects, and detect abnormalities
X-ray [3] or out of field counts in a PET scan [4]. Virtually
(e.g. textural markings on a region expected to be plain).
all imaging systems also perform filtering on the image
Though modern digital cameras are equipped with low acquisition data both at an electronic level prior to
noise electronics and excellent lenses that minimize pin- reconstruction as well as during the image reconstruction
cushion (and similar) distortions, images also contain noise phase. Indeed, much recent advancement in reconstruction
and artefacts such as red-eye in flash images. Widely techniques for 3D imaging focus on including noise removal
distributed software packages such as Photoshop provide a as part of the reconstruction optimization process [5].
set of ‘‘filtering’’ operations which enable the user to Most initial attempts at removing image noise focus on
improve the image in some way: from image smoothing ‘‘smoothing’’ the pixel or voxel data by performing some
(typically local averaging) that removes noise and high sort of local averaging function. For example, Gaussian
frequencies, sharpening that increases high frequency smoothing is an easily implemented smoothing algorithm;
content, contrast stretching, through to specialized algo- however it is clearly not desirable to locally smooth a data
rithms, for example for red-eye reduction. Such image set in all cases (effectively removing high-frequency and
filtering is designed to improve the appearance of an highly spatially localized image components). For exam-
image, relying on the human visual system to disregard ple, as we noted above, a mammographic X-ray is only
any unwanted change of content of the image. diagnostically valuable if the resolution and spatial
Medical image analysis poses a far tougher challenge. accuracy is sufficient to capture attenuation due to micro-
First, there is an even greater need for image filtering, calcifications. Therefore increasingly ‘‘smart filters’’ based
because medical images have a poorer noise-to-signal ratio on techniques such as anisotropic diffusion [6], which
than scenes taken with a digital camera, the spatial smoothes the image to different extents in the direction of
resolution is often frustratingly low, the contrast between the intensity gradient (across a boundary) and along the
anatomically distinct structures is often too low to be boundary, or wavelets [7] (a standard but highly mathe-
computed reliably using a standard image processing matical reference) or [8] (an easily understood introduction
technique, and artefacts are common (e.g. motion and bias to the Matlab wavelet toolkit) are very useful because
field in MRI). Second, changes to image content must they can remove noise from an image while recognizing
be done in a highly controlled and reliable way that does that certain noise-like components need to be preserved. In
not compromise clinical decision-making. For example, this way the entire fields of image processing and com-
whereas it is generally acceptable to filter out local bright puter vision open up to yield interesting techniques for
patches of noise, care must be taken in the case of embedding knowledge of anatomy, tissue characteristics
mammography not to remove microcalcifications. and the physics of imaging into the filtering process [9].
This paper briefly explores some of the key areas of By way of example, the images in Figure 1 show a small
development in the area of filtering in Medical Imaging segment of an X-ray mammogram that has been digitized
and how these techniques impact generally available at 50 mm and which contains two small clusters of
software packages in routine use in a diagnostic setting. calcifications and a vessel. It is evident that the image is
It is interesting to note that a great deal of image filtering extremely noisy: the visual impact of the noise being
takes place at what is usually regarded as a ‘‘pre- accentuated by visualizing the image as a surface (height
processing’’ stage in the formation of a medical image equates to brightness). By simply performing an iterative
and is relatively invisible to a radiologist. However, there local averaging or smoothing process [10], the overall
is an increasing awareness of the impact of post-processing structure of the image fragment becomes clearer. However,
algorithms – particularly filtering – in diagnostic software the precise locations of the features in the image are poorly
applications and an awareness of these types of techniques preserved, essentially because the filter cannot discriminate
is useful. between what is high-frequency noise and what is a highly

S126 The British Journal of Radiology, Special Issue 2004

Image filtering techniques

(a) (b) (c) (d)

Figure 1. (a) A small section of a 50 mm mammogram with microcalcifications and a vessel visible. (b) The unprocessed image
displayed as a surface map. (c) The filtering of the image segment using diffusion (smoothing) techniques [10] with (d) showing the
benefit using a more selective filtering approach such as a wavelet [7] which has better structure preservation.

spatially localized (and therefore also noise-like) ‘‘spike’’ removed. This type of artefact removal is the one that is
of feature which is a calcification. A ‘‘smart’’ filter, based most commonly thought of as being an image filtering
on a suitable wavelet, and which is matched to the step, but it is important to recognize that ‘‘filtering’’ has
expected shape of a microcalcification has the effect of much broader applicability.
removing the noise, by some local averaging; but, when a A good example of a filtering technique that is, in a
calcification is encountered, the image structure is better certain sense, at the opposite extreme from noise filtering is
preserved [11]. MRI bias field correction. Small variations in the magnetic
As radiology becomes overwhelmingly digital for all field introduced by the radiofrequency (RF) system (the B1
modalities, including those which have traditionally been field introduce slowly undulating (low frequency) inho-
film-based, clinicians will need to have a deeper under- mogeneities in the image which can be visually distracting),
standing of the relationship between the imaging process can impact the textural significance of an area, and
and the display of the image. A lot of filtering and post- because they can substantially reduce the contrast in
processing is performed either to enhance visual charac- different image regions, is a barrier to using any kind of
teristics of images or to make an image more quantitative. segmentation or region delineation tool which is based on
Understanding these techniques and their impact on the thresholding (assigning all of those voxels above a fixed
image characteristics is important for good decision-making. intensity to a particular tissue class). In this case, filtering
aims to remove a low-frequency component to the image,
rather than predominantly high frequency noise as in the
Bias field correction
previous section, as Figures 2 and 3.
In the previous example of filtering for ‘‘de-noising’’ an In this method, a filtering approach is used to estimate
image, the high-frequency (‘‘spiky’’) parts of the image are an intensity correction distribution, which is applied to the

Figure 2. An example of bias field correction using ‘‘smart filters’’ that can detect inhomogeneities in the image [12, 13]. The left
image shows an MRI slice of the colon with clear bias field artefacts. The right image has been significantly improved, both for
visualization and the application of quantitative and computer-aided techniques.

The British Journal of Radiology, Special Issue 2004 S127

 C P Behrenbruch, S Petroudi, S Bond et al

Figure 3. Bias field corrected images (refer to Figure 2) can have automated algorithms successfully applied to reconstruct the shape
of the colon and provide an important starting point for computer-aided detection algorithms.

image [12, 13]. This is also an example of a ‘‘smart filter’’ Visualization-driven filtering
because in order to work robustly, the filter needs to
include knowledge that the magnetic susceptibility of tissue Visual filters are likely to be the filters that are most
varies, depending on the tissue type. Therefore, in order familiar to the reader. ‘‘Low Pass’’, ‘‘High Pass’’, ‘‘Edge
for the filtering algorithm to work properly, the different Enhancement’’, BNR (noise reduction) are commonly
tissue components in the image need to be identified and a available operator options in most radiology software
series of separate filters for each tissue class created. packages. Adjusting image sharpness and contrast can
Bias field correction is an example of a fundamental image be very valuable for differentially discerning abnormalities
processing step which is critical to building computer-aided in an image. The challenge for software designers is to
tools for aiding diagnosis – such as computer-aided make these filtering actions fast and as interactive as
detection (CAD) algorithms [14] and three-dimensional possible.
(3D) visualization tools which can selectively segment Perhaps less obvious to the user of a radiological
anatomy for improved display and interpretation. A good software package who may not be expert in image analysis
example is the automated segmentation of the colon for is the relationship between image interpolation and
building tools to help detect the presence of polyps. In this filtering. The reader may be familiar with terms such as
example, an automated contouring algorithm looks for the a ‘‘nearest neighbour’’ or ‘‘bi-cubic’’ interpolation. These
intensity transition of the colon so that cleverer feature techniques are used to make low resolution data or highly
detection algorithms have a more focused search region in zoomed images appear smother and more consistent.
which to look for textural inconsistencies in the colonic These interpolation techniques are effectively filtering
volume. Equally importantly, focusing the search ensures operations which generally provide a better visual effect
that any features detected will be in precisely the part of but which also dramatically change the spatial character-
the image that is of interest to the clinician, avoiding the istics of the image data. Indeed, the true voxel accuracy of
sharp reduction in the clinician’s confidence were a feature an image can be deceptive when highly interpolated. This
to be reported in an image region that is not anatomically is illustrated in Figure 4 for a neurological PET image.
relevant. Just the same way as bias fields make diagnosis When performing filtering on top of an interpolated
more difficult for a radiologist, algorithms (which are a lot image, one is effectively performing a sequence of two or
less intelligent than radiologists…) are also confused, so more filtering operations. Users of imaging software need
this correction is critical. to be aware that it may be better to perform filtering on

Figure 4. Most radiologists do not consider different interpolation techniques as a filtering process, but it clearly is. The magnified
image segments illustrate (from left to right) ‘‘nearest neighbour’’, linear and cubic spline interpolation, respectively. These images
importantly illustrate how many basic visualization techniques can dramatically impact the presentation – and misrepresentation – of
the true image data.

S128 The British Journal of Radiology, Special Issue 2004

Image filtering techniques

(A) (B)

(C) (D)

Figure 5. (A) A low quality therapy planning CT with typically thick slab profile and hence poor through-plane resolution. (B) and
(C) illustrate the same volume with linear and spline interpolation, respectively. In this example, the spline interpolation (usually con-
sidered to be the best method) actually generates a poorer quality re-sampled image with ‘‘ringing’’ artefacts clearly visible as shown
in the magnified image D. Although a technical example, illustrated for clarity, caution is relevant to many filtering and re-sampling
functions.

Figure 6. A very nice example of physics-

based filtering where a model of the image
formation process of an X-ray mammo-
graphy image is used to construct a special
filter which can improve the noise and spa-
tial representation of microcalcification
clusters [9]. The top insert image shows
how scatter and the spectral characteristics
of the X-ray source have blurred the
image, almost completely masking the cal-
cifications (indicative of ductal carcinoma).
The bottom insert image shows a vast
improvement in the quality of the appear-
ance of the microcalcifications.

uninterpolated data than on the results of highly inter- is interpolated using cubic spline interpolation (Figure 5).
polated data – especially if the data is low resolution. In The resolution and the intensity transitions of the skull
the example below, a low resolution therapy planning CT cause (Gibbs ‘‘ringing’’) artefacts that are visually disruptive.

The British Journal of Radiology, Special Issue 2004 S129

 C P Behrenbruch, S Petroudi, S Bond et al

(a) (b)

Figure 7. Classification of a mammogram into Wolfe Patterns using textons. (a) A mammogram belonging to Wolfe class N1.
(b) The texton ‘‘labelled’’ image of the mammogram. The texton histogram corresponding to this image is used to classify the image
into the pattern Wolfe N1. This type of classification forms the basis of ‘‘next generation’’ computer-aided detection (CADe) and
diagnosis (CADi) algorithms for digital mammography.

Physics-based filtering Texture analysis

Often when deciding upon a filtering strategy to improve In normal English usage, the word ‘‘texture’’ refers to a
image quality it is important to consider the fundamental pattern of colour, shading, or material changes on a
physics of the acquisition process. For example, nuclear surface. Some textures are geometrically regular, for
medicine and PET scanners utilize scatter and out-of-plane example a check pattern. Some textures are primarily
count simulation to build models that can sharpen picture ‘‘statistical’’ in nature, for example a carpet weave of
quality. Although the model is driven by the physics and slightly different colours, in the sense that there is no
the engineering of the imaging system, the computation of obvious geometrical regularity to the pattern. Most often,
the image enhancement is essentially a smart filter. In fact, textures have both a geometrically regular component and
physics-based filtering is a really key aspect of building a a statistical component: for example, Harris Tweed or a
complete imaging system because sometimes it may be raffia pattern. In image analysis, the word texture is
more effective to model an image degrading process and generalized to refer to regions of images that are the
correct it in software, than to change hardware or acquisi- projections of spatially distributed ‘‘entities’’, for example
tion protocol. a lawn, a pebbled path, or the foliage of a tree.
A good example of this is scatter correction in digital The human visual system is remarkable in many ways;
mammography. An antiscatter grid is used to provide a but not least in its ability to discern regions of images that
degree of collimation so that image quality is sharpened are homogeneously textured. We can segment images in to
(scatter has the effect of blurring an X-ray image). The textured regions that we do not recognize as being other
trade off is that the grid attenuates the X-ray beam, so to than ‘‘textures’’, and we can recognize effortlessly regions
compensate the amount of radiation exposure to the that correspond to textures (grass, leaves, pebbles, …) that
patient is increased. The challenge of building a directly we know well. Unfortunately, it has proven almost impos-
digital system which balances image sharpness with sible to date to develop mathematical and computational
radiation dose is one that is moving strongly into the models that approach human competence in texture
domain of software, where scatter correction can be recognition and segmentation. Various models have been
partially performed in both hardware and software to developed that attempt to capture the idea of ‘‘repetition’’,
optimize both aspects. An example of image sharpening at least in a statistical sense, based essentially on estimating
through software-based scatter correction for X-ray the local image autocorrelation. Other models have been
mammography is illustrated in Figure 6 [9]. developed in the frequency domain – again attempting to

S130 The British Journal of Radiology, Special Issue 2004

Image filtering techniques

(a)

(b)

Figure 8. This figure illustrates how feature detection across a series of image frames can be used to estimate the motion of the heart. In
the set of images (a), the shape and location of the endocardium and epicardium is estimated from image features over successive frames
and then used to drive a deformation model [19, 20]. (b) The wall motion kinetics and thickening measurements between the sets of con-
tours can be used to create a map of the ‘‘activity’’ level of the myocardium which correlates well with other modalities [20].

capture the notion of regular repetition. Since a texture is Most recently, attempts have been made [16–18] to learn
localized both in the spatial domain and in the frequency texture models from a set of training examples. A
domain, it is natural to turn to recently developed mathe- computer system is presented with dozens of instances
matical tools such as wavelets [7, 8]. One particular mathe- of a texture, all of which are examples of the texture of
matical model of a texture is based on the concept of a interest, but under different illumination, orientation, and
fractal – a shape (or surface) which is self similar at all scales other typical variations, applies a bank of filters and from
(or nearly so) and whose dimension is fractional [15]. Despite the responses, builds a ‘‘texton’’ model that enables
significant effort over the past 30 years, we still lack a subsequent recognition of texture instances. Initial results
mathematical or computational model of textures that works have been encouraging (Figure 7).
well in general: rather, we have a stock of techniques that Filtering a medical image to facilitate texture classifica-
work well for classes of textures and/or applications. tion or segmentation has broad application. For example,

The British Journal of Radiology, Special Issue 2004 S131

 C P Behrenbruch, S Petroudi, S Bond et al

new form CJD tends to be accompanied by a characteristic characteristics, which may be lost in the enormous com-
texture on the surface of the cortex. In general, abnormal plexity of a medical image.
patterns, such as architectural distortions, are often the Finally, an important consideration is the fact that
first sign that pathology may be present. One instance of although imaging algorithms such as filters are generally
this is again in mammography, an idea first promulgated considered to be part of the post-processing domain, a
by Wolfe and more recently refined in the BIRADS tremendous amount of filtering takes place before the image
classification of dense breast tissue in a mammogram (or is even presented to a radiologist. To this end, a greater
the six category classification). Petroudi et al [18] have awareness of how an image is processed prior to display is
shown that the recently developed idea of learning texture increasingly important as radiology becomes digital. By
classifications can be applied to mammography and have extension, an awareness of the capabilities of image analysis
trained a system to discriminate between the Wolfe classes techniques such a filtering, may drive a greater acceptance of
with very encouraging results. the role of computer-aided radiology.

Feature detection and tracking References

1. Hashemi RH, Bradley WG. MRI the basics. Lippincott
The brief discussion about texture analysis in the Williams and Wilkins, 2002.
previous section introduced the idea that a collection of 2. Wagner RF. Statistics of speckle in ultrasound B-scans. IEEE
filters can be created to look for different types of image Trans Sonics Ultrasonics 1983;30:156–63.
characteristics and classify an image accordingly. This has 3. Highnam RP, Brady M, English R. Detecting film-screen shot
obvious applications in computer-aided detection as well noise in mammography using a model-based approach. IEEE
as a being an important algorithmic component in a lot of Trans Med Imaging 1999;18:1016–24.
imaging applications, which are designed to optimize the 4. Cherry SR, Sorenson JA, Phelps ME. Physics in nuclear
workflow around measuring things (such as accurately medicine. W B Saunders, 2003.
delineating blood vessels in an angiography study). 5. Qi J, Leahy R. Resolution and noise properties of MAP
reconstructions in fully 3D PET. IEEE Trans Med Imaging
The use of ‘‘libraries’’ of filters to track features over
2000;19:493–506.
time is particularly important in the case of dynamic 6. Weickert J. Anisotropic diffusion in image processing, ECMI
imaging or longitudinal studies. Most often, such feature Series. Stuttgart, Germany: Teubner-Verlag1998.
tracking is intended to help automate the measurement of 7. Mallat S. A wavelet tour of signal processing. Academic
a change in a feature – such as a tumour volume in Press, 1998.
response to chemotherapy. Filtering for feature tracking 8. Wavelet Toolbox User’s Guide (Matlab), The Mathworks,
also has application in studies where specific types of 1997–2000.
feature motion or change is indicative of pathology. A 9. Brady M, Highnam R. Mammographic image analysis.
good example is cardiac ultrasound where endocardial and Kluwer Series on Medical Image Understanding. Kluwer, 1999.
epicardial wall motion can be used to highlight ischaemic 10. Forsyth DA, Ponce J. Computer vision: a modern approach.
Prentice Hall, 2003.
regions of the heart [19, 20]. In order to build such wall
11. Behrenbruch CP. Image fusion for the management of breast
motion analysis tools, the image features corresponding to cancer. Doctoral Thesis, Department of Engineering Science,
the myocardium must be filtered from ultrasonic speckle University of Oxford, 2002.
noise over successive imaging frames (Figure 8) [1]. 12. Guillemaud R, Brady M. Estimating the bias field of MR
As a general note, ultrasound is a modality that is images. IEEE Trans Med Imaging 1997;16:238–51.
heavily dependent on filtering for optimal clinical use, both 13. Styner M. Parametric estimate of intensity inhomogeneities
as a visual post-processing step as well as for building applied to MRI. IEEE Trans Med Imaging 2000;19:153–65.
quantitative applications. A tremendous amount of 14. Hayton P, Brady M, Smith S, Moore N. A non-rigid
filtering is used to control the visualization of harmonics registration algorithm for dynamic breast MR images.
and even techniques such as ‘‘colour flow’’ or Doppler Artificial Intelligence 1999;114:125–56.
15. Dubuisson MP, Dubes RC. Efficacy of fractal features in
imaging are dependent on the good separation of temporal
segmentation images of natural textures. Pattern Recognition
and spatial echo returns – itself a filtering operation. Letters 1994;15:419–31.
16. Leung T, Malik J. Representing and recognizing the visual
appearance of materials seeing three-dimensional textons. Int
Conclusions J Computer Vision 2001;43:29–44.
17. Varma M, Zisserman A. Classifying images of materials:
The field of medical image analysis is extremely broad achieving viewpoint and illumination independence.
and filtering is a fundamental aspect of virtually every Proceedings of the European Conference on Computer
implementation of an image analysis and visualization Vision, Copenhagen, Denmark. 2002;255:271.
solution. Filtering is a system component of all medical 18. Petroudi S, Kadir T, Brady M. Automatic classification of
imaging modalities and as illustrated in this brief paper, a mammographic parenchymal patterns: A statistical approach.
basic part of many diagnostic applications. In: Proceedings of EMBC, International Conference on
This overview paper has tried to address some of the Engineering in Medicine and Biology. IEEE, 2003:798–801.
misconceptions about medical image post-processing and 19. Jacob G, Noble JA, Behrenbruch C, Kelion AD, Banning AP. A
shape-space-based approach to tracking myocardial borders
the idea that filtering is somehow about reducing the
and quantifying regional left-ventricular function applied in
diagnostic value of an image. It is true that filtering can be echocardiography. IEEE Trans Med Imaging 2002;21:226–38.
inappropriately used and can have a negative impact on 20. Boukerroui D, Noble JA, Robini MC, Brady M.
the diagnostic content of an image (a few simple examples Enhancement of contrast regions in suboptimal ultrasound
were shown). However, ‘‘smart’’ filters can significantly aid images with application to echocardiography. Ultrasound
diagnosis by highlighting regions and accentuating image Med Biol 2001;27:1583–94.

S132 The British Journal of Radiology, Special Issue 2004