Digital Image Classification

Image Processing and Interpretation

Chapter 5
Chapter Objectives

Steps used in image classification

Hardy Vs Soft (Fuzzy) classifiers
Supervised vs Unsupervised
Methods of Supervised Classification
Methods of Unsupervised Classificstion
Methods of accuracy assessment
Two calculation examples of accuracy assessment
 Introduction
Image classification is the basis for many environmental studies
Classifying RS data into a thematic map remains a challenge because many
factors can affect the success of a classification. Such factors
include:
- the complexity of the landscape in a study area
- selected remotely sensed data and,
- image-processing and classification approaches,

The continuous emergence of new classification algorithms makes it

difficult to have a comprehensive up-to-date review.
An analysis of the major methods can nonetheless provide an idea of a
suitable classification for a particular study.
 Multispectral image classification

• Very widely used method of

extracting thematic
information
• Use multispectral (and
other) information
• Separate different land
cover classes based on
spectral response, texture,
….
• i.e. Pattern recognition
Spectral response curves

Multispectral image Space

 The classification process
Aim of the classification process: categorize all pixels in a digital image
into one of several land cover classes.
The spectral profile for each pixel within the scene is used as the
numerical basis for categorization.
Image classification requires a number of steps:
- careful selection of datasets,
- selection of classifications systems,
- data pre-processing,
- careful feature extraction and selection of test areas,
- post classification processing and evaluation of the classification
process,
- Evaluation of classification accuracy.
 Suitable data selection
First important step in the classification process: selection of suitable data
sets. RS data vary in radiometric, spatial, spectral resolutions
The data choice must suit the envisaged output scale, image resolution and
users’ needs:
At local level, a fine-scale classification system is needed for a
classification, requiring high spatial resolution data e.g. IKONOS and
SPOT 5 HRG.
At regional scale, medium spatial resolution data such as Landsat
TM/ETM+/L8, and Terra ASTER are sufficient.
At a continental or global scale, coarse spatial resolution data such as
AVHRR, MODIS, and SPOT Vegetation can be used.
Atmospheric conditions also affect the choice of data. For instance,
RADAR may be preferable in areas of frequent cloud cover
Choice of classification
systems and training samples
Classification system are designed based on the user needs, spatial
resolution of selected remotely sensed data, compatibility with
previous work, image-processing and classification algorithms
available, and time constraints.
Collecting a sufficient number of training areas is important for image
classification. Training samples are usually collected from fieldwork,
or from fine spatial resolution aerial photographs and satellite images
Collection strategies could be per-pixel, or use of polygons.
In heterogeneous landscapes there is an increased risk of sampling
mixed pixels.
Important when sampling: consider the spatial resolution of the remote-
sensing data, availability of ground reference data and complexity of
the scene’s landscape.
Data pre-processing
Image pre-processing may include the detection and restoration of bad
lines, geometric rectification or image registration, radiometric
calibration and atmospheric correction, as well as topographic
correction.
If a single-date image is used in classification, atmospheric correction
may not be necessary.
However, when multi-temporal or multisensor data is used,
atmospheric calibration must be carried out
A wide range of methods, ranging from simple relative calibration and
dark-object subtraction to calibration approaches based on complex
models, have been developed for radiometric and atmospheric
normalization and correction
Topographic correction may by required if the study area is located in
rugged or mountainous regions.
Feature extraction and selection
Choosing suitable variables and input bands is a critical step for
successfully implementing an image classification.
Input variables may be include spectral signatures, vegetation indices,
transformed images, textural or contextual information, multitemporal
images, multisensor images, as well as ancillary data.
Additional bands like PC bands, etc. Some of these methods reduce data
redundancy whilst others extract specific land cover types or
information.
Lastly, the optimal selection of spectral bands for classifications is
imperative. Graphical analysis (e.g. spectral plots) and statistical methods
are typically used to find optimal bands.
 Feature extraction and selection

In this spectral plot it is evident that band 4 is best for the extraction
of feature A, and band 7 is best for the extraction of feature B.

Source: http://www.sc.chula.ac.th/courseware/2309507/Lecture/remote18.htm
Selection of suitable classification method
Spatial resolution of the RS data, different sources of additional data, a
classification system, and availability of classification software must be
taken into account when selecting a classification method for use.

Classification results vary from the use of different classifiers.

A more detailed summary of the common classification methods will be
presented later.
Post classification processing
Classification result problems:
Traditional per-pixel classifiers may lead to ‘salt and pepper’ effects in
classification maps. A filtering algorithm (e.g. mean) is often used to
smooth the image and reduce the noise.
Additionally, because of the complexity of biophysical environments,
spectral confusion is common among land-cover classes. In such cases,
ancillary data can be used to modify the classification image.
For instance, forest distribution in hilly or mountainous areas is
related to elevation, slope, and aspect. Thus, data describing terrain
characteristics and the knowledge of specific vegetation classes and
topographic factors can be useful when modifying classification
results.
Post-classification processing is an important step in improving the quality
of classifications.
Evaluating classification experience
Different approaches are possible, e.g. a qualitative evaluation based on
expert knowledge or a quantitative accuracy assessment based on
sampling within the classified image.

Criteria for the evaluation of classification accuracy: Accuracy,

reproducibility, robustness, ability to fully use the information
content of the data, uniform applicability, and objectiveness.
Criteria for the evaluation of the suitability of algorithms:
Classification accuracy, computational resources, stability of the
algorithm, and robustness to noise in the training data.

Notably, classification accuracy assessment is the most common approach

for an evaluation of classification performance.
Approaches to Classification

Multispectral classification may be performed using a variety of

methods, including:

- use of hard (crisp) or soft (fuzzy) classification logic,

- The use of supervised or unsupervised classification logic,

- use of per-pixel or object-based classification logic, and

- hybrid approaches
Hard VS Fuzzy Classifiers

Hard Classifiers

Hard discrete categories are produced, e.g. forest, agriculture

Fuzzy Classifiers

Takes into account the heterogeneity nature of the real word

Thematic output will have fuzzy information

Figure: Fuzzy Classification
Supervised classification
Requires significant user input/expertise

Identity and location of land cover types are known Priori

- Fieldwork
- Interpretation form Aerial Maps
- Map analysis
- Personal experience
From Lillesand, Kiefer and Chipman (2015)
Supervised classification: classification stage

•Minimum distance to means (MDM)

• Need rule(s) to decide into which class we put given pixel

• e.g. Minimum distance to means (MDM)
– for each land cover class, calculate the mean vector in feature
space (i.e. the mean value in each waveband)
- Calculate the mean distance to each mean vector from the
unknown pixel
– Put every pixel into nearest class/cluster
– define a limit beyond which a pixel remains unclassified
• a simple and fast technique but has major limitations…
 Supervised classification: MDM

• Find closest cluster mean for

each pixel
• Simple and quick BUT what
about points 1, 2?
• i.e. MDM insensitive to variance
of clusters
• Can we improve?

21
 Supervised classification: parallelepiped (‘box’)

• Assign boundaries around the spread of a class in feature space i.e. take
account of variance
• typically use minimum/maximum of DN in a particular class to define
limits, giving a rectangle in 2D, box in 3D (if we have > 2 bands) etc.
• pixels outside of these regions are unclassified (which is good or bad,
depending on what you want!!)
• problems if class regions overlap or if high covariance between different
bands (rectangular box shape inappropriate)
– can modify algorithm by using stepped boundaries with a series of
rectangles to partially overcome such problems
• simple and fast technique
• takes some account of variations in the variance of each class
 Supervised classification: parallelepiped (‘box’)

Simple boxes defined by min/max

limits of each training class. But
overlaps……..? 15
 Supervised classification: Gaussian maximum likelihood

• assumes data in a class are (unimodal) Gaussian (normal)

distributed
– class then defined through a mean vector and covariance
matrix
– calculate the probability of a pixel belonging to any class
using probability density functions defined from this
information
– we can represent this as equiprobability contours & assign a
pixel to the class for which it has the highest probability of
belonging to

24
 Supervised classification: Gaussian maximum likelihood

• Now we use probability rather than distance in feature space

• Which class is each pixel “most likely” to belong to??

FIGURE . Maximum likelihood classification. These frequency distributions represent

pixels from two training fields; the zone of overlap depicts pixel values common to both
categories. The relation of the pixels within the region of overlap to the overall frequency
distribution for each class defines the basis for assigning pixels to classes. Here, the
relationship between the two histograms indicates that the pixel with the value “45” is more
likely to belong to the Forest (“F”) class rather than the Crop (“C”) class.

25
Unsupervised Classification (clustering)

Little input from user required (few assumptions)

- BUT means results hard to interpret (may not represent classes we
recognise)
- cluster pixels in feature space based on some measure of their
proximity

interpretation of results / assigned classes

- can be useful, e.g. in picking up variations within what would otherwise
be distinguished as a single class e.g. stressed/unstressed crop in a
single field)

- clusters can be of little intrinsic value in themselves

e.g. sunlit trees, shaded trees is perhaps not a useful discrimination if one
simply wants to classify 'trees', and so clusters may have to be combined
 Unsupervised Classification: K-means

• A large number of clustering algorithms exist

• K-means
– input number of clusters desired
– algorithm typically initiated with arbitrarily-located 'seeds' for
cluster means
– each pixel then assigned to closest cluster mean
– revised mean vectors are then computed for each cluster
– repeat until some convergence criterion is met (e.g. cluster means
don't move between iterations)
– computationally-expensive because it is iterative

27
Unsupervised classification: ISODATA (Iterative self-organising data
analysis) algorithm

• Same as K-means but now we can vary number of clusters (by

splitting / merging)
– Start with (user-defined number) randomly located clusters
– Assign each pixel to nearest cluster (mean spectral distance)
– Re-calculate cluster means and standard deviations
– If distance between two clusters < some threshold, merge
them
– If standard deviation in any one dimension > some threshold,
split into two clusters
– Delete clusters with small number of pixels
– Re-assign pixels, re-calculate cluster statistics etc. until
changes of clusters < some fixed threshold

28
 ISODATA example: 2 classes, 2 bands

DN Cluster means move towards

DN Initial cluster Ch 1 pixels 1 and 2 respectively
a
Ch 1 means Assign pixel 1 to
a
cluster a, 2 to b etc. Pixel 2

Pixel 2 Pixel 1
Pixel 1 b

DN
DN Ch 2
Ch 2

DN DN
Ch 1 All pixels Ch 1
assigned to a or SD of cluster a Split a into 2,
b - update stats too large? recalculate.
Repeat….
New positions of
New positions of
cluster means
cluster means DN
DN
Ch 2
Ch 2
29
 Hybrid Approaches

• useful if large variability in the DN of individual classes

• use clustering concepts from unsupervised classification to derive sub-
classes for individual classes, followed by standard supervised methods.
• can apply e.g. K-means algorithm to (test) subareas, to derive class
statistics and use the derived clusters to classify the whole scene
• requirement that all classes of interest are represented in these test
areas
• clustering algorithms may not always determine all relevant classes in an
image e.g. linear features (roads etc.) may not be picked-up by the
textural methods described above

30
Assessing classification accuracy
Accuracy assessment should be an important part of any classification.
Usual procedure: compare the classification with actual reference data that
is known to accurately reflect the true land-cover.
Such reference data could include field or ground truth, higher
resolution satellite images or aerial photos, topographical maps derived
from aerial photo interpretation and GIS vector datasets.
It is important to choose an appropriate sampling method and sample
size. There must be at least some observations for each class.
The accuracy assessment is a reflection of the difference between the
classification and the corresponding reference data.
Assessing classification accuracy
Ideally, accuracy assessment should not be based on the training pixels.
Reason: training pixels are usually not randomly selected and the
classification itself depends on these training pixels.
Increasing sample size results in higher precision of the estimated
accuracy measures. Decreasing sample size will lower the precision.
The Confusion Matrix
One of the most common methods of accuracy assessment is the use of a
confusion matrix. It consists of a table that relates the classes of the
classified pixels to their actual classes based on reference data.
The producer’s accuracy is referred to as the omission error. It describes
how well a certain area can be classified. The producer’s accuracy refers to
the probability that a certain land-cover of an area on the ground is
classified correctly
User’s accuracy is referred to as the commission error. The user’s accuracy
refers to the probability that a pixel labelled as a certain land-cover class on
the classification map really belongs to this class.
Overall accuracy provides an estimate of the ‘correctness’ of the
classification after talking both user and producer accuracy into account.
Kappa accuracy reflects the difference between actual agreement and the
agreement expected by chance. Kappa of 0.65 means there is a 65% better
agreement than by chance alone
Confusion matrix

In this table there were 61 possible pixels of water but only 46 were
correctly attributed to water. Thus, 15 were omitted (omission error)
On the other hand 50 pixels were attributed to water but 4 of them (2 of
agriculture and 2 of forest) were mistakenly included (commission error)
The overall classification accuracy is calculated by adding up all the
correctly classified pixels along the diagonal and dividing by the total
number of pixels:
46 + 37 + 44 x 100 = 84,7%
150
Source: http://www.yale.edu/ceo/OEFS/Accuracy_Assessment.pdf
Confusion matrix

there were 61 possible pixels of water but only 46 were correctly

attributed to water. Thus, 15 were omitted (omission/producer error)
46 x 100 = 75,4 %
61
50 pixels were attributed to water but 4 of them (2 of agriculture and
2 of forest) were mistakenly included (commission/user error)
46 x 100 = 92 %
50

Source: http://www.yale.edu/ceo/OEFS/Accuracy_Assessment.pdf
Confusion matrix

Kappa = observed accuracy – chance agreement

1 – chance agreement
Observed accuracy is determined by the diagonal. Chance agreement
incorporates the sum of the product between the row and column totals
Total number of pixels = 150
Observed accuracy = 46+37+44 = 127
Chance agg trace. = (50x61)+(50x40)+(50x 49) = 7500
Kappa = 150 (127) – 7500 = 77 %
150² - 7500
Source: http://www.yale.edu/ceo/OEFS/Accuracy_Assessment.pdf
Taxonomy of image classification methods
Criteria Categories Characteristics Example of classifiers
Whether training Supervised classification Land cover classes are defined. Sufficient reference data are Maximum likelihood, minimum
samples are used or approaches available and used as training samples. The signatures distance, artificial neural
not generated from the training samples are then used to train network, decision tree
the classifier to classify the spectral data into a thematic map. classifier.
Unsupervised Clustering-based algorithms are used to partition the spectral ISODATA, K-means clustering
classification image into a number of spectral classes based on the statistical algorithm.
approaches information inherent in the image. No prior definitions of the
classes are used. The analyst is responsible for labelling and
merging the spectral classes into meaningful classes.

Whether parameters Parametric classifiers Gaussian distribution is assumed. The parameters (e.g. mean Maximum likelihood, linear
such as mean vector vector and covariance matrix) are often generated from discriminant analysis.
and covariance training samples. When landscape is complex,
matrix are used or parametric classifiers often produce ‘noisy’ results. Another
not major drawback is that it is difficult to integrate ancillary
data, spatial and contextual attributes, and non-statistical
information into a classification procedure.

Non-parametric No assumption about the data is required. Non-parametric Artificial neural network,
classifiers classifiers do not employ statistical parameters to calculate decision tree classifier,
class separation and are especially suitable for incorporation evidential reasoning, support
of non-remote-sensing data into a classification procedure. vector machine, expert
system.
Which kind of pixel Per-pixel classifiers Traditional classifiers typically develop a signature by combining Most of the classifiers, such as
information is used the spectra of all training-set pixels from a given feature. The maximum likelihood,
resulting signature contains the contributions of all materials minimum distance, artificial
present in the training-set pixels, ignoring the mixed pixel neural network, decision tree,
problems. and support vector machine.
Subpixel classifiers The spectral value of each pixel is assumed to be a linear or Fuzzy-set classifiers, subpixel
non-linear combination of defined pure materials (or classifier, spectral mixture
endmembers), providing proportional membership of each analysis.
pixel to each endmember.
Source: Lu, D. & Weng, Q. 2007.
Taxonomy of image classification methods
Criteria Categories Characteristics Example of classifiers
Which kind of pixel Object-oriented classifiers Image segmentation merges pixels into objects and classification eCognition.
information is used is conducted based on the objects, instead of an individual
pixel. No GIS vector data are used.
Per-field classifiers GIS plays an important role in per-field classification, integrating GIS-based classification
raster and vector data in a classification. The vector data are approaches.
often used to subdivide an image into parcels, and
classification is based on the parcels, avoiding the spectral
variation inherent in the same class.
Whether output is a Hard classification Making a definitive decision about the land cover class that each Most of the classifiers, such as
definitive decision pixel is allocated to a single class. The area estimation by hard maximum likelihood,
about land cover classification may produce large errors, especially from coarse minimum distance, artificial
class or not spatial resolution data due to the mixed pixel problem. neural network, decision tree,
and support vector machine.
Soft (fuzzy) classification Providing for each pixel a measure of the degree of similarity for Fuzzy-set classifiers, subpixel
every class. Soft classification provides more information and classifier, spectral mixture
potentially a more accurate result, especially for coarse spatial analysis.
resolution data classification.
Whether spatial Spectral classifiers Pure spectral information is used in image classification. A Maximum likelihood, minimum
information is used ‘noisy’ classification result is often produced due to the high distance, artificial neural
or not variation in the spatial distribution of the same class. network.
Contextual classifiers The spatially neighbouring pixel information is used in image Iterated conditional modes,
classification. point-to-point contextual
correction, frequency-based
contextual classifier.
Spectral-contextual Spectral and spatial information is used in classification. ECHO, combination of para
classifiers Parametric or non-parametric classifiers are used to generate metric or non-parametric and
initial classification images and then contextual classifiers are contextual algorithms.
implemented in the classified images.
Source: Lu, D. & Weng, Q. 2007.