Image and Video Processing

Colour Image Processing

2 Motivation

• Color is a powerful descriptor

that often simplifies object
identification and extraction from
a scene.
• Human can discern thousands
of color shades and intensities,
compared to about only two
dozen shades of gray.
3 Contents
We’ll look at color image processing, covering:
– Color fundamentals and models
• Color spectrum vs. electromagnetic spectrum
• CIE standard, R, G, B as the primary colors
• Chromaticity diagram
• Color models – RGB, HSI etc.
– Color based Processing and application in
machine vision
5 Some Questions?
• What does it mean when we say an object has a
certain color?
 What is color?
• The result of interaction
between physical light in the
environment and our visual
system.
• A psychological property of
our visual experiences when
we look at objects and lights,
not just the objective physical
property of those objects or
6
lights.
Color and light
• Color of light arriving at camera depends on
• Spectral radiance of light falling on that patch
• Spectral reflectance of the surface (that light is leaving)

• Color perceived depends on

• Physics of light
• Visual system receptors
• Brain processing, environment
7
8 Some Questions?
• What does it mean when we say an object has a
certain color?
• Why are the primary colors of human vision red,
green, and blue?
9 Colors of human vision
The colors that humans and most animals
perceive in an object are determined by Colours
Absorbed
the nature of the light reflected from the
object
Chromatic light spans the electromagnetic spectrum from
approximately 400 to 700nm
10 Colors of human vision (cont..)
• As we mentioned before human colour vision is
achieved through 6 to 7 million cones in each eye
•Approximately 66% of these cones are sensitive to
red light, 33% to green light and 3% to blue light
• For this reason, red, green, and blue are referred to
as the primary colors of human vision, thus the term
trichromacy.
11 Some Questions?
• What does it mean when we say an object has a
certain color?
• Why are the primary colors of human vision red,
green, and blue?
• Is it true that different proportions of red, green, and
blue can produce all the visible color?
12 Color matching experiments
• Goal: find out what spectral radiances produce
same response in human observers.

12
 13 Color matching experiments

Observer adjusts weight (intensity) for primary

lights to match appearance of test light.
Slide credit: Kristen Grauman Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 After Judd & Wyszecki.
 14 Color matching experiments
• Goal: find out what spectral radiances produce
same response in human observers.
• Assumption: simple viewing conditions, where
we say test light alone affects perception
– Ignoring additional factors like adaptation, complex
surrounding scenes, etc.

14
Slide credit: Kristen Grauman
 Color matching experiment 1

15
Slide credit: Bill Freeman
 Color matching experiment 1

p1 p2 p3
Slide credit: Bill Freeman
16
 Color matching experiment 1

p1 p2 p3
Slide credit: Bill Freeman
17
 Color matching experiment 1
The primary color
amounts needed
for a match

p1 p2 p3
Slide credit: Bill Freeman
18
19 CIE Chromacity Diagram
• Different colors can be systemati-
cally specified using the CIE
chromacity diagram (a projection of
XYZ color space)
• On this diagram the x-axis roughly
represents the proportion of red and
the y-axis represents the proportion of
red used
•The proportion of blue used in a
color is calculated as:
z = 1 – (x + y)
20 CIE Chromacity Diagram (cont…)
Green: 62% green,
25% red and 13%
blue
Red: 32% green,
67% red and 1%
blue
21 CIE Chromacity Diagram (cont…)
• Any colour located on the boundary of the chromacity
chart is fully saturated
• The point of equal energy has equal amounts of each
colour and is the CIE standard for pure white
• Any straight line joining two points in the diagram
defines all of the different colours that can be
obtained by combining these two colours additively
• This can be easily extended to three points
22 CIE Chromacity Diagram (cont…)
This means the entire colour
range cannot be displayed
based on any three colours
The triangle shows the
typical colour gamut
produced by RGB monitors
The strange shape is the
gamut achieved by high
quality colour printers
23 Some Questions?
• What does it mean when we say an object has a
certain color?
• Why are the primary colors of human vision red,
green, and blue?
• Is it true that different proportions of red, green, and
blue can produce all the visible color?
• What kind of color model is the most suitable one to
describe human vision? What is more suitable for
digital processing?
24 Colour Models
From the previous discussion it should be
obvious that there are different ways to model
colour
We will consider two very popular models
used in colour image processing:
– RGB (Red Green Blue)
– HSI (Hue Saturation Intensity)/HSV ((Hue
Saturation Value)
25 RGB
In the RGB model each colour appears in its primary
spectral components of red, green and blue
The model is based on a Cartesian coordinate system
– RGB values are at 3 corners
– Cyan magenta and yellow are at three other corners
– Black is at the origin
– White is the corner furthest from the origin
– Different colours are points on or inside the cube represented
by RGB vectors
26 RGB (cont…)
27 RGB (cont…)
Images represented in the RGB colour model consist
of three component images – one for each primary
colour
When fed into a monitor these images are combined to
create a composite colour image
The number of bits used to represent each pixel is
referred to as the colour depth
A 24-bit image is often referred to as a full-colour
image as it allows (2 ) = 16,777,216 colours
3
8
28 RGB (cont…)
29 The HSI Colour Model
• RGB is useful for hardware implementations and is
related to the way in which the human visual system
works
• However, RGB is not a particularly intuitive way in
which to describe colours
• Rather when people describe colours they tend to
use hue, saturation and brightness
• So, HSI model is particularly good for colour
description and manipulation
30 The HSI Colour Model (cont…)
The HSI model uses three measures to describe
colours:
– Hue: dominant color perceived by an observer (say
yellow, orange or red)
– Saturation: Gives a measure of relative purity or how
much white is mixed with a pure colour (hue)
– Intensity: Intensity is the same achromatic notion that
we have seen in grey level images.
31 HSI, Intensity & RGB
• Intensity can be extracted from RGB images –
which is not surprising if we stop to think about it
• Remember the diagonal on the RGB colour cube
that we saw previously ran from black to white
• Now consider if we stand this cube on the black
vertex and position the white vertex directly above
it
32 HSI, Intensity & RGB (cont…)
Now the intensity component
of any colour can be Intensity
line
determined by passing a
plane perpendicular to
the intenisty axis and
containing the colour
point
The intersection of the plane
with the intensity axis gives us the intensity component of
the colour
33 HSI, Hue & RGB
In a similar way we can extract the hue from the RGB
colour cube Colors on this triangle
Have the same hue
Consider a plane defined by the
three points cyan, black and white
saturation
All points contained in
this plane must have the
same hue (cyan) as black
and white cannot contribute
hue information to a colour
34 The HSI Colour Model
Consider if we look straight down at the RGB cube as it was
arranged previously
We would see a hexagonal shape
with each primary colour separated
by 120°and secondary colours
at 60°from the primaries
So the HSI model is composed of
a vertical intensity axis and the
locus of colour points that lie on planes perpendicular to that axis
35 The HSI Colour Model (cont…)
To the right we see a hexagonal
shape and an arbitrary colour
point
– The hue is determined by an angle
from a reference point, usually red
– The saturation is the distance from the origin to the point
– The intensity is determined by how far up the vertical
intensity axis this hexagonal plane sits (not apparent
from this diagram
36 The HSI Colour Model (cont…)
Because the only important things are the angle
and the length of the saturation vector this
plane is also often represented as a circle or a
triangle
37 Problem of using Hue component

dis-continuous

Un-defined
over gray
axis
38 Converting From RGB To HSI
Given a colour as R, G, and B its H, S, and I
values are calculated as follows:
 

H =
if B  G
 = cos 
−1
 (R − G) + (R − B) 
1
2

 360 −  if B  G
 
1

 (R − G ) (
2
+ R − B )(G − B ) 
2

 

3
S = 1− min (R,G, B) I= 1
(R + G + B)
(R + G + B) 
3


 39 Converting From HSI To RGB
Given a colour as H, S, and I it’s R, G, and B
values are calculated as follows:
– RG sector (0 <= H < 120°)
 S cos H 
R = I1 +  G = 3I − (R + B) B = I (1− S)
 cos(60 − H )

– GB sector (120

° <= H < 240°)
  S cos(H − 120)
R = I (1− S) G = I1 +  B = 3I − (R + G)
 cos (H − 60) 
40 Converting From HSI To RGB (cont…)
– BR sector (240° <= H <= 360°)
 S cos(H − 240)
R = 3I − (G + B) G = I (1− S) B = I1 + 
 cos(H − 180)



41 HSI & RGB

RGB Colour Cube

H, S, and I Components of RGB Colour Cube

42 Quiz
• Consider the following RGB image, in which the
squares are fully saturated and each of the colors
is at maximum intensity. An HSI image is
generated from this image. Describe the
appearance of each HSI component image.
43 Manipulating Images In The HSI Model
In order to manipulate an image under the HSI
model we:
– First convert it from RGB to HSI
– Perform our manipulations under HSI
– Finally convert the image back from HSI to RGB
RGB RGB
HSI Image
Image Image

Manipulations
44 RGB -> HSI -> RGB

RGB Hue
Image

Saturation Intensity
45 RGB -> HSI -> RGB (cont…)

Hue Saturation

Intensity RGB
Image
46

Color based Processing and

application in machine vision
47 Pseudo-color image processing
• Assign colors to gray values based on a specified criterion
• For human visualization and interpretation of gray-scale
events
Color 1
• Intensity slicing

Color 2

Image
plane
48 Intensity slicing (cont.)
• Alternative representation of intensity slicing
49 Intensity slicing (cont.)
• More slicing plane, more colors
50 Example Applications

Radiation test pattern 8 color regions

* See the gradual gray-level changes

51 Example Applications

Rainfall statistics
52 Full Color Image Processing
• A pixel at (x,y) is a vector in the color space
– RGB color space
 R ( x, y ) 
c( x, y ) = 
G ( x , y ) 


 B ( x, y ) 
c.f. gray-scale image
f(x,y) = I(x,y)
53 How to deal with color vector?
• Per-color-component processing
– Process each color component
• Vector-based processing
– Process the color vector of each pixel
• When can the above methods be equivalent?
– Process can be applied to both scalars and vectors
– Operation on each component of a vector must be
independent of the other component
55 Color transformation
• Similar to gray scale transformation
– g(x,y)= T [f(x,y)]
• Color transformation
si = Ti (r1 , r2 ,..., rn ) , i = 1,2,..., n
g(x,y) f(x,y)
s1 T1 r1
s2 T2 r2
… … …
sn Tn
rn
56 Which color model in color transformation?

• RGB  HSI
• Theoretically, any transformation can be
performed in any color model
• Practically, some operations are better
suited to specific color model
57 Ex: modify intensity of a color image
• Example: g(x,y)=k f(x,y), 0<k<1
• HSI color space
– Intensity: s3 = k r3
– Note: transform to HSI requires complex
operations
• RGB color space
– For each R,G,B component: si = k ri
58
59

I H,S
60
Histogram Processing

Histogram
Equalizing the
Intensity
61 Color image smoothing
• Neighborhood processing
62 Color image smoothing: averaging mask

1
c( x, y ) =
K
 c( x, y )
( x , y )S xy
vector processing

Neighborhood
Centered at (x,y)
1 
  R ( x, y ) 
K ( x , y )S xy 
1 
c( x, y ) =   G ( x , y )  per-component processing
K ( x , y )S xy 
1 
K  B ( x , y ) 
 ( x , y )S xy 
63
original R

G G
64 Example: 5x5 smoothing mask
RGB model Smooth only I
in HSI model difference
65 Color slicing
• Highlighting/Extracting a specific range of colors in an
image
• Use the region defined by the colors as a mask for
further processing
• Recall the gray level slicing
66 Implementation of color slicing
• How to take a region of colors of interest?

prototype color prototype color

Sphere region Cube region

67 Implementation of color slicing
1. Colors of interest are enclosed by cube (or hypercube for n>3)
  W
0.5 if  r j − a j 
si =   2
 any1 j  n , i = 1,2,..., n
r
 i otherwise

2. Colors of interest are enclosed by Sphere

 n
 0.5
si = 
if  (r
j =1
j − a j ) 2  R02
, i = 1,2,..., n

ri otherwise
68 Example

cube sphere
69 Color-based extraction for robot soccer

Towards Eliminating Manual Color Calibration at RoboCup. Mohan

Sridharan and Peter Stone. RoboCup-2005: Robot Soccer World Cup IX,
Springer Verlag, 2006
http://www.cs.utexas.edu/users/AustinVilla/?p=research/auto_vis
69
Color as a low-level cue for content based Image Retrieval (CBIR)

Pixel counts
R G B
• Color histograms:
Use distribution of
Color intensity colors to describe
image

• No spatial info –
invariant to
translation,
rotation, scale
70
 Color-based image retrieval
• Given collection (database) of images:
– Extract and store color histogram(s) per image
• Given new query image:
– Extract its color histogram(s)
– For each database image:
• Compute intersection between query and database
histogram(s)
– Sort intersection values (highest score = most similar)
– Rank database items relative to query based on this sorted
order
 Color-based image retrieval

Example database 72
Slide credit: Kristen Grauman
 Color-based image retrieval

Example retrievals 73
Slide credit: Kristen Grauman
 Color-based image retrieval

Example retrievals 74
Slide credit: Kristen Grauman
 Color-based skin detection

M. Jones and J. Rehg, Statistical Color Models with Application to Skin Detection, IJCV 2002.
 Generative Modelling
Percentage
of skin pixels
P(R| skin) in each bin

Bayes’ theorem Feature R = Hue

P(R| not skin)