UNIT 5 (Part 2)- Range Image Processing

Range
data
 Range data is a 2-1/2 D or 3-D representation of the
scene.
 An image d(i, j), which records the distance d to the
corresponding scene point (X, Y, Z) for each image pixel
(i, j).
 It could be provided as a set of 3-D scene points
(point cloud).

2
Imaging
Techniques
 Passive imaging.

Stereo imaging
 Active range sensing
o Time-of-flight sensors
o Triangulation-based
sensors
o Structured Light
 Active range sensors
• Active range sensors are devices that use emitted
energy, such as light or sound, to measure the distance
between the sensor and an object.
• These sensors are commonly used in various
applications, including robotics, industrial automation,
automotive systems, and more.
• They work by emitting a signal, measuring the time it
takes for the signal to bounce off an object and return,
and then using this information to calculate the
distance.
 Time-of-Flight (ToF) Sensors
• ToF sensors use light, typically infrared, to measure the
time it takes for a light pulse to bounce off an object
and return.
• They provide accurate distance measurements and are
used in applications like gesture recognition,
augmented reality, and indoor navigation.
 Time-of-Flight Range
Sensors
Source
t: Time taken to travel and
the forward and return detector
path. v: speed of light collocate
Use a
in the
given moving d.
medium. mirror to Pulse
Distance: scan the d
beam. laser
d: (v x t)/2
Laser-based time-of- Limitation:
flight range sensors: o the minimum observation
light detection and time
ranging (LIDAR) or thus limited by the
laser radar (LADAR) minimum distance
sensors.
 Triangulation-based Sensors
• Triangulation-based active range sensors are a category
of active sensors that determine the distance to an
object by measuring the angle and position of the
reflected signal.
• Triangulation is a geometric principle used to calculate
distances by forming a triangle between the sensor, the
object, and the point of reflection.
• These sensors are widely used in various applications,
including industrial automation, 3D scanning, robotics,
and more.
 Triangulation based Sensors
Known for a
The scanning path of the
predetermined
camera beam.
and light
(X,Y,Z
source ) Apply
are triangulation to
(i,j
calibrated ) get the 3D point.
(u,v
. )

Observe
d in A light
camera.
Camera
Yi-Chih Hsieh, Decoding structured light patterns for three-dimensional imaging systems, Pattern Recognition 34 (2001) 343-349 .
 Imaging
principle
Encoding the
position of
3-D Scan the ray
projected in a
ray. (X,Y,Z predetermined
)
calibrated
(i,j path.
)
(u,v
)

A light
Camer
a
Yi-Chih Hsieh, Decoding structured light patterns for three-dimensional imaging systems, Pattern Recognition 34 (2001) 343-349 .
 Structured Light Sensors
• These sensors project a structured pattern (such as
grids or stripes) of light onto an object and use the
deformation of the pattern on the object's surface to
calculate distance.
• They are commonly used in 3D scanning and industrial
metrology.
 Structured
Light
Encoding the Project a strip
3-D position of or pattern.
projected ray. Get 3-D
(X,Y,Z
)
positions of all
the scene
(i,j
)
points lying on
(u,v
)
the projected
strip.

Structur
Camer ed light
a
Yi-Chih Hsieh, Decoding structured light patterns for three-dimensional imaging systems, Pattern Recognition 34 (2001) 343-349 .
 Range Data Segmentation
• Range data segmentation is the task of segmenting (dividing) a range
image, an image containing depth information for each pixel, into
segments (regions), so that all the points of the same surface belong
to the same region, there is no overlap between different regions and
the union of these regions generates the entire image.
• The goal of range data segmentation is to identify and
isolate objects, obstacles, or distinct regions within the
sensor's field of view for further analysis and decision-
making.
There have been two main approaches to the range
segmentation problem: region-based range segmentation and
edge-based range segmentation.
1. Region-based range segmentation
• Region-based range segmentation algorithms can be further categorized into two major groups:
parametric model-based range segmentation algorithms and region-growing algorithms.

• Algorithms of the first group are based on assuming a parametric surface model and grouping data
points so that all of them can be considered as points of a surface from the assumed parametric
model (an instance of that model).

• Region-growing algorithms start by segmenting an image into initial regions. These regions are
then merged or extended by employing a region growing strategy. The initial regions can be
obtained using different methods, including iterative or random methods. A drawback of
algorithms of this group is that in general they produce distorted boundaries because the
segmentation usually is carried out at region level instead of pixel level.
Edge-based range segmentation
• Edge-based range segmentation algorithms are based on edge
detection and labeling edges using the jump boundaries
(discontinuities).
• They apply an edge detector to extract edges from a range image.
Once boundaries are extracted, edges with common properties are
clustered together.
• The segmentation procedure starts by detecting discontinuities using
zero-crossing and curvature values.
• The image is segmented at discontinuities to obtain an initial
segmentation.
• At the next step, the initial segmentation is refined by fitting quadratics
whose coefficients are calculated based on the Least squares method.
• In general, a drawback of edge-based range segmentation algorithms
is that although they produce clean and well defined boundaries
between different regions, they tend to produce gaps between
boundaries.
• In addition, for curved surfaces, discontinuities are smooth and hard to
locate and therefore these algorithms tend to under-segment the
range image. Although the range image segmentation problem has
been studied for a number of years, the task of segmenting range
images of curved surfaces is yet to be satisfactorily resolved.
 Range Image Registration
Range image registration is the process of aligning and combining multiple range images,
which may be captured from different viewpoints or at different times, to create a single,
coherent 3D model. The key steps involved in range image registration include:
1. Feature Extraction: Extract features or keypoints from the range images. These
features serve as distinctive points that can be matched across different images.
2. Feature Matching: Match corresponding features in pairs of range images to establish
their relative pose (position and orientation) with respect to each other.
3. Pose Estimation: Calculate the transformation (usually translation and rotation) that
aligns the range images accurately. Common algorithms for this purpose include
Iterative Closest Point (ICP) and its variants.
4. Global Registration: Combine the relative transformations to register all range images
into a common global coordinate system.
5. Refinement: Fine-tune the alignment to minimize any residual errors or
misalignments.
6. Data Fusion: Merge the registered range images into a single, unified 3D model.
 Model Acquisition:
Model acquisition is the process of creating a 3D model from the registered range images. This representation
can take the form of a point cloud, a mesh, or other 3D data structures. The typical steps in model acquisition
include:
• Point Cloud Generation: Convert the registered range images into a 3D point cloud, where each point
corresponds to a 3D coordinate in space and includes depth information from the range data.
• Mesh Generation: In some applications, a 3D mesh is generated from the point cloud, often consisting of
interconnected triangles. This mesh can provide a more detailed and structured representation of the 3D
model.
• Texture Mapping: Apply color or texture information to the 3D model, typically using images or textures
captured in conjunction with the range data.
• Mesh Simplification: For real-time rendering and storage efficiency, reduce the complexity of the 3D
model by simplifying the mesh while preserving important geometric features.
• Texture Projection: Project textures from the original images onto the 3D model to enhance its appearance
and realism.
• Post-Processing: Perform various data cleaning, noise reduction, hole filling, and optimization tasks to
ensure that the 3D model is suitable for its intended application.
 Object Recognition
• Object recognition is a computer vision technique for
identifying objects in images or videos. Object
recognition is a key output of deep learning and
machine learning algorithms.
• When humans look at a photograph or watch a video,
we can readily spot people, objects, scenes, and visual
details.
• The goal is to teach a computer to do what comes
naturally to humans: to gain a level of understanding of
what an image contains.
Object recognition is a key technology behind driverless cars, enabling them to recognize a stop sign or
to distinguish a pedestrian from a lamppost. It is also useful in a variety of applications such as disease
identification in bioimaging, industrial inspection, and robotic vision.
 Object recognition typically consists of several key steps:
1.Object Detection:
• Localization: Determine the location and extent of objects within an image. This
is often done by drawing bounding boxes around the objects.
• Class Labeling: Assign a label or category to each detected object (e.g., "car,"
"person," "dog").
2. Feature Extraction: Extract distinctive features from the detected objects.
Common features include color, texture, shape, and key points.
3. Feature Representation: Transform the extracted features into a suitable
format for further analysis. This often involves creating feature vectors or
descriptors that capture the essential characteristics of the object.
4. Classification: Use machine learning algorithms, such as deep neural networks
(e.g., CNNs), support vector machines (SVMs), or decision trees, to classify objects
based on their feature representations.
5. Recognition and Decision-Making: Determine the identity of recognized
objects based on the classification results. This may involve associating objects
with known object categories or labels.
6. Post-Processing: Enhance the recognition results by applying additional
techniques, such as non-maximum suppression to remove redundant bounding
boxes or smoothing techniques to improve object tracking.
 How Object Recognition Works
• You can use a variety of approaches for object
recognition. Recently, techniques in machine learning
and deep learning have become popular approaches to
object recognition problems. Both techniques learn to
identify objects in images, but they differ in their
execution.
 Object Recognition Using Deep
Learning
• Deep learning techniques have become a popular
method for doing object recognition.
Deep learning models such as convolutional neural
networks, or CNNs, are used to automatically learn an
object’s inherent features in order to identify that
object.
• For example, a CNN can learn to identify differences
between cats and dogs by analyzing thousands of
training images and learning the features that make
cats and dogs different.
There are two approaches to performing object recognition using deep
learning:
• Training a model from scratch: To train a deep network from scratch, you gather a very large
labeled dataset and design a network architecture that will learn the features and build the model.
The results can be impressive, but this approach requires a large amount of training data, and you
need to set up the layers and weights in the CNN.
• Using a pretrained deep learning model: Most deep learning applications use the transfer learning
approach, a process that involves fine-tuning a pretrained model. You start with an existing
network, such as AlexNet or GoogLeNet, and feed in new data containing previously unknown
classes. This method is less time-consuming and can provide a faster outcome because the model
has already been trained on thousands or millions of images.
Deep learning offers a high level of accuracy but requires a large amount of data to make accurate
predictions.
 Machine learning techniques for
object recognition
• Machine learning techniques are also popular for object
recognition and offer different approaches than deep
learning. Common examples of machine learning
techniques are:
• HOG feature extraction with an SVM machine learning
model
• Bag-of-words models with features such as SURF and
MSER (Maximally Stable External Regions)
• The Viola-Jones algorithm, which can be used to
recognize a variety of objects, including faces and upper
bodies
Machine Learning vs. Deep Learning
for Object Recognition
• The main consideration to keep in mind when choosing
between machine learning and deep learning is whether
you have a powerful GPU and lots of labeled training
images.
• If the answer to either of these questions is No, a
machine learning approach might be the best choice.
• Deep learning techniques tend to work better with more
images, and a GPU helps to decrease the time needed
to train the model.
Thank you!