CS 678

Computer Vision

Instructor: Guodong Guo

Guodong.Guo@mail.wvu.edu
 Welcome!
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
 Journals
 Conferences
 Instructor
 Guodong Guo
 http://cs.wisc.edu/~gdguo
 Major Research Interest
 Computer Vision, Machine Learning,
Pattern Recognition, Biometrics,
Multimedia, and HCI
 About You …
What do you know already?
 C/C++ (Visual C++)
 Matlab

 Images

 OpenCV

http://sourceforge.net/projects/opencvlibrary/
Install OpenCV in your PC or laptop,
Read the manual introduction
Try to load and save images (homework #0)
 Outline
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
 Meeting Times
 Lectures
 W 5:00-7:20 pm
 Room: ESB 215

 Office hours
 Thursday 4:00-6:00 pm (AER 345, or
AER 321)
 Or by appointment
 Grading
 The final grade depends on:
 Homework and programming assignments:
40%
 Exam: 40%

 Final project (may include class presentation):

20%
 Presentation (not determined)

 Class participation: (-5%, if absent = 3 times or

more)
 Extra: 1~10% (for creative ideas, paper

submission based on this course, etc.)

 Textbook
 Computer Vision:
Algorithms and
Applications,
Richard Szeliski,
http://szeliski.org/B
ook/
 Optional Textbook
 Computer Vision: A
Modern Approach,
2th Edition, by
David Forsyth and
Jean Ponce,
Prentice Hall, 2003
 Optional Textbook
 Introductory
Techniques for 3-D
Computer Vision,
E. Trucco and A.
Verri, Prentice
Hall, 1998. ISBN 0-
13-261108-2
 Look at the Syllabus
 Course Objectives
 Expected learning outcomes
 Detailed list of topics (maybe
updated)
 Outline
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
 What is Computer
Vision?
 Given an image or more, extract
properties of the 3D world
•Traffic scene
• Number of vehicles
• Type of vehicles
• Location of closest obstacle
• Assessment of congestion
• Location of the scene
captured
•…
 Computer Vision vs.
Graphics
 3D2D implies information loss
graphics

vision

 sensitivity to errors
 need for models
 Computer Vision vs.
Machine Learning
 Machine learning is a scientific
discipline that is concerned with the
design and development of algorithms
that allow computers to change behavior
based on data, such as from sensor data
or databases (from Wikipedia)
 A major focus of machine learning
research is to automatically learn to
recognize complex patterns and make
intelligent decisions based on data.
 Computer Vision vs.
Machine Learning
 Machine Learning is very useful for
Computer Vision (e.g., learning for vision)
 Computer Vision is more than just learning
 Modeling
 Example based learning

 In Machine Learning, it usually does not

care about how to obtain the data or sensors
 In Computer Vision, we care how to obtain
the visual data (sensor design, active vision),
how to represent the visual data, and others
 Vision
 Vision is the process of discovering
what is present in the world and
where it is by looking.
 Computer Vision
 Computer Vision is the study of
analysis of pictures and videos in
order to achieve results similar to
those as by people.
 Why Computer Vision
 An image is worth 1000 words
 Many biological systems rely on
vision
 The world is 3D and dynamic
 Cameras and computers are cheap
 …
 Computer Vision
Examples

Finding People in images

Problem 1: Given an image I
Question: Does image I contain an
image of a person?
“Yes” Instances
“No” Instances
Some Computer Vision
Topics
Imaging Geometry
Camera Modeling

 Pinhole Cameras
 Lenses
 Camera
Parameters and
Calibration
Image Filtering and
Enhancing
 Linear Filters
and Convolution
 Image
Smoothing
 Edge Detection
 Pyramids
Image Filtering and
Enhancing (cont.)
Region Segmentation
Color
Texture
Image Restoration

Original Synthetic
Perceptual Organization
Perceptual Organization
Shape Analysis
Stereo
Motion and Optical Flow
High Level Vision
Image Mosaic
 One Very Successful
Example
 Face detection in a digital camera
 The camera detects faces in a scene
and then automatically focuses (AF) and
optimizes exposure (AE) and, if needed,
flash output.
 Outline
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
 Applications
 autonomous cars, planes, missiles, robots,
...
 space exploration
 aid to the blind, ASL recognition
 manufacturing, quality control
 surveillance, security, biometrics
 image retrieval
 medical imaging and analysis
 ...
Current State of the Art
Earth viewers (3D modeling)

Image from Microsoft’s Virtual Earth

(see also: Google Earth)
 Optical character
recognition
Technology (OCR)
to convert scanned docs to text
• If you have a scanner, it probably came with OCR software

Digit recognition, AT&T labs License plate readers

http://www.research.att.com/~yann/ http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
 Face detection

 Many new digital cameras now detect

faces
 Canon, Sony, Fuji, …
 Smile detection?

Sony Cyber-shot® T70 Digital Still Camera

Object recognition (in
supermarkets)

LaneHawk by EvolutionRobotics
“A smart camera is flush-mounted in the checkout lane,
continuously watching for items. When an item is detected and
recognized, the cashier verifies the quantity of items that were
found under the basket, and continues to close the transaction.
The item can remain under the basket, and with
LaneHawk,you are assured to get paid for it… “
Face recognition

Who is she?
Vision-based biometrics

“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
 Login without a
password…

Face recognition systems now

Fingerprint scanners on
beginning to appear more widely
many new laptops, http://www.sensiblevision.com/
other devices
Object recognition (in mobile
phones)

 This is becoming real:

 Microsoft Research
 Point & Find, Nokia
Special effects: shape
capture

The Matrix movies, ESC Entertainment, XYZRGB, NRC

Special effects: motion
capture

Pirates of the Carribean, Industrial Light and Magic

Click here for interactive demo
 Slide content courtesy of Amnon Shashua
Smart cars

 Mobileye
 Vision systems currently in high-end BMW, GM, Volvo
models
 By 2010: 70% of car manufacturers.
 Vision-based interaction (and
games)

Digimask: put your face on a 3D avatar.

as camera-based IR
. See Lee’s work atCMU on clever tricks on using it to
ouch display!

“Game turns moviegoers into Human Joysticks”, CNET

Camera tracking a crowd, based on this work.
 Vision in space

NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.

Vision systems (JPL) used for several tasks

• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
 Robotics

NASA’s Mars Spirit Rover http://www.robocup.org/

http://en.wikipedia.org/wiki/Spirit_rover
 Medical imaging

Image guided surgery

3D imaging
Grimson et al., MIT
MRI, CT
Current state of the art
 You just saw examples of current systems.

 This is a very active research area, and rapidly

changing
 Many new apps in the next 5 years

 To learn more about vision applications and

companies
 David Lowe maintains an excellent
overview of vision companies
 http://www.cs.ubc.ca/spider/lowe/
vision.html
 Outline
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
Computer Vision focuses
on:
 What information should be
extracted?
 How can it be extracted?
 How should it be represented?
 How can it be used to achieve the
goal?
 Related disciplines
 Image processing
 Pattern recognition
 Photogrammetry
 Computer graphics
 Artificial intelligence
 Machine learning
 Projective geometry
 Control theory
 Active Research Topics
 Object recognition
 Human behavior analysis
 Internet and computer vision
 Biometrics and soft biometrics
 Large scale 3D reconstruction (city level)
 Medical image processing
 Vision for robotics
 …
 Outline
 Introductions
 Administrative Matters
 Course Outline
 Applications of Computer Vision
 Computer Vision Focus
 Computer Vision Publications
 Computer Vision
Publications
 Journals
 IEEE Trans. on Pattern Analysis and Machine
Intelligence (TPAMI)
 #1 IEEE, Thompson-ISI impact factor: 5.96
 #1 in both electrical engineering and artificial

intelligence
 #3 in all of computer science

 Internal Journal of Computer Vision (IJCV)

 ISI impact factor: 5.358, Rank 2 of 94 in “CS, artificial
intelligence
 IEEE Trans. on Image Processing
 …
 Importance of CV
 From these major journal rankings,
we can see the importance of
Computer Vision research in the
whole areas of
 Computer Science
 Electrical Engineering
 Computer Vision
Publications
 Conferences
 International Conference on Computer
Vision (ICCV), once every two years
 Conf. of Computer Vision and Pattern

Recognition (CVPR), once a year

 Europe Conference on Computer Vision

(ECCV), once every two years

…