Introduction To Augmented Reality

William El Kaim
Oct. 2016 - V 2.0
 This Presentation is part of the
Enterprise Architecture Digital Codex

Copyright © William El Kaim 2016 http://www.eacodex.com/ 2

 Plan

Augmented Reality Definition

• Augmented Reality Market
• Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
• Augmented Reality Technologies and Platforms
• Feeling and Sensing
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 3

 What is Augmented Reality?
• Augmented reality (AR) is a field of computer research which deals with the
combination of real-world and computer-generated data.

Copyright © William El Kaim 2016 4

 AR systems have the following three characteristics

Combines real and virtual

objects in a real environment

Is interactive in real-time

Registers (aligns) real and

virtual objects with each other.

Copyright © William El Kaim 2016 5

 All About Augmentation

• Augmented reality (AR) is a live direct or indirect view of a physical, real-

world environment whose elements are augmented (or supplemented) by
computer-generated sensory input such as sound, video, graphics
or GPS data.
• As a result, the technology functions by enhancing one’s current perception of reality.
• Augmentation is conventionally in real-time and in semantic context with
environmental elements
• Ex: sports scores on TV during a match.

Copyright © William El Kaim 2016 6

 Plan

• Augmented Reality Definition

Augmented Reality Market
• Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
• Augmented Reality Technologies and Platforms
• Feeling and Sensing
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 7

 A Growing Market

Total 2018 annual software revenue is forecasted at $2.8bn

from 47.6m active users. Cumulative software revenue from
2014 to 2018 is estimated at $7.7bn.
Copyright © William El Kaim 2016 Source: Kzero 8
 A Growing Market

Copyright © William El Kaim 2016 9

 Plan

• Augmented Reality Definition

• Augmented Reality Market
Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
• Augmented Reality Technologies and Platforms
• Feeling and Sensing
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 10

 Milgram’s reality-virtuality continuum

• In 1994 Paul Milgram and Fumio Kishino defined a mixed reality as

"anywhere between the extrema of the virtuality continuum”, where the
Virtuality Continuum extends from the completely real through to the
completely virtual environment with augmented reality and augmented
virtuality ranging between.

Copyright © William El Kaim 2016 11

 Milgram’s reality-virtuality continuum

Copyright © William El Kaim 2016 12

Copyright © William El Kaim 2016 13
Copyright © William El Kaim 2016 14
 Milgram’s reality-virtuality continuum

Augmented Reality is the art of superimposing computer

generated content over a live view of the world.
Augmented Augmented
Reality Virtuality

Real Virtual
Environment Reality-Virtuality Continuum Environment

Augmented reality is the integration of digital information

with live video or the user's environment in real time.
Copyright © William El Kaim 2016 15
 Augmented Reality v.s. Virtual Reality

VR technologies completely
In contrast, AR allows the user
immerse a user inside a
to see the real world, with
synthetic environment. While
virtual objects superimposed
immersed, the user cannot
upon or composited with the
see the real world around
real world.
him.

Copyright © William El Kaim 2016 16

 Augmented Reality Example

Copyright © William El Kaim 2016 17

 Virtual Reality Example

Copyright © William El Kaim 2016 http://daqri.com/ 18

 Plan

• Augmented Reality Definition

• Augmented Reality Market
• Augmented Reality vs. Virtual Reality
Augmented Reality Devices
• Augmented Reality Technologies and Platforms
• Feeling and Sensing
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 19

 Augmented Reality Interactions

• With the help of advanced AR technology (e.g. adding computer

vision and object recognition) the information about the surrounding real
world of the user becomes interactive and digitally manipulable.
• Artificial information about the environment and its objects can be overlaid on
the real world
• In order to benefit from it you need a dedicated device
• Laptop or computer
• Handheld like Smartphone or Tablets
• Head-Mounted Device (HMDs) often employ sensors for six degrees of freedom
monitoring that allow the system to align virtual information to the physical world and
adjust accordingly with the user's head movements
• Smart Glasses
• Gesture based solution (Kinect, Leapmotion)

Copyright © William El Kaim 2016 20

 Head-Mounted Display (HMDs)
• A head-mounted display or helmet mounted display, both abbreviated HMD,
is a display device, worn on the head or as part of a helmet, that has a small
display optic in front of one (monocular HMD) or each eye (binocular HMD).
• Basic one
• Google CardBoard and VRAse use your smartphone
• HMD available today
• Carl Zeiss VR One Plus
• HTC Vive
• Microsoft HoloLens
• OculusVR bought by facebook
• Samsung GearVR
• Sensics invented the Smart Goggles
• Sony PlayStation VR
• Sulon Technologies

Copyright © William El Kaim 2016 21

 Handheld: Google CardBoard

Copyright © William El Kaim 2016 Google CardBoard 22

 Head-Mounted Device (HMDs)
vrAse

Copyright © William El Kaim 2016 http://www.vrase.com/ 23

 Head-Mounted Device (HMDs)
META.01

Video

Copyright © William El Kaim 2016 https://www.spaceglasses.com/ 24

 Head-Mounted Device (HMDs)
Oculus

Copyright © William El Kaim 2016 https://www.oculus.com/ 25

 HMD: OSVR HDK
• The OSVR HDK presents
a more modular approach
than any other headset
currently known about.
• While its resolution is just
about identical to Oculus’
DK2 devkit, its open-
source nature means
it’s 3D-printable and
allows for nearly
unfettered modification
and addons.

Copyright © William El Kaim 2016 26

 HMDs Compared

Copyright © William El Kaim 2016 27

 HMDs Compared

Copyright © William El Kaim 2016 28

Copyright © William El Kaim 2016 29
 Smart Glasses

• Wearable computing device in the form of computerized eyeglasses.

• Also called optical head-mounted display (OHMD), which is a wearable
display that has the capability of reflecting projected images as well as
allowing the user to see through it.
• Early smart glasses had limited functionality and typically were simply miniature video
monitors that were plugged into a video source.
• Modern devices typically possess enhanced data processing functionality similar to a
smartphone or tablet, are able to run mobile apps and include specialty features such
augmented reality overlays and GPS and mapping capability.

Copyright © William El Kaim 2016 30

 Smart Glasses

• APX labs
• Arglass by Optinvent
• Epson Moverio BT-200
• EyeBorg Eyecam 3.0
• Google Glass
• Laster Wave
• Lumus
• Meta AR Eyeglass Hologram Computer
• Sony HMZ-T1 Personal 3D Viewer
• Penny Interactive Glasses
• Vuzix comes the M100 smart glasses

Copyright © William El Kaim 2016 31

 Smart Glasses: Google Glass

Copyright © William El Kaim 2016 https://developers.google.com/glass/distribute/glass-at-work 32

 Smart Glasses: APX Lab

http://www.apx-labs.com/skylight/ 33
Copyright © William El Kaim 2016
 Smart Glasses: Optinvent

Copyright © William El Kaim 2016 http://optinvent.com/ 34

 Gesture Based Solution

2.5cm 60cm 2m 4m

Copyright © William El Kaim 2016

http://kinectgeniusbar.com/2013/02/26/complementarite-kinect-et-leap-motion/ 35
 Gesture Based Solution

Copyright © William El Kaim 2016 36

 Gesture Based Solution
Amadeus Travel Seeker

Copyright © William El Kaim 2016 37

 Mash Up VR and Hand-Tracking

• Leap Motion and the OSVR open-source ecosystem announced a

partnership that would allow Razer’s OSVR HDK (hacker development kit) to
ship later this year with a Leap Motion sensor embedded in its faceplate.
• Leap Motion shipped a $20 mount that could attach its sensor to the front of
an Oculus Rift devkit.

Copyright © William El Kaim 2016 38

 Plan

• Augmented Reality Definition

• Augmented Reality Market
• Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
Augmented Reality Technologies and Platforms
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 39

 Key Tracking Technologies

Marker Tracking Marker Less

Tracking 3D Object Tracking Face Tracking

Motion detection &

Tracking Environment
Tracking

Copyright © William El Kaim 2016 40

 Marker Based (“Black Square”)

• One method for an AR system to recognize elements of the physical world is

through the use of so-called ‘markers’ that, when processed through the
camera, alert the system that relevant information for overlay can be
downloaded from the Internet.
• A marker, generally a black square is used by the app on the mobile as a
placeholder for “augmented information” to be displayed.
• As is the case with many encoded markers or tags, QR (Quick Response)
codes were originally conceived for use in the industrial sector to rapidly
scan and track products along the supply chain.
• With the advent of the Internet-enabled smartphone and accompanying
camera, apps have been developed to allow the wider public to scan these
codes and access further information.

Copyright © William El Kaim 2016 41

 Marker Based (“Black Square”)
• Marker-based AR uses a camera
and a visual marker to determine
the center, orientation, and range
of its spherical coordinate system.
• Most of the time, the applications
using markers are based on open
source technology.
• QR codes have found common
use in AR as markers, particularly
due to the fact that they are well
suited for use where mobile
devices are concerned.

Copyright © William El Kaim 2016 42

 Marker Based (“Black Square”)
Marker

Copyright © William El Kaim 2016 43

 Marker Based (“Black Square”)

Copyright © William El Kaim 2016 44

 Ex: Augmented Reality With Markers

Copyright © William El Kaim 2016 45

 Ex: Augmented Reality With Markers

Copyright © William El Kaim 2016 46

 Ex: Augmented Reality With Markers

Copyright © William El Kaim 2016 Source: IKEA 47

 Markerless Tracking (Images)

• Where the intention to present AR is in a particularly dynamic environment,

such as while on the move, outside and so on, it is infeasible that markers
will be positioned on the majority of surrounding objects.
• Consequently, in the majority of envisioned use cases, where markers are
unavailable, an image recognition system is required.
• Owing to the fact that real-world entities, such as humans, vehicles and
buildings present far greater complexity to the system than a simple 2D
marker, image recognition software requires high processing power.
• Passive: the user simply views AR through a suitable display, where the camera is
orientated away from the user (view-only mode, where the use does not interact directly
with the AR presentation).
• Active: the user is tracked by a camera and analyses of these movements are translated
to the image display.

Copyright © William El Kaim 2016 48

 Markerless Tracking (Images)
• Active tracking and
recognition of real
environment on any type
of support without using
special placed markers.
• Example of recognition of
Lego box.

Copyright © William El Kaim 2016 49

 Location Based Tracking

• Refers to tracking based on geo-location information obtained from the

device’s location sensors (longitude, latitude, altitude, compass bearing,
accelerometer readings for pitch and roll).
• This term is used to make a distinction between systems that rely on location sensors
alone in contrast to systems that can track objects using optical (image recognition)
techniques.
• Location based tracking is generally less accurate than optical methods and only works
in outdoor environments.

Copyright © William El Kaim 2016 50

 Google Glass Travel Guide

Source: PSFK
Copyright © William El Kaim 2016 51
 Simultaneous localization and mapping (SLAM)

• Technique used by robots and autonomous vehicles to build up a map within

an unknown environment (without a priori knowledge), or to update a map
within a known environment (with a priori knowledge from a given map),
while at the same time keeping track of their current location.
• Enable a 360 degrees augmented view (see video)

Copyright © William El Kaim 2016 52

 SLAM

Copyright © William El Kaim 2016 http://developer.pointcloud.io/sdk 53

 AR to Learn and Train

Copyright © William El Kaim 2016 Source: Organic Motion 54

 Holography
• Holography is a technique that
enables a light field, which is
generally the product of a light source
scattered off objects, to be recorded
and later reconstructed when the
original light field is no longer present,
due to the absence of the original
objects.

Copyright © William El Kaim 2016 Source: Wikipedia 55

 Holography vs. Photography
• A hologram represents a recording of information regarding the light that
came from the original scene as scattered in a range of directions rather than
from only one direction, as in a photograph.
• This allows the scene to be viewed from a range of different angles, as if it were still
present.
• A photograph can be recorded using normal light sources (sunlight or electric
lighting) whereas a laser is required to record a hologram.
• A lens is required in photography to record the image, whereas in
holography, the light from the object is scattered directly onto the recording
medium.
• A holographic recording requires a second light beam (the reference beam)
to be directed onto the recording medium.
• A photograph can be viewed in a wide range of lighting conditions, whereas
holograms can only be viewed with very specific forms of illumination.
Copyright © William El Kaim 2016 Source: Wikipedia 56
 Holography vs. Photography

• A photograph is a two-dimensional representation that can only reproduce a

rudimentary three-dimensional effect, whereas the reproduced viewing range
of a hologram adds many more depth perception cues that were present in
the original scene. These cues are recognized by the human brain and
translated into the same perception of a three-dimensional image as when
the original scene might have been viewed.
• A photograph clearly maps out the light field of the original scene. The
developed hologram's surface consists of a very fine, seemingly random
pattern, which appears to bear no relationship to the scene it recorded.

Copyright © William El Kaim 2016 57

 Hologram

https://youtu.be/GtbBpjPPR7Y http://www.holusion.com/

Copyright © William El Kaim 2016 58

 Mini Augmented Reality

Mini Augmented Vision 59

Copyright © William El Kaim 2016
 Sony …

Copyright © William El Kaim 2016 60

 Google Cardboard

Copyright © William El Kaim 2016

https://www.google.com/get/cardboard/ 61
 Location Based Tracking

• Refers to tracking based on geo-location information obtained from the

device’s location sensors (longitude, latitude, altitude, compass bearing,
accelerometer readings for pitch and roll).
• This term is used to make a distinction between systems that rely on location sensors
alone in contrast to systems that can track objects using optical (image recognition)
techniques.
• Location based tracking is generally less accurate than optical methods and only works
in outdoor environments.

Copyright © William El Kaim 2016 62

 Simultaneous Localization And Mapping
• Simultaneous localization and mapping (SLAM)
• Technique used by robots and autonomous vehicles to build up a map within an
unknown environment (without a priori knowledge), or to update a map within a known
environment (with a priori knowledge from a given map), while at the same time keeping
track of their current location.
• Enable a 360 degrees augmented view (see video)

Copyright © William El Kaim 2016 63

 Augmented Reality Browsers

• The use of the term “AR browser” suggests these products are comparable
to generic web browsers such as Internet Explorer.
• But while both are technically HTTP user agents and consume content from
the web, the comparison ends there.
• Generic web browsers all adhere more or less strictly to a set of standards
for mark-up and therefore different browsers can consume and render the
same web content in a consistent manner.
• In contrast standards for augmented reality applications are still in their
infancy and there is no interoperability between AR “browsers”.
• This means that content cannot be shared between different AR browsers

Copyright © William El Kaim 2016 64

 Augmented Reality Browsers
Augmented Reality Markup Language (ARML)

• Augmented Reality Markup Language (ARML) is a data standard to describe

and interact with augmented reality (AR) scenes.
• It is developed within the Open Geospatial Consortium (OGC) by a dedicated ARML 2.0
Standards Working Group.
• ARML consists of both an XML grammar to describe the location and
appearance of virtual objects in the scene, as well as ECMAScript bindings
to allow dynamic access to the properties of the virtual objects, as well as
event handling

Copyright © William El Kaim 2016 65

 Augmented Reality Browsers
X3D Standard

• X3D is a royalty-free open standards file format and run-time architecture to

represent and communicate 3D scenes and objects using XML.
• It is an ISO ratified standard that provides a system for the storage, retrieval
and playback of real time graphics content embedded in applications, all
within an open architecture to support a wide array of domains and user
scenarios.
• The development of real-time communication of 3D data across all
applications and network applications has evolved from its beginnings as the
Virtual Reality Modeling Language (VRML) to the considerably more mature
and refined X3D standard.

Copyright © William El Kaim 2016 66

 Augmented Reality Browsers
Markup Language Comparison

Copyright © William El Kaim 2016 67

 Augmented Reality Browsers

• Argon, Arlab, Aurasma, Layar, Mixare (open source), Wikitude

• Two majors ones are:
• Wikitude uses location-based content from social media and groups data based on
certain business niches and verticals. Wikitude allows for scanning print journals,
magazines, brochures, ads and labels for 3D entries.
• Wikitude SDK is compatible with most of development platforms and frameworks including
iOS, Android, tablet, smart glasses, PhoneGap, Cordova, Tamarin, Titanium, Unity 3D and
others.
• Layar, unlike Wikitude, aims to revive and augment static content pages (e.g., journals,
magazines, booklets).
• Besides offering the Layar browser app and building bespoke AR solutions for clients, Layar
offers its own SDK and value-added services to application developers. Using them,
developers can create AR layers and integrate them in their app with PhoneGap framework
plugin. Layar SDK isn't free of charge and publishing through it requires purchasing additional
page credits.

Copyright © William El Kaim 2016 68

 Augmented Reality Platforms

• Augment
• Eon Reality
• Hoppala: world’s largest content platform for geo based AR
• Jaunt
• OnVert
• OpenHybrid
• String
• Total Immersion
• Vuforia

Copyright © William El Kaim 2016 69

 Layar

Copyright © William El Kaim 2016 70

 Wikitude

• Wikitude world Browser is widely regarded as the king of all augmented

reality browsers
• While using your smartphone’s camera in a given area, the virtual browser (along with
more than 3,500 associated content providers) offers you just about any geographically-
relevant information you may find valuable in your travels.
• Moreover, the app allows users to find hotels and similar accommodations
through Yelp, TripAdvisor, and the like, while offering mobile deals and
coupons for local stores in the vicinity.

Copyright © William El Kaim 2016 71

 Bought by Apple
Junaio

Copyright © William El Kaim 2016 72

 Augmented Reality JS API

• awe.js
• Three.js

Copyright © William El Kaim 2016 73

 Augmented Reality Providers

• ARToolworks: camera-based augmented reality, first showing its core

product - ARToolKit
• Atheer Labs: Gesture-based 3D Augmented Interactive RealityPlatform for
the enterprise
• Blippar: AR marker based platform
• Catchoom: Image Recognition and Augmented Reality tools.
• Daqri: Editor of Daqri 4D studio
• InGlobe Technologies:development of Augmented Reality solutions and
Perceptual Computing systems.
• Innovega: eyewear-based platforms based on the patented iOptikTM display
system

Copyright © William El Kaim 2016 74

 Technologies: Virtual Reality Engine

• Virtual Universe
• ExitReality
• LindenLab
• VrSIM
• unity3D

Copyright © William El Kaim 2016 75

Copyright © William El Kaim 2016 76
 Development Companies

• Glorar
• Specialized in 3D app on mobile, works with AR23D design company if needed
• Immersion
• Company in France specialised in AR
• Total Immersion
• AR for Brands
• Kudan
• Augmented reality provider
• Marxent Labs
• Mainly developing on Vuforia
• Merchlar
• Design Agency (AR)

Copyright © William El Kaim 2016 77

 Plan

• Augmented Reality Definition

• Augmented Reality Market
• Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
• Augmented Reality Technologies and Platforms
Feeling and Sensing
• Augmented Reality Applications Examples

Copyright © William El Kaim 2016 78

 Virtual Reality You Can Feel

Copyright © William El Kaim 2016 http://axonvr.com/ 79

Copyright © William El Kaim 2016 80
Copyright © William El Kaim 2016 81
 Virtual Reality You Can Feel

Copyright © William El Kaim 2016 http://www.priovr.com/ 82

 Disney’s Crazy Invention Lets You Feel Phantom
Objects Floating In Air
• Aireal is the result of research by University of Illinois PhD student Rajinder
Sodhi and Disney Reseach’s Ivan Poupyrev.
• When set by your television or connected to an iPad, this diminutive machine will puff air
rings that allow you to actually feel objects and textures in midair
• no special controllers or gloves required.
• Video here

83
 Disney’s Crazy Invention Lets You Feel Phantom
Objects Floating In Air
• Feel a butterfly projected on your hand.

84
 Disney’s Crazy Invention Lets You Feel Phantom
Objects Floating In Air
• …a flock of birds flying all around you

85
Smelling, odors piloted by your phone

86
 Plan

• Augmented Reality Definition

• Augmented Reality Market
• Augmented Reality vs. Virtual Reality
• Augmented Reality Devices
• Augmented Reality Technologies and Platforms
• Feeling and Sensing
Augmented Reality Applications Examples

Copyright © William El Kaim 2016 87

 Pokemon GO

Copyright © William El Kaim 2016 IOS and Android 88

 Ink Hunter

Copyright © William El Kaim 2016 IOS 89

 Star Chart

Copyright © William El Kaim 2016 IOS and Android 90

 Ingres

Copyright © William El Kaim 2016 IOS and Android 91

 IKEA Catalogue

Copyright © William El Kaim 2016 IOS and Android 92

 EA Digital Codex Twitter
http://www.eacodex.com/ http://www.twitter.com/welkaim
Linkedin SlideShare
http://fr.linkedin.com/in/williamelkaim http://www.slideshare.net/welkaim

Claudine O'Sullivan
Copyright © William El Kaim 2016 93