0% found this document useful (0 votes)
88 views27 pages

E57 General Data Management

The document describes the ASTM E57 file format, an open standard developed over 5 years for exchanging 3D imaging system data. The format was created to address problems with proprietary, ad-hoc, and domain specific formats by providing a single, common format to reduce data conversion and ensure long-term stability and interoperability. The E57 format stores 3D point clouds, images, and metadata in an extensible, self-describing format to enable widespread use across domains.

Uploaded by

Azrin Rahman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
88 views27 pages

E57 General Data Management

The document describes the ASTM E57 file format, an open standard developed over 5 years for exchanging 3D imaging system data. The format was created to address problems with proprietary, ad-hoc, and domain specific formats by providing a single, common format to reduce data conversion and ensure long-term stability and interoperability. The E57 format stores 3D point clouds, images, and metadata in an extensible, self-describing format to enable widespread use across domains.

Uploaded by

Azrin Rahman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 27

The ASTM E57 File Format for

3D Imaging Data Exchange

Image courtesy Intelisum, Inc.

Daniel Huber
The Robotics Institute
Carnegie Mellon University
Member of the ASTM E57 Committee on 3D Imaging Systems
Sub-committee on Data Interoperability
The E57 Format
What have we done?
Over the past 5 years, we have
developed an open standard for 3D
imaging system data exchange.
n  Store and exchange:
n  3D data (point clouds,
range images)
n  Associated images
n  Meta-data to support
downstream processing
n  General purpose – terrestrial,
aerial, mobile
n  Extendable
Who is “we”?
ASTM E57 Committee on 3D
Imaging Systems, Sub-committee on
Data Interoperability (E57.04)
2
What is Three Dimensional Imaging?

Laser Scanners

Triangulation

3
Uses of 3D Imaging: Modeling and Visualization

4
Uses of 3D Imaging: Robotics

5
Uses of 3D Imaging: Civil Engineering
Quality assurance

Infrastructure inspection

Reverse engineering

6
How Do People Store 3D Data Today?

Proprietary Ad-Hoc Domain Specific


Formats Formats Formats
n  PTS n  PLY n  LAS
n  PTX n  homebrew
n  DXF

7
What’s the Problem with the Status Quo?

Proprietary Ad-Hoc Domain Specific


Formats Formats Formats
n  Data exchange n  Storage efficiency n  Limited applicability
combinatorial n  Limited across domains
explosion documentation
n  Loss of information n  Variations in
when converting implementations
n  Long-term stability n  Limited use for data
exchange

8
How Does the E57 Format
Address These Problems?
Proprietary Ad-Hoc Domain Specific
Formats Formats Formats
n  Data exchange n  Storage efficiency n  Limited applicability
combinatorial n  Limited across domains
explosion documentation
n  Loss of information n  Variations in
when converting implementations
n  Long-term stability n  Limited use for data
exchange

The E57 Format


n  Single, common format n  Thorough, extensive
n  Reduced need to convert documentation
n  Standardized definition n  Reference implementation
n  Binary storage and data n  Widespread use
compression n  General purpose, with domain-
specific extendibility
9
The Road to the E57 Standard

Develop
Specify Design the Write Vote on Encourage
qualification
requirements format standard standard adoption
process

Begin Specify

version 2 requirements

Develop Develop
supporting Testing additional
software tools

10
Guiding Principles

n  Reliable interoperability – Data transferable between vendors


n  Open – Freely available, well documented, unrestricted, and
vendor neutral
n  Low barrier for adoption – Development cost kept to a
minimum
n  Minimalist design – Keep design as simple as possible
n  Extensibility – Allow new capabilities in the future without
breaking core functionality

11
Information to Store in an E57 File

n  Unorganized point clouds


or gridded data
n  Multiple return data
n  Multiple data sets (but in
one coordinate system)
n  Associated images and
pose information

n  Intended for data


exchange and archiving
Image courtesy Intelisum, Inc.
n  Not intended as a
“working” format.
n  Limit meta-information
and derived information

12
Secondary Goals

n  Support for internationalization


n  Support for extremely large file sizes
n  Self-describing – e.g., should not require external lookup
tables
n  Computer readable – i.e., allow automatic verification of syntax
n  Speed and storage efficient – smaller and faster than ASCII
n  Memory efficient – Allow microcontroller implementation
n  LAS compatibility – Superset of LAS functionality

13
E57 Design Basics

Common file format types


Fixed sized fields and records Flexible, self-documenting

n  Rigid, but compact n  Flexible, but inefficient


and efficient and more verbose

E57 Format – A hybrid of the two


n  Flexible, self-documenting framework
n  Fixed sized, user-defined records for large, repeating structures
(e.g., point clouds)
14
E57 Hierarchical File Structure

points
points
points
(PointRecord)
(PointRecord)
(PointRecord)

data3D
bar
(Data3D) pose
bar
(RigidBodyTransform)

pointGroupingSchemes groupingByLine groups


groups
(PointGroupingSchemes) (GroupingByLine) groups
(LineGroupRecord)
(LineGroupRecord)
e57Root (LineGroupRecord)
(E57Root)
pose
(RigidBodyTransform)

images2D
bar
(Image2D) visualReferenceRepresentation
bar
(VisualReferenceRepresentation)

sphericalRepresentation
(SphericalRepresentation)

15
Parts of an E57 File

Header

Binary section (points)

Binary section (points)

Binary section (image)

XML section

16
Error Checking
Header
Binary section
(points)
Logical data stream
Binary section
(points)

Binary section
(image)

XML section 1020 byte logical block

4 byte checksum

Physical data stream

17
XML Hierarchy – The E57 Element Types

Terminal types Non-terminal types

Integer Structure

Float Vector

ScaledInteger CompressedVector

String

Blob

18
Point Data Storage
Example point record prototype:

<points type="CompressedVector" fileOffset="68" recordCount="10">


<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/>
<cartesianY type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/>
<cartesianZ type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/>
<cartesianInvalidState type="Integer" minimum="0" maximum="2"/>
<rowIndex type="Integer" minimum="0" maximum="1"/>
<columnIndex type="Integer" minimum="0" maximum="4"/>
<returnIndex type="Integer" minimum="0" maximum="0"/>
<returnCount type="Integer" minimum="1" maximum="1">1</returnCount>
<timeStamp type="Float"/>
<intensity type="Integer" minimum="0" maximum="255"/>
<colorRed type="Float" precision="single" minimum="0" maximum="1"/>
<colorGreen type="Float" precision="single" minimum="0" maximum="1"/>
<colorBlue type="Float" precision="single" minimum="0" maximum="1"/>
<demo:extra2 type="String"/>
</prototype>

19
Binary Encoding

Blobs
n  Opaque encoding
n  Images, user-defined data

CompressedVectors
n  Point data storage, user-defined data
n  Data organized into chunks for semi-random access
n  Index packets, data packets
n  Data serialization using codecs (bit packing)

20
Image Storage

n  Images stored in blobs (jpg or png format)


n  Image distortion removed
n  Mask to handle non-rectangular images
n  Four camera models ximage
n  Visual reference yimage
imaging plane
n  Pinhole
n  Spherical
principal scene
n  Cylindrical point

center of projection
y
focal
length
z x

21
Extensions

n  Extend format to add new capabilities


Example: LAS extension

n  Define new element types


Example las:edgeOfFlightLine

n  Support backward and limited forward compatibility

22
Implementation – The libE57 software

n  Reference implementation is critical to rapid adoption


n  Goals
n  Cross-platform http://www.libe57.org
n  Open source

n  Foundation API –


Comprehensive
n  Simple API – Easy to use,
designed for common use cases

23
Ongoing Work

n  Finishing development and testing of libE57


n  Working with companies to help with adoption of standard
n  Beginning to work on Version 2 of the standard
n  Advanced compression
n  Representing uncertainty
n  Mobile scanning
n  Improved representation of intensity

24
Supporting Partners

n  ASTM International n  Leica Geosystems


n  Bechtel Corporation n  LiDAR News
n  Carnahan-Proctor and Cross, n  Optech, Inc.
Inc. n  Pointools, Inc.
n  Carnegie Mellon University n  Quantapoint, Inc.
Robotics Institute
n  Riegl Laser Measurement
n  Course Six, Inc. Systems GmbH
n  FARO Technologies Inc. n  Trimble Navigation Limited
n  InteliSum, Inc. n  University of California Davis
n  Inovx, Inc. n  Zoller+Fröhlich GmbH
n  Kubit, GmbH

… and more every week

25
Summary

E57 format – A standard format for storing data from 3D imaging


systems

libE57 – A free reference implementation of the E57 standard


(http://www.libe57.org)

Interested in helping out?


Contact me: Daniel Huber (dhuber@cs.cmu.edu)

26
Comparison with LAS

LAS E57

Domain Aerial sensing General purpose

Point record fields Fixed – several templates User-selectable

Data field bit-resolution Fixed User-selectable

Data geometry Lines Lines, Gridded,


Unorganized
Extensions No Yes

Max data size 4.2E9 records 1.8E19 bytes

27

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy