The ASTM E57 File Format for 3D Imaging Data Exchange

Daniel Huber The Robotics Institute

Carnegie Mellon University Member of the ASTM E57 Committee on 3D Imaging Systems

Sub-committee on Data Interoperability

Image courtesy Intelisum, Inc.

2

The E57 Format What have we done? Over the past 5 years, we have developed an open standard for 3D imaging system data exchange. n  Store and exchange:

n  3D data (point clouds, range images)

n  Associated images n  Meta-data to support

downstream processing n  General purpose – terrestrial,

aerial, mobile n  Extendable Who is “we”? ASTM E57 Committee on 3D Imaging Systems, Sub-committee on Data Interoperability (E57.04)

3

What is Three Dimensional Imaging?

Laser Scanners

Triangulation

4

Uses of 3D Imaging: Modeling and Visualization

5

Uses of 3D Imaging: Robotics

6

Uses of 3D Imaging: Civil Engineering Quality assurance

Reverse engineering

Infrastructure inspection

n  LAS

How Do People Store 3D Data Today?

7

Proprietary Formats n  PTS n  PTX n  DXF

Ad-Hoc Formats

Domain Specific Formats

n  PLY n  homebrew

What’s the Problem with the Status Quo?

8

Proprietary Formats n  Data exchange

combinatorial explosion

n  Loss of information when converting

n  Long-term stability

Ad-Hoc Formats

Domain Specific Formats

n  Storage efficiency n  Limited

documentation n  Variations in

implementations n  Limited use for data

exchange

n  Limited applicability across domains

n  Standardized definition n  Widespread use n  Reference implementation

n  Binary storage and data compression

n  Thorough, extensive documentation

n  General purpose, with domain-specific extendibility

n  Reduced need to convert n  Single, common format

How Does the E57 Format Address These Problems?

9

Proprietary Formats n  Data exchange

combinatorial explosion

n  Loss of information when converting

n  Long-term stability

Ad-Hoc Formats

Domain Specific Formats

n  Storage efficiency n  Limited

documentation n  Variations in

implementations n  Limited use for data

exchange

n  Limited applicability across domains

The E57 Format

… Specify requirements

Begin version 2

Develop additional tools

Testing Develop supporting software

Develop qualification process

Encourage adoption

Vote on standard

Write standard

Design the format

10

The Road to the E57 Standard

Specify requirements

11

Guiding Principles

n  Reliable interoperability – Data transferable between vendors n  Open – Freely available, well documented, unrestricted, and

vendor neutral n  Low barrier for adoption – Development cost kept to a

minimum n  Minimalist design – Keep design as simple as possible n  Extensibility – Allow new capabilities in the future without

breaking core functionality

12

Information to Store in an E57 File

n  Unorganized point clouds or gridded data

n  Multiple return data n  Multiple data sets (but in

one coordinate system) n  Associated images and

pose information

n  Intended for data exchange and archiving

n  Not intended as a “working” format.

n  Limit meta-information and derived information

Image courtesy Intelisum, Inc.

13

Secondary Goals

n  Support for internationalization n  Support for extremely large file sizes n  Self-describing – e.g., should not require external lookup

tables n  Computer readable – i.e., allow automatic verification of syntax n  Speed and storage efficient – smaller and faster than ASCII n  Memory efficient – Allow microcontroller implementation n  LAS compatibility – Superset of LAS functionality

14

E57 Design Basics

Common file format types Fixed sized fields and records

n  Rigid, but compact and efficient

Flexible, self-documenting

n  Flexible, but inefficient and more verbose

E57 Format – A hybrid of the two n  Flexible, self-documenting framework n  Fixed sized, user-defined records for large, repeating structures

(e.g., point clouds)

15

E57 Hierarchical File Structure

barbarimages2D(Image2D)

pose(RigidBodyTransform)

pointGroupingSchemes(PointGroupingSchemes)

points(PointRecord)

points(PointRecord)

points(PointRecord)

groupingByLine(GroupingByLine) groups

(LineGroupRecord)

groups(LineGroupRecord)

groups(LineGroupRecord)

pose(RigidBodyTransform)

visualReferenceRepresentation(VisualReferenceRepresentation)

sphericalRepresentation(SphericalRepresentation)

e57Root(E57Root)

barbardata3D(Data3D)

16

Parts of an E57 File

Header

Binary section (points)

XML section

Binary section (image)

Binary section (points)

17

Error Checking

Logical data stream

Physical data stream

1020 byte logical block

4 byte checksum

Header

Binary section (points)

XML section

Binary section (image)

Binary section (points)

XML Hierarchy – The E57 Element Types

18

Terminal types

Integer

Float

ScaledInteger

String

Blob

Non-terminal types

Structure

Vector

CompressedVector

19

Point Data Storage

<points type="CompressedVector" fileOffset="68" recordCount="10"> <prototype type="Structure"> <cartesianX type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/> <cartesianY type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/> <cartesianZ type="ScaledInteger" minimum="0" maximum="32767" scale="1e-003"/> <cartesianInvalidState type="Integer" minimum="0" maximum="2"/> <rowIndex type="Integer" minimum="0" maximum="1"/> <columnIndex type="Integer" minimum="0" maximum="4"/> <returnIndex type="Integer" minimum="0" maximum="0"/> <returnCount type="Integer" minimum="1" maximum="1">1</returnCount> <timeStamp type="Float"/> <intensity type="Integer" minimum="0" maximum="255"/> <colorRed type="Float" precision="single" minimum="0" maximum="1"/> <colorGreen type="Float" precision="single" minimum="0" maximum="1"/> <colorBlue type="Float" precision="single" minimum="0" maximum="1"/> <demo:extra2 type="String"/> </prototype>

Example point record prototype:

20

Binary Encoding

Blobs n  Opaque encoding n  Images, user-defined data CompressedVectors n  Point data storage, user-defined data n  Data organized into chunks for semi-random access n  Index packets, data packets n  Data serialization using codecs (bit packing)

21

Image Storage

n  Images stored in blobs (jpg or png format) n  Image distortion removed n  Mask to handle non-rectangular images n  Four camera models

n  Visual reference n  Pinhole n  Spherical n  Cylindrical scene

x

y

z

ximageyimage

focallength

center of projection

principalpoint

imaging plane

scene

x

y

z

ximageyimage

focallength

center of projection

principalpoint

imaging plane

22

Extensions

n  Extend format to add new capabilities Example: LAS extension

n  Define new element types

Example las:edgeOfFlightLine n  Support backward and limited forward compatibility

23

Implementation – The libE57 software

n  Reference implementation is critical to rapid adoption n  Goals

n  Cross-platform n  Open source

http://www.libe57.org

n  Foundation API – Comprehensive

n  Simple API – Easy to use, designed for common use cases

24

Ongoing Work

n  Finishing development and testing of libE57 n  Working with companies to help with adoption of standard n  Beginning to work on Version 2 of the standard

n  Advanced compression n  Representing uncertainty n  Mobile scanning n  Improved representation of intensity

Supporting Partners

n  ASTM International n  Bechtel Corporation n  Carnahan-Proctor and Cross,

Inc. n  Carnegie Mellon University

Robotics Institute n  Course Six, Inc. n  FARO Technologies Inc. n  InteliSum, Inc. n  Inovx, Inc. n  Kubit, GmbH

n  Leica Geosystems n  LiDAR News n  Optech, Inc. n  Pointools, Inc. n  Quantapoint, Inc. n  Riegl Laser Measurement

Systems GmbH n  Trimble Navigation Limited n  University of California Davis n  Zoller+Fröhlich GmbH

25

… and more every week

26

Summary

E57 format – A standard format for storing data from 3D imaging systems libE57 – A free reference implementation of the E57 standard (http://www.libe57.org) Interested in helping out? Contact me: Daniel Huber (dhuber@cs.cmu.edu)

27

Comparison with LAS

LAS E57

Domain Aerial sensing General purpose

Point record fields Fixed – several templates User-selectable

Data field bit-resolution Fixed User-selectable

Data geometry Lines Lines, Gridded, Unorganized

Extensions No Yes

Max data size 4.2E9 records 1.8E19 bytes