Bridging the Gap to the Real

Wojciech Matusik

Adobe Systems, Inc.

Computer Graphics: Past, Present, Future

• Goal of this Talk– Look at Trends– Extrapolate– New Challenges

Disclaimer:

My personal (very biased) view

I have not even taken a cg course.

Time

Mile

stones

Time

Mile

stones

Computer Graphics: Past, Present, Future

• Goal of this Talk– Look at Trends– Extrapolate– New Challenges

Disclaimer:I will present my own biased view.

A Simulation to Generate Images or Videos

Simulation

Lights

Shading

Motion

Geometry

Traditional Computer Graphics

Simulation

Lights

Shading

Motion

Geometry

• Components– Hand Modeled– Procedurally

Defined– Analytical

Formulas

Courtesy of Wikimedia

Traditional Computer Graphics: Geometry

Simulation

Lights

Shading

Motion

Geometry

axiom = Fproduction rule F -> FF-[-F+F+F]+[+F-F-F]

L-Systems

Fractals

Perlin Noise, Hypertextures

Courtesy of Wikimedia

Courtesy of Ken Perlin

Courtesy of Wikimedia

Traditional Computer Graphics: Lights

Simulation

Lights

Shading

Motion

Geometry

Directional

Point

Spot

Courtesy of Wikimedia

Traditional Computer Graphics: Shading

Simulation

Lights

Shading

Motion

Geometry

Cook’s Shade Trees

Phong

Blinn-Phong

Cook-Torrance

Courtesy of Rob Cook

Traditional Computer Graphics: Animation

Simulation

Lights

Shading

Motion

Geometry

Animation Controllers

Keyframing

Courtesy of Jessica Hodgins

Traditional Computer Graphics

Real World

Lights, Geometry, Shading, Motion,

Simulations

Editing and User

Input

Computer Graphics

Data-driven Computer Graphics: 90s - now

Real World

Lights, Geometry, Shading, Motion,

Simulations

Input Devices

• Progress in development of sensors

• Cheap storage and computation power for data processing

Data-driven Computer Graphics: Geometry

Simulation

Lights

Shading

Motion

Geometry

3D Scanning

Courtesy of Marc Levoy

Data-driven Computer Graphics: Lights

Simulation

Lights

Shading

Motion

Geometry

Environment MappingHDR Imaging

Courtesy of Paul DebevecCourtesy of Wikimedia

Data-driven Computer Graphics: Shading

Simulation

Lights

Shading

Motion

Geometry

Measured Appearance

Courtesy of Kristin Dana

Courtesy of Wikimedia

Data-driven Computer Graphics: Animation

Simulation

Lights

Shading

Motion

Geometry

Motion Capture

Courtesy of Wikimedia

Data-driven Computer Graphics

Real World

Lights, Geometry, Shading, Motion,

Simulations

Input Devices

Data-driven Modeling and Editing

Real World

Lights, Geometry, Shading, Motion,

Simulations

Input Devices

Editing and User

Input

Data-driven Modeling and Editing: Geometry

Simulation

Lights

Shading

Motion

Geometry

Modeling by Example, 2004Funkhouser et al.

Data-driven Modeling and Editing: Lights

Simulation

Lights

Shading

Motion

Geometry

Skyfinder : Attribute-based Sky Image Search, 2009Tao et al.

Data-driven Modeling and Editing: Shading

Simulation

Lights

Shading

Motion

Geometry

Data-driven Reflectance Model, 2003Matusik et al.

Data-driven Modeling and Editing: Animation

Simulation

Lights

Shading

Motion

Geometry

Motion Graphs, 2002Kovar et al., Lee et al., Arikan & Forsyth

Data-driven Computer Graphics: 90s - now

Real World

Representations

AbstractionsSimulations

Input Devices

Editing and User

Input

Computer Graphics: Future

Real World

Representations

AbstractionsSimulations

Input Devices

Output Devices

Editing and User

Input

Shape Capture, Modeling, and Fabrication

Real Object 3D Model3D Scanning 3D Printing Output

User Edits

Courtesy of Wikimedia

Motion Capture, Modeling, and Robotics

Human Motion Motion ModelMocap ControllerDesign

HumanoidRobot

User Edits

Courtesy of WikimediaCourtesy of Wikimedia

Light Field Capture, Modeling, and Display

3D Scene Light FieldCameraArray

AutostereoscopicDisplay

Light Out

User Edits

Courtesy of M.Levoy

Courtesy of F. Durand

Appearance Capture, Modeling, and Fabrication

Real Material Data-drivenModel

Measurement Fabrication Output

User Edits

Challenges: Output Mapping

Data Abstraction

Device Capabilities Device

Capabilities

Data Abstraction

Output Mapping Examples

– Gamut Mapping

– Tone Mapping

– Image Retargeting

Courtesy of Wikimedia

Courtesy of Shai Avidan

Courtesy of Wikimedia

• Different Phenomena– Reflectance, Scattering, Deformations, Motion

• Rules– Linear vs. Perceptually-Based– Global vs. Local

Generalized Output Mapping Algorithms

• Complex aggregates from base materials with known properties

Future Printers, Displays, and Output Devices

Future Output Devices

Future Printers, Displays, and Output Devices

Data-drivenRepresentatio

nsand

Abstractions

Output Mapping

Computer Graphics: Closing the Loop

Real World

Representations

AbstractionsSimulations

Input Devices

Output Devices

Editing and User

Input

My Vision Statement

The future of Computer Graphics and Interactive Techniques lies in novel input and output devices that operate in real-world environments and are accessible to everyone.