Semantic ScrollingHow to navigate structured documents

An Introduction

In the World of Data• Data exists in multiple forms

INT

RO

DU

CT

ION

• Many ways of navigating data

• What is Semantic Scrolling?

A hypothetical example.• Let’s look at Twitter

Scrolling by pixels

Why can’t we scroll by tweets?

Or by time?

INT

RO

DU

CT

ION

Google Maps Directions.• Side Panel: a list of directions from A to B

Each item maps from a heading in the side panel to a geographic location

scroll semantically

INT

RO

DU

CT

ION

Example. INT

RO

DU

CT

ION

Microsoft Word Search.• Little known function of the navigation panel

Move through document by Graphics, Tables, Equations, etc.

scroll semantically

INT

RO

DU

CT

ION

Example. INT

RO

DU

CT

ION

How to categorize?• By the document structure?

INT

RO

DU

CT

ION

• By the mechanics?

rendering, storage etc.

Input, Interactive elements, etc.

Doc Structure

1. The Data Structure

• Linear – one level of detail

DO

CU

ME

NT

ST

RU

CT

UR

E

• Hierarchical – multiple levels of detail

• Data Structure = how the data is stored

2. Rendering• Rendering = how the data is represented to the user for consumption

DO

CU

ME

NT

ST

RU

CT

UR

E

• Linear & Hierarchical applies here as well

ex. Linear data rendered linearly

ex. Hierarchical data rendered linearly

ex. Hierarchical data rendered hierarchically

ex. Linear data rendered hierarchically ex. Linear data rendered hierarchically

3. Exposed• Exposed = how the data is represented to the user for navigation

DO

CU

ME

NT

ST

RU

CT

UR

E

• Linear & Hierarchical applies here as well

• Note: How a document is Exposed can be independent from how it is Rendered

Putting it Together

DO

CU

ME

NT

ST

RU

CT

UR

E

Data Rendered Exposed Example

Data Rendered Exposed Example

Linear Linear Linear - Simple lists

Hierarchy - Lists w/ categorization

Data Rendered Exposed Example

Linear Linear Linear - Simple lists

Hierarchy - Lists w/ categorization

Hierarchy Linear Linear - List of nodes in tree (ordered?)

Hierarchy - eBooks table of contents

Data Rendered Exposed Example

Linear Linear Linear - Simple lists

Hierarchy - Lists w/ categorization

Hierarchy Linear Linear - List of nodes in tree (ordered?)

Hierarchy - eBooks table of contents

Hierarchy Linear - List of nodes in tree (ordered?)

Hierarchy - Semantic zooming, i.e. expand / collapse tree at a depth

Mechanics

1. Inputs• Touch:

ME

CH

AN

ICS

• Mouse:

tap, swipe, drag, pinch, etc.

click, drag, mouse wheel, secondary buttons

• Keyboard: (not really interesting)

arrow keys, hot keys

2. Targeting

up-down semantic movements in data (up/down level of detail)

ME

CH

AN

ICS

prev-next semantic movements in data (across same level of detail)

• Absolute movements in data (random access)

• Relative movements in data

3. Interaction Manifestation• Invisible – interaction is done to the content direction, no other feedback

ME

CH

AN

ICS

• Invoke – an element is invoked for the semantic scrolling to take place

• Persistent – a persistent element exists for semantic scrolling

Framework

Doc Structure & Mechanics

FR

AM

EW

OR

K

Data Rendered Exposed

Linear Linear Linear

Hierarchy

Hierarchy Linear Linear

Hierarchy

Hierarchy Linear

Hierarchy

• InputTouch, Mouse,

Keyboard

• TargetingRelative, Absolute

• Interaction Manifestation Invisible, Invoke,

Persistent

Size of the design space

FR

AM

EW

OR

K• Total Doc Structure categories = 6

• Total Mechanics categories = 18 Input = 3, Targeting = 2, IM = 3

• So, 108 categories in total?

at least 108 prototypes?

NOT EXACTLY

Focus• Certain categories are dull

FR

AM

EW

OR

K

• Certain combinations are dull

Keyboard inputs, Data storage structure, etc.

Ex. Keyboard & persistent/invoked

Ex. Touch & absolute movement

• Focus: Mechanics that compliment certain document structures

Demo

Demo #1 overview• Document structure

DE

MO

Data: Hierarchical

• Mechanics

Render: LinearExposed: Hierarchical

Input: Keyboard (hotkeys)Targeting: Relative Interaction Manifestation: Invisible

Demo #1

DE

MO

• Purpose:Show the backend scrolling algorithmDocument has 4 levels of detail (LOD)

Parts, Chapters, Sections, SubsectionsScrolling at a certain LOD will scroll through

the parts of the document at that LOD and higher, but not lower. Like a depth limited tree traversal Prevents jumps, gives context

Demo #2 overview• Document structure (same document)

DE

MO

Data: Hierarchical

• Mechanics

Render: LinearExposed: Hierarchical

Input: TouchTargeting: Relative Interaction Manifestation: Invisible

Demo #2a

DE

MO

• Purpose:Touch gesture for relative movement with 2

taps scrolling2 taps to right = more fine (higher LOD)2 taps to left = more coarse (lower LOD)

Demo #2b

DE

MO

• Purpose:Touch gesture for relative movement with 2

finger scrollingWidth of space between 2 fingers determines

LOD

FINAL SLIDE• Additional comments?

TH

E E

ND

• Thank you for your time.