iPaper@GlobIS - Interactive Paper Research
-
Upload
beat-signer -
Category
Science
-
view
1.269 -
download
3
description
Transcript of iPaper@GlobIS - Interactive Paper Research
CSCW Seminar 2007, June 12, 2007
iPaper@GlobIS – Interactive Paper Research
CSCW Seminar
Beat Signer, Nadir Weibel, Adriana Ispas and Moira C. Norrie
Global Information Systems Research Group
Department of Computer Science, ETH Zurich
http://vub.academia.edu/BeatSigner
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Problems of Existing Applications
Many projects focus on the input device, paper,
printing and other hardware technologies rather
than on the data integration and information
management aspects isolated solutions
The linking of paper tends to be based on
physical rather than information-centric concepts
difficult to integrate new input / output devices
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Our Approach
General interactive framework
different forms of paper-based interactions (enhanced
reading, enhanced writing, …)
rapid prototyping as well as large scale publishing
integration of new types of resources and services
integration of new input devices
The key to a highly integrated interactive paper
solution lies in the introduction of a platform for
general cross-media information management
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Active Areas and Virtual Page Layers
image
video clip
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iPaper Architecture
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iServer Metamodel and Resource Plug-ins
iPaper
Shape Page
Application Database
DB ObjectData/Metadata
Query
Movie Clips
Time Span /
Spatial InfoMovie Clip
HTML Documents
XPointer XHTML
Document
Selectors
Entities
Resources
link
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iServer
layer
|Layers|
layer
Active
Layers
resource
Containers
activeComponent
Active Com-
ponents
medium
Media
medium
Movies
medium
Images
medium
Webpages
OnLayer
Contains
(1,1)
(1,1) (0,*) (0,*)
document
Documents(0,*)
shape
Shapes
partition (2,*)
(0,*)
rectangle
Rectangles
ellipse
Ellipses
polygon
Polygons
circle
Circles
complexShape
Complex
Shapes
HasShapes
page
PagesOnPage
Contains
Pages
(0,*)
(1,1)
(1,1)
(0,*)
Resources
parameter
Preferences
link
Links
user
Users
group
Groups
individual
Individuals
HasMembers
HasTarget
(0,*)
(0,*)(0,*)
(0,*)
(0,*)
(0,*) (0,*)
(1,*)
(0,*)
(0,*)
(1,1)
(0,*)
partitionpartition
(1,*)
CreatedBy
RefersTo
Has
Preferences
parameter
Properties
selector
Selectors
resource
Resources
HasSource
(0,*)
AccessibleTo
InaccessibleTo
(0,*)
(0,*)
entity
EntitiesHasProperties
Users
Plug-ins
Links
(0,*)
Layers
iPAPER
iSERVER
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iServer Interfaces
Java API
XML import and export
Interface based on XML and XSLT
XHTML
WAP, i-mode
VoiceXML
Web Service
Peer-to-peer (P2P)
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iWeb Firefox Extension
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iWeb Plug-in
Firefox Web Client
SOAP Request
SOAP Response
World Wide Web
HTTP
Request
HTTP
Response
Link DBiWeb Plug-in
iServExt
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Active Content
Execute code (services) on the client and server side
Active Component
A
Active Component
B
Active Component
C
iS
erv
er
AC Manager
Server
R
eq
ue
st S
en
de
rActive Component
Stub B
AC Names
ACS Manager
C
lien
t D
evic
e
AC Names
Client
HT
TP
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Active Components
Button
Slider
Slider2D
DateSelector
Browser
MapSelector
Rating
Capture
CaptureAndSpeak
Image
Sound
Movie
SQLClient
XCMRequest
ClientCtrlRequest
PaperPointControl
PaperPointAnnot
...
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Active Components …
<?xml version="1.0" encoding="UTF-8" ?>
<iserver>
<activeComponent id="s1a" creator="beat">
<name>Show slide 1</name>
<properties>
<parameter>
<key>org.ximtec.iserver.ac:command</key>
<value>showSlide</value>
</parameter>
<parameter>
<key>org.ximtec.iserver.ac:slide</key>
<value>1</value>
</parameter>
</properties>
<identifier>PAPERPOINT_CONTROL</identifier>
</activeComponent>
...
</iserver>
PAPERPOINT_CONTROL
Show slide 1
command showSlide
slide 1
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
PaperPoint Presentation Tool
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
XML-Based Authoring
...
<document id="handout" creator="beat">
<name>Slide Handout</name>
<id>h1</id>
<size>
<width>210</width>
<height>297</height>
</size>
<content>handout.pdf</content>
</document>
<page id="p1" creator="beat" document="h1">
<name>Page 1, Slide Controller</name>
<number>1</number>
</page>
...
<rectangle id="fs" creator="beat"
layer="default" resource="p1">
<name>First Slide</name>
<upperLeft>
<point><x>24</x><y>12</y></point>
</upperLeft>
<size>
<width>25</width><height>8</height>
</size>
</rectangle>
...
...
<activeComponent id="fsa" creator="beat">
<name>First slide action</name>
<properties>
<parameter>
<key>org.ximtec.iserver.ac:command</key>
<value>firstSlide</value>
</parameter>
</properties>
<identifier>PAPERPOINT_CONTROL</identifier>
</activeComponent>
...
<link id="fsl" creator="beat" sources="fs"
targets="fsa">
<name>Show the first slide</name>
</link>
...
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Digital Link Authoring Tool
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
PaperPoint Architecture
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
EdFest Components
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
EdFest Architecture Overview
iPaper Client
XCMcontent publishing
platform
Metadata DB Appln DB
iServer/iPapercross-media
link server
Link DB
Context
Engine
Context DB
Client
Controller
Active
Components
Text-to-Speech
Engine ICR Software
handwriting
recognition
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Content Publishing Framework
XCMcontent publishing
XML data
of festival
venues
and events
iServercross-media link server
• paper link data
• active areas on paper
• layers
• digital resources
• user modelimport
publish
XML link
definition
data
iPublishpublishing framework
Structure Style
XSL CSS
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Print-n-Link
Citations build the basis
for a scientific web of
publications
Use digital services to
search / retrieve cited
publications
Automatic authoring
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Print-n-Link Architecture
Display
Printer
Driver
Document DB
iPAPER
iDOC
Publish
Document
Pattern
Semantic Mapper
Citation
Analyser
Citation
Retrieval
Document
Lookup
Citation DB
Print-n-Link Engine
iSERVER
Active Content
Link DB
iPaper Plug-in
Publish
Positions and
Active Content
Definition
Publish
Citations
Desktop
(x,y)
Coordinates
Text-to-
Speech
Web Search
Services
iPaper
Client
Print-n-Link: Weaving the Paper Web
Moira C. NorrieDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
Beat SignerDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
Nadir WeibelDept. of Computer Science
ETH Zurich8092 Zurich, Switzerland
ABSTRACT
Citations form the basis for a web of scientific publications.Search engines, embedded hyperlinks and digital librariesall simplify the task of finding publications of interest onthe web and navigating to cited publications or web sites.
However the actual reading of publications often takes placeon paper and frequently on the move. We present a systemPrint-n-Link that uses technologies for interactive paper toenhance the reading process by enabling users to access dig-ital information and/or searches for cited documents from a
printed version of a publication using a digital pen for inter-action. A special virtual printer driver automatically gener-ates links from paper to digital services during the printingprocess based on an analysis of PDF documents. Depend-ing on the user setting and interaction gesture, the systemmay retrieve metadata about the citation and inform the
user through an audio channel or directly display the citeddocument on the user’s screen.
Categories and Subject Descriptors
D.2.11 [ Software Engineering ]: Software Architectures;H.4.m [ Information Systems Applications ]: Miscella-neous; H.5.m [ Information Interfaces and Presenta-tion ]: Miscellaneous
General Terms
Design, Algorithms, Experimentation
Keywords
Interactive paper, document integration, digital library, ci-tation management
1. INTRODUCTIONNowadays we are all familiar with the notion of hyperlinks
as a means of creating interconnected document spaces thatcan be easily searched and navigated with the help of search
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.DocEng’06, October 10–13, 2006, Amsterdam, The Netherlands.Copyright 2006 ACM 1-59593-515-0/06/0010 ...$5.00.
engines and web browsers. The ease with which we cannot only find documents on the web, but also follow linksto related documents is considered to be one of the mainadvantages of working with digital documents as opposedto paper equivalents. Yet many studies have shown thatpaper is still the preferred med ium when it comes to reading
documents for reasons of comfort, mobility and also ease ofannotation.
In this paper, we show how emerging technologies for in-teractive paper enable a bridge to be built between paperand digital document spaces that allows users to travel backand forth between the two using the medium most appropri-
ate to the current task and sit uation. Users can print doc-uments for reading and, by activating hyperlinks on paperwith a digital pen, they can not only access digital informa-tion about related documents, but also initiate web searchesto retrieve copies of those documents. The retrieved docu-
ments may in turn be printed for reading with automaticallygenerated embedded hyperlinks on the paper. Mobility issupported by providing disconnected users on the move withaudio information about related documents and the optionof adding them to a request list to be processed when recon-nected.
As an application, we take the everyday academic task ofreading scientific publications. Whether the reading activ-ity is for the purpose of knowledge discovery or reviewing,readers regularly look up citations to other publications byflicking to the reference list at the back of the paper. In somecases, a search for the cited publication may follow and, if
found, often it is printed and added to a pile of documents tobe read. Citations are the established meansof providing thelinks that form the basis of the scientific web of publicationsthat spans the digital and paper worlds. Just as citationswithin digital documents can be used to automatically gen-erate hyperlinks to cited documents, we can automatically
generate links within paper documents through an analysisof the PDF file when the document is printed.
Section 2 presents a more detailed look at the activityof reading scientific publications and a discussion of relatedwork. Section 3 describes the digital pen and paper tech-
nologies and how these can be used to support interactionbetween paper and digital services. We then present thegeneral functionality of our enhanced reading system in Sec-tion 4 and the overall architecture in Section 5. Details ofthe virtual printer component used to analyse PDF docu-ments for automatic link generation as well as printing are
given in Section 6. Concluding remarks follow in Section 7.
Interactive Paper Architecture
PrintingSemantic Mapper
Document Retrieval
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Printed-Digital Mapping
static physical rendering
dynamic logical document
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Printed-Digital Mapping
Source Digital
DocumentPaginated
(Print-Ready)
Document
Enhanced
Virtual Printer
Interactive
Document
(e.g. Anoto)
Augmenting
Technology
(ex. Anoto)
Geometrical Shapes,
Active Components
Logical
RepresentationPhysical
Representation
iServer
(ID, page, x,y)
iDoc
Digital
Elements
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Mixed Physical-Digital Model
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Interactive Paper Applications
Enhanced Reading interactive paper maps
interactive festival brochures
educational materials
auction brochures
scientific publications
…
Enhanced Writing photo album
researcher's notebook
mammography
search system
…
Paper-Based Interfaces
PaperPoint
digital libraries
video analysis tool
educational games
Artists' Installations
Lost Cosmonaut
Generosa Enterprise
Interactive TableTops
http://www.globis.ethz.ch/research/paper/applications/
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Mammography Annotations
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
The Lost Cosmonaut
Writing as a collaborative
act of memory and story-
telling
Reading, writing and
narrative as an act of
making sense
Handwriting as a tool for
human computer
interaction
In collaboration with Axel Vogelsang,
Artists in Labs Programme
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Generosa Enterprise
Interactive drawing installation
Exhibited at the 150 Years ETH Zurich Jubilee (more
than 200'000 visitors)
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iGesture Framework
Common interface for
different gesture
recognition algorithms
Tools for implementing
and evaluating new
algorithms
Gesture set management
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
iGesture Admin Tab
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Capture of Samples
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Conclusions
Interactive paper framework (iPaper)
information-centric approach supports different
technologies, functionalities and applications
rapid prototyping (reusable active components)
Extensible platform for cross-media information
management (iServer)
Cross-media authoring / publishing tools
manual, content-driven and automatic link authoring
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
Student Projects
Basic technologies and infrastructure
iServer resource plug-ins, …
Integration of new tracking technologies
Interactive paper authoring tools
iGesture recognition framework (www.igesture.org)
Paper-based applications
PaperPoint, paper-based interface to digital
agenda/todo list (e.g. Outlook), paper-digital
notebook, paper-digital photo album, …
34
We offer Master and Semester projects and are also looking for Hilfsassistenten
CSCW Seminar 2007, June 12, 2007 Beat Signer, [email protected]
References
http://www.ipaper.inf.ethz.ch
http://www.igesture.org
Fundamental Concepts for Interactive Paper and
Cross-Media Information Spaces, B. Signer,
Diss ETH Zurich Nr. 16218, 2006
35