Max Mühlhäuser
description
Transcript of Max Mühlhäuser
Ubiquitous Computing
Max Mühlhäuser, Iryna Gurevych (Editors)
Preface: Systematic Approach to Ubiquitous Computing and its Application to Real-Time Enterprises
Max Mühlhäuser
UbiquitousComputing
P-0: Outline - What this chapter covers
P-1 A first introduction– ‘Definition’: What is Ubiquitous Computing?– Importance: (why) is Ubiquitous Computing (UC) so important? First
glanceP-2 UC challenges – a first approach
– The challenge of humane UC– The challenge of integrated UC
P-3 The S.C.A.L.E. classification of Ubiquitous Computing– Intro: broad vs. deep: what should a treatment of UC cover – and why – Explanation of & introduction to the S.C.A.L.E classification– Relation to major Computer Science research fields
P-4 Reader’s Digest– Relation of S.C.A.L.E. classification to further chapters– Digest for Readers with different background/goals
2
UbiquitousComputing
P-1.1 Motivation & Definition: What is UC?
• Ubiquitous Computing describes the 3rd “era” of computing– Era 1: Mainframe Computing MC - many users use one computer– Era 2: Personal Computing PC - one user uses one computer– Era 3: Ubiquitous Computing UC - one user uses many computers
• Note, for Era 3:– some computers are worn (mobile phone, PDA; body sensors, life recorder…)– other computers are encountered on-the-move, more or less consciously
(cf. info kiosk / data projection versus badge-reader / surveillance cam)
• UC means “networked computers everywhere”– µC’s were embedded in VCRs, washing machines etc. since decades already!– BUT: now, with UC, the embedded computers become
• networked, i.e. part of a “web” of interacting computers surrounding us• “programmable”: changing firmware / SW is easier, less noticeable to user
(good: cf. bug fixes; bad: cf. undesired functionality, e.g. spying)
3
UbiquitousComputing P-1.1 Definition: What is UC?
• UC connects general & special purpose computers– the distinction is more a spectrum than two distinct classes– the power / resource spectrum (CPU, memory, bandwidth, …)
becomes huge!
How to ‘define’ Ubquitous Computing?A simple definition is given above: UC is the third era of
computing according to the aforementioned categorization
Our more detailed and self-sustained definition:Ubiquitous Computing is the dawning era of computing, in which individuals are surrounded by many networked, spontaneously yet tightly cooperating computers, some of them worn or carried, some of them encountered on the move, many of them serving dedicated purposes as part of physical objects, all of them used in an intuitive, hardly noticeable way with limited attention”.
4
UbiquitousComputing
P-1.1 Importance: Sample Nodes in UC networks
5
Tiny, rather general purpose computers acting, e.g., as nodes in sensor networks (cf. UC Berkeley’s motes, Particle Computer GmbH’s particles)Wearables and mobile computing
devices. cf. handhelds for ware-house picking, washable computerjackets, or companions like the “lovegetty” pictured: stores user’sprofile, beeps when a “compatible”person – with lovegetty – appears
Networked appliances, e.g., “smart vending machines” thattransmit fill status, errors etc. to head offices; press rumors about adaptive pricing caused protests
Smart labels such as RFIDs(above) and active badges(below) identify (option: locate, characterize)objects, humans, animals; benefits and privacy threatswidely discussed in press
UbiquitousComputing
P-2.3 Imporance: Exploding # of nodes
issue #1: on the networking side, the exponential growth (# of nodes connected in the Internet) continues way beyond the world’s populationconsequence: scalability becomes even more crucial
6
Appli-ances
everyday things
„Web“
wirelessInternet mobile non tech users
P2P P2M M2M People Machines
2000 time
Billions of
Zillions of
109
108
107
106
1010
time
log (capacity) Internet #nodes, wired speed
UbiquitousComputing
P-2.3 Importance: exploding # of nodes, consequence
issue #2: different doubling rates for networks, CPUs (Moore’s law) etc.; impact on: system architectures, algorithms, etc. (e.g., RPC becomes cheaper, mobile computing maybe more expensive?)
7
CPU (Moore‘s Law)
2000 time
log (capacity) Internet (# of nodes)
unpredictable: wireless-net bandwidth (bps)battery capacity
distributed computing:- disconnected-op?- mobile code?- server architectures
?? Impact on ...
wired-net bandwidth (bps) graphic cards (#polygons/sec)
UbiquitousComputing P-1.2 Importance: Conclusion
At the 30,000 ft. level, UC is “the next key technology to change our society” for 3 reasons:
1. UC describes the next era of computing– therefore, all of computer science / IT is potentially impacted– … it will be difficult to be selective in designing this course/book
(but we will see that a course / book on UC still makes a lot of sense)
2. UC has potential impact on every facet of our lives– computing is no more “what we do when we sit at the computer” +
“what is encapsulated/hidden deep inside VCRs etc.”– see the more / less comparison on next slide
3. UC is impossible and inevitable at the same time– components (“gadgets”) are developed and deployed, cf.
slide “UC sample nodes” above– more and more UC scenarios become profitable industry will push– many issues of “the whole” i.e. integrated UC systems still unresolved
8
UbiquitousComputing
P-2.1 From Importance to Challenges
A first glance (not yet structured, non exclusive) at challenges…
Anytime/anywhere presence of networked computers means:• more sensitivity less protection (cf. security)• more dependence less perfection (cf. reliability)• more obtrusion less attention (cf. HCI)• more garrulity less throughput (cf.
networking)
In other words: as computers become ubiquitous …• there is a risk that “the whole may be way less than the sum of the
parts” i.e. the desired integral functionality may lack way behind• some known problems aggravate considerably – we’ve got to care!• in short, UC is a problem of complex integrated systems, not of gadgets
compare to: new Airbus, Space Shuttle, not better rocket-fuel, headlights
9
UbiquitousComputing
P-2.1 Humane & Integrated UC - Overview
10
Top Level Challenges:A. Systems Integration
(Strength of Europe)B. Humane Computing
(Chance for Europe)
Outlook (see below): Scalability Connectivity Adaptability Liability (=Security +
+) Ease-of-Use
features of UC „gadgets“ are not the problem …
… but making them „humane“ and integratingthem into a useful whole
UbiquitousComputing P-2.2 Challenges: Humane UC
Mainstream UC Research emphasizes two issues:• Context-Aware Computing
ill defined; basically, the quest for adaptive software that …• understands the „situation in which the user currently works“• adapts to that situation by
– reducing complexity (e.g., adjusting UI to current needs & options)– automating input (e.g., retrieving data from „environment“, not user)– optimizing functionality
• Multiple Modalitiesagain, ill defined; modality is a „way of doing I/O“, often an
input and/or output „channel“ (what is that?) like graphics, voice,keyboard/mouse, gesture (note: examples from different categories!)
• UC applications on appliances (cell phone, PDA… today, more tomorrow) Multiple modalities to be supported• truly multimodal UI: flexible combination of „channels“, user‘s choice
• We will put these two issues in a larger context
11
UbiquitousComputing P-2.3 Challenges: UC Integration
12
?
Pertinent Application:Real-Time Enterprise.full coverage:production/supply chain,retail, post-sales
High-tier
Low-tier
Satellite
High Mobility Low MobilityWide Area
RegionalArea
LocalArea
Global Internet: seamlesscontext-aware (service!) network
UbiquitousComputing
P-3.1: S.C.A.L.E. Overview: Breadth vs. Depth
• Problem: UC touches almost every area of Computer Science, and more!– readers need background for understanding why what will be emphasized
later– a categorization of issues (taxonomy) is needed for organizing the topics
and for getting oriented cf. the S.C.A.L.E. taxonomy• So, is it a research/teaching subject in its own right?
– yes, compare to Distributed Systems:• there is distributed simulation, programming, algorithms, databases, AI, …
“distributed everything”!• yet, there are courses on Distributed Systems
– but: how to master breadth vs. depth?• be broad first, organize “problem space” S.C.A.L.E., see below• then emphasize pertinent problems
– either “new” aspect in UbiComp– or “much more relevant” aspect with UbiComp than without
• standard computer scientist should hardly need add’l background• … but will sometimes want to dig deeper in some subtopic … which is normal
• But then, is course/book going to be “complete”?– no, but more complete than any other book/course before– … and more well organized, due to S.C.A.L.E. scheme, see below
13
UbiquitousComputing P-3.1: S.C.A.L.E. - Rationale for Taxonomy
UC is novel subject reader needs overview of problem space
“Problem Space” approached in two ways:1. Holistic view of pertinent research fields
– S.C.A.L.E. classification scheme devised for that purpose– explained below, used throughout for structuring major “guide” through residual chapters
2. Holistic view of “the global UC network=system”– Mundo Reference Architecture devised for that purpose
• imagine as rough floorplan for an “entire” UC system• shows differences to Distributed Systems Ref-Arch’s and Definitions
– Mundo not supposed to be accepted worldwide• (might, at best, influence a future Ref.Arch. standard)• it’s purpose is to help readers in understanding & organizing UC field
14
UbiquitousComputing P-3.1 - S.C.A.L.E. Scheme
Major UC challenges, on a very high level: S – SCALABILITY
- how to support cooperation of “zillions” of components?- how to support nomadic users around the globe?
C – CONNECTIVITY- how to “easily” connect these zillions? Several levels of abstraction:
- wireless networks – a blessing and a curse (unreliable!)- most issues come above wired/wireless net (note: overlap with scalability):
- how to find/understand your peers? How to enable zero configuration- how to design networks for zillions2 of connections, without central-server bottleneck
A – ADAPTABILITY- usage during daily work, surrounded by 100s of components: need minimal interaction- major approach: context-aware computing – use it to automate tasks & reduce options- why “adaptability”? adapting-to-particular-user (user modeling) focussed beyond context-
awareness L- LIABILITY
- term indicates: we must go beyond today’s IT security solutions (not goals), since- today’s solutions do not scale (centralized components?), are not “humane” (see below), …- … & don’t flexibly consider conflicting (privacy, traceability) & related goals (dependability etc.)
E – EASE-OF-USE- adaptability permits “minimal” …, ease-of-use means “optimal” interaction (related!)- issue: optimal use & combination of modalities, advancement of specific modalities- issue: “understanding” natural input: a) xxx-to-text; b) text-to-meaning; “intelligence”?
15
UbiquitousComputing P-3.1 S.C.A.L.E - Scalability
S – SCALABILITY(is a “top priority” challenge reflected in acronym: scale) A. Mathematical Scalability (in the mathematical sense)
UC leads to (potential) cooperation of “zillions” of devices- thus, solutions need to work efficiently with zillions of components- comes to mind: complexity of the made the born (K. Kelly!) learn from the born
most relevant areas (basically, alternatives for addressing technical scalability):1. bionics i.e. bio-analog computing
- neural networks, cooperating robots: huge fields, only marginal importance for UC- ant colonies – often simulated / executed on single computer today; swarms, autonomous computing,
…; - what else? plants as examples? brain-like modeling human memory: example given in main chapter
“S”2. socionics: learning from complex societies … is still in its infancies wrt. UC3. Future communication/cooperation (pub/sub; P2P, Grid) – see further below (C –
connectivity)B. Technical-Economical Scalability (in the sense of global interoperability)
- human-attached encountered components need to cooperate globally- standards come to mind, but global standards are not sufficient & (often) not realistic:
e.g., interface definition languages (cf. remote method/procedure call etc.):- they define syntax (types/name of operations/operands exported @ interface- they usually don’t define, e.g.: valid call sequences, preconditions, cost/quality etc. (approaches exist)- they don’t define semantics: what does an operation actually do? approach??: ontologies & service discovery
most relevant areas (to be combined into an overall solution):1. WebServices and Business Processes2. Ontologies a holy grail for making services understand each other (cf. what does the op’
do) ?3. Service Discovery that should take into account SLA (service level agreements), QoS
(quality of Service – in the business sense of “get what you pay for”), and more 16
UbiquitousComputing P-3.1 S.C.A.L.E: Connectivity
Global interconnection of UC components is related to the following(and to “scalability” above):A. Scaleable Communication
1. wireless networks: (often!) a prerequisite for higher layers needbasic understanding of technology [ZigBee … WiMax] and beyond
2. event based communication: praised as the UC connectivity approach• means “push” paradigm, a prerequisite for scaleable open cooperation of
components (supersedes client/server!); note relation to “S”• maybe one day superseded by bionic/socionic approach; today: best offer• remaining problems (advertising, openness, integration of other paradigms)• plus: what else is needed in UC middleware (services? tools? which ones?)
B. Scaleable Cooperation1. Overlay Nets: special subnetworks in Internet; at least 3 classes:
a. Peer-to-Peer Networks: no centralized bottleneck, scale well?b. Opportunistic Networks: ad hoc net (node proximity) parallels global human “network”c. Grid Networks: dynamically available resources parallel fluctuating resource demandsd. Smart Item Networks: AutoID and beyond: towards scaleable networks of “machines”
2. Service Discovery: prerequisite for zero configuration• component environment, makes itself known & learns about environment• sounds like solution to “economic scalability” above, but isn’t: many
assumptionsabout components are made rather a connectivity issue
17
UbiquitousComputing P-3.1 S.C.A.L.E: Adaptability
A. Context Awareness (adaptation to “situation of use”)• sensed context: what sensors can measure (temP-, shock, location, …)• modeled context: info held in “other” software/DB: tasks & activities etc.• inferred context: built from (several?) sensed or modeled contexts, e.g.,
GPSstreet … co-located chemicals dangerous!, is always modeled• note: context “ages”, is “probabilistic”/maybe contradictory
(sensor imprecise? calender entry gives different location than GPS sensor?)
• most investigated context: location• may be absolute (“X is room 253”) or relative (“voltmeter is with Mr. X”)• absolute: “2x2” technologies to survive? outdoor GPS & cell?; indoor UWB & IR?• relative: technologies for tags (RFID, RF+, IR, cell, …) will coexist?
B. User Awareness (adapt to user(s) - and, mid term, provider?)• technology? “usual suspects”: user models, profiles&preferences, user
agents• great challenge: the huge crowd of “new” users
• unexperienced, hands/eyes free, little attention• understand their actions support them! [TK]
• for UC: ubiquitous user models? UC components contribute&use only fractions
18
UbiquitousComputing P-3.1 S.C.A.L.E: Liability
Liability :== protection of actors and those concerned by actions (peers, third parties, society) through “right” mix of protection/prosecution – all in presence of zillions of peers; liability = “security and beyond”
A. Scaleable security: old goals (privacy, authentication, …), but new aspects:• machine2machine communication, ad hoc “encounters”: PKI not viable
• cannot check certificate chain with (central!) roots zillions of times per second!• even if so: a party may change (e.g., due to virus) zilliseconds after check• maybe: not always/reliably connected! • early approaches: resurrecting duckling++, TCI, … : viable overall solution?
• there ain’t no end2end encryption (cf. https-connection vs. “frontend” password-spyware)B. Anthropomorphic liability: UC in everyday life conforms with human intuition?
1. intuitive models: “human-understandeable” models of trust, risk, recommendation etc. 2. user friendliness: security measures accepted observed by users?
• novel concepts, e.g.: minimal trusted, concealed entity (“ME”, see below)?3. “Social” compliance: conflicting goals conflicting forces (the core of liability!!)
• e.g., privacy often traded off with public interest, peer’s interest (cf. traceability, non repudiation)• required: max. flexibility, “lever” for tradeoff (different place/time/parties: change mix)
C. Collateral goalsmost important: dependability / reliability / correctness • above terms: overlapping definitions; security added “trustworthiness”• classical dependability: possible in light of zillions of bugs of MS Windows alone?1. Aspect Trusted Computing (TCI): no a priori trust in peers rely on TCI? (doubts!)
• idea: virtual machine layers “underneath my SW” (CPU, Bios, OS, VM) “frozen” via digital signature2. Aspect Accounting & Contracts
• user: I will get what I pay for; provider: I will get paid for what I provide19
UbiquitousComputing P-3.1 S.C.A.L.E: Ease-of-Use
remember: UI-adaptability may be considered a subset of ease-of-use (UI reducted to “what makes sense”)issue: optimal use & combination of modalities, advancement of specific modalities
A. Consider variety of I/O devices in UC world “multimodal interaction”simple distinction: hands&eyes- vs. mouth&ears-interaction 1. advanced hands&eyes interaction: GUIs predominant, but further developments needed
• examples: focus+context-displays, 3rd dimension (VR), 4th dimension (dynamic displays), immersion, narration2. mouth&ears interaction: voice underdeveloped today, great potential! nomadic (hands-/eyes free) operation
• but: needs speech understanding, is transient difficult3. integration of HCI and SWE: the grand challenge!
• today: HCI before & after SWE – but “incompatible”4. multimodality … : “beyond transcoding (program output XML, XSLT produces device-specific XML variant)”
• quest for “abstract interaction layer”: shields modalitity details from “core” program yet leverages off modality advantages• true multimodality: parallel use of modalities, lots of freedom “fusion” of inputs? “fission” of outputs?• integration with other advanced UIs? (“context-sensitive, adapting/proposing to user [AI techniques?])
… & federated devices: optimal use of multiple I/O devices currently surrounding user (includes “liability” issues)
B. Approach human “intuition”: new post-desktop metaphors – a cross cutting concern• post-desktop metaphors: cross cutting wrt. all aspects above
• example: today, eMail folders directories …; tomorrow: all app’s adapt MY favorite structure?• methaphors invented for graphical UIs approproate ones for voice, etc.?
• … and related to scalability: humans must understand scalable = probabilistic behaviour in UC as inherent characteristic• example mass data (cf. DigiCam recordings swamping user disks)• options today:reduce/delete or: classify or: accept chaos• tomorrow: model brain classify / rank automatically! ( prioritize/summarize/reduce; even: forget i.e. delete?)
• Approach ease of human-to-human dialoguesA. natural dialogue: natural language processing NLP intelligent user interfaces
• grammar and semantics – possible for limited domain / world knowledge? simple/full grammar? beyond phrases: understand dialog?
• formal knowledge: structured, semantically annotated documents <-> “informal” knowledge (eMails, Forums, WebSites, Blogs…)
• even worse: electronic communication / informal knowledge ruled by abbreviated expression1. understanding “natural” input
• means: handwriting, speech, gesture, mimics (emotion!, “true” meaning) --- to be combined with NLP 20
UbiquitousComputing
P-3.2 Relation to Computer Science Research
21
S.C.A.L.E.
scalab-ility
connect-ivity
adaptab-ility
liab-ility
ease-of-use
DistributedSystems
IT Security
HCI
ComputerNetworks
Software Engineering
Bionics
Socionics
context awarecomputing
usermodelling
AI (/NLP/KM)
Dependability
Quality-of-Service,SLA, Accounting
Dependability
UbiquitousComputing
P-4.1 SCALE Taxonomy As Introduced
22
S.C.A.L.E.
scaleab-ility
connect-ivity
scaleablecommun-
ication
scaleablecooperation
tech/math
econ-omic
bionics
socionics
wirelessnetworks
pub/sub,eventComm
pub/sub,eventComm
adaptab-ility
liab-ility
ease-of-use
WebSer-vices
Ontologies
Discovery
ServiceDiscovery
contextawareness
userawareness
scaleablesecurity
antropo-morphicliability
collateralgoals
multimodalinteraction
postdesktop
metaphors
humanedialogue
OverlayNetworks
locationawareness
usermodeling
Peer2Peer Grids Opportun-istic Nets
SmartItemsNets
trustedcomputing
accoutning& contracts
intuitivemodels
user friend-ly security
socialcompliance
advancedhands&eyesinteraction
mouth&earinteraction
integrationwith Soft-ware Eng.
multimodal& federatedinteraction
intelligentuser
interfaces
ambientlearning
cooperativeinteraction
under-standingnaturalinput
UbiquitousComputing P-4.1 SCALE Taxonomy vs. Chapters
23
S.C.A.L.E.
scaleab-ility
connect-ivity
scaleablecommun-
ication
scaleablecooperation
tech/math
econ-omic
bionics
socionics
wirelessnetworks
pub/sub,eventComm
pub/sub,eventComm
adaptab-ility
liab-ility
ease-of-use
Ontologies
WebServices
Discovery
ServiceDiscovery
contextawareness
userawareness
scaleablesecurity
antropo-morphicliability
collateralgoals
multimodalinteraction
postdesktop
metaphors
humanedialogue
OverlayNetworks
locationawareness
usermodeling
Peer2Peer Grids Opportun-istic Nets
SmartItemsNets
trustedcomputing
accoutning& contracts
intuitivemodels
socialcompliance
user friend-ly security
advancedhands&eyesinteraction
mouth&earinteraction
integrationwith Soft-ware Eng.
multimodal& federatedinteraction
intelligentuser
interfaces
ambientlearning
cooperativeinteraction
under-standingnaturalinput
2
3
4 5
6
7
8
9
10
11 12 13
14
15
16 17
18
19
20
21
22
23
24
25
1
26
27
28
UbiquitousComputing P-4.2 Reader’s Digest
24
S.C.A.L.E.
scaleab-ility
connect-ivity
scaleablecommun-
ication
scaleablecooperation
tech/math
econ-omic
bionics
socionics
wirelessnetworks
pub/sub,eventComm
pub/sub,eventComm
adaptab-ility
liab-ility
ease-of-use
Ontologies
WebServices
Discovery
ServiceDiscovery
contextawareness
userawareness
scaleablesecurity
antropo-morphicliability
collateralgoals
multimodalinteraction
postdesktop
metaphors
humanedialogue
OverlayNetworks
locationawareness
usermodeling
Peer2Peer Grids Opportun-istic Nets
SmartItemsNets
trustedcomputing
accoutning& contracts
intuitivemodels
socialcompliance
user friend-ly security
advancedhands&eyesinteraction
mouth&earinteraction
integrationwith Soft-ware Eng.
multimodal& federatedinteraction
intelligentuser
interfaces
ambientlearning
cooperativeinteraction
under-standingnaturalinput
2
3
4 5
6
7
8
9
10
11 12 13
14
15
16 17
18
19
20
21
22
23
24
25
1
26
27
28
IntroductoryChapters: 1,2,3,7,14,22Sufficient for anOverview of UC; expertsmight want to leave out“their” intro chapter
Readers who want to focus onDistributed Systems: read introchapters + 8-13, add 15-17 and5+6 according to interests
For Ubiquitous Security,read intro chapters andchapters 19-21
For HCI aspects,read intro chapters andchapters 23-28, plus 17,15-16, 4-6 according to interests
Practitioners: read chapters “SAP pilots” chapters at the end of the book