Nsf iii events and experiences 100422

© Ramesh Jain NSF III Workshop April 23, 2010

Ramesh Jain

Department of Computer Science University of California, Irvine

[email protected]

Events and Experiences


Data Data Everywhere but Not a relevant byte of

Information/Knowledge/Wisdom

to help.

Too many silos with too much data = CONFUSION


Inventing the Future by

Learning from the Past


What is most Important in inventing the future?

  Imagination is more important than knowledge. For knowledge is limited to all we now know and understand, while imagination embraces the entire world, and all there ever will be to know and understand. Albert Einstein


The only source of knowledge is experience.


Mechanics: Statics and Dynamics

Statics is the branch of mechanics concerned with the analysis of loads (force, torque/moment) on physical systems in static equilibrium, that is, in a state where the relative positions of subsystems do not vary over time, or where components and structures are at a constant velocity. The study of moving bodies is known as dynamics, and in fact the entire field of statics is a special case of dynamics. From Wikipedia

Issac Newton 1643- 1727 1687


Newton’s Laws   Law 1: Every object in a state of uniform

motion tends to remain in that state of motion unless an external force is applied to it.

  Law 2: The relationship between an object's mass m, its acceleration a, and the applied force F is F = ma.

  Law 3: For every action there is an equal and opposite reaction.

http://teachertech.rice.edu/Participants/louviere/Newton/


The role of these laws


  Newton: Mechanics   Einstein: Theory of relativity.   Babbage: Digital Computer.   Tesla: AC   Gutenberg: Moveable print   Edison: Electric Bulb   . . .

What is the most important invention in the last 1000

years?


Life is a set of experiences.


Experiences Life =

Events

+


The Story of Human Civilization is the Story of Experiential

Communication among Humans.

Sharing Experiences Recording Experiences


Share experiences : Languages


Across Time : Writing


Making it portable : Paper


Democratization : Gutenberg’s Movable Press


Telegraph: Instantaneous Remote


Telephone: Remote Speech


Photography: Recording Visual Experience


Audio Recording: Electrical Signals enter recording space


Radio: Live Broadcast


Television: Live Broadcast combines two Media


Digital Processing


Internet: The Game Changer

Medium is the message. – McLuhan was right THEN. Medium is just that – Medium. NOW


Communicating Experiences: Spatial and Temporal

Inventions Application Impact

Languages Communicate symbolic experiences

Written Languages Record Symbolic experiences (time)

Paper Portability (space)

Print Mass distribution (time and space)

Telegraph Remote narrow communication (space)

Telephone Remote analog communication (space)

Radio Analog broadcasting of sound (space)

Television Combining two senses – media (space)

Recording media Photos, audio, video (time)

Digital processing Machine enhancement and processing

Internet Interactive Multimedia communication

Most influential Invention in history

Future of Communication


Why is Gutenberg’s Moveable Print so influential?

  Brought knowledge to masses: creation, dissemination, and access .

  Resulted in increased awareness of scientific approaches.

  Reduced misinformation and influence of religions.


History: Gopher to Google

  We had Internet.   Lots of computers were connected to each other.   Computers had files on them.   We had GOPHER and other FTP mechanisms.


Tim Berners-Lee thought:

  Suppose all the information stored on computers everywhere were linked.

  Suppose I could program my computer to create a space in which anything could be linked to anything.

Others – including Bush -- had that idea earlier but the technology was not ready.


That gave us WWW, Google, …

  Search   Maps   Instantaneous access to immense amount of

knowledge in documents


Continuing On

  Flickr   YouTube   Facebook   Twitter

The 4 most popular companies that emerged in the last 4 years.


And the 4 most popular Internet concepts in the last 4 years.

  Social Networks   Micro-blogging (Ambient Awareness)   Tags   Real Time Search

Surprisingly, they did not come from Academia or GYM.

Why???


Interesting!!!

  Social Networks   Microblogging   Tags   Real Time Search

  Flickr   YouTube   Facebook   Twitter

What does this tell us?


Messages are clear

  New media: Text based media is not enough.   Medium is NOT the message: medium is just

the medium.   Experiences: People want to experience and

share experiences – with minimal latency.   Socialize: Family and friends remain a strong

influence in all facets of life.   Family and friends are closer to each other today

than ever!!!


Bits and Bytes

Alphanumeric Characters

Lists, Arrays, Documents, Images …

Transforma)ons

Crossing the Structure Chasm: Semantic Gap


Computer Science: Modeling the World

  Data   Objects   Events

Both Objects and Events are essential to model the real world. Remember mechanics?


Events

  Take place in the real world.   Captured using different sensory

mechanism.   Each sensor captures only a limited aspect of the

event.   Are used to understand a Situation.


Event Representation

EXPERIENTIAL


Events: Different types and Granularities

  Conferences   Days

  Sessions   Talks

  Purpose of the talk   Clicking for the next slide-event

  Wedding   An Earthquake   The Big Bang   9/11   Formation of Google   Attending WWW   Me

  My Birth,   Being here, and   Dying in 100 years.


Tim

e

1- dimensional Space Events Events Everywhere, …


Butterfly effect:

Does the flap of a butterfly’s wings in Brazil set off a hurricane in Florida?


Continuing the Evolution of the Web

  Consider a Web in which each node   Is an event   Has informational as well as experiential data   Is connected to other nodes using

  Referential links   Structural links   Relational links   Causal links

  Explicit links can be created by anybody   This EventWeb is connected to other Webs.


Tim

e

1- dimensional Space EventWeb


Stages in EventWeb  Cyberspace: Every event is ‘created’ in

cyberspace and exists there.   Calendars, Status updates, Twitter,

 Physical and Cyberspace: Events in real space are also linked. Sentient EventWeb linked with the Web.   Foursquare, Geomapping, my camera, …


Sensors are the Bridge

  Our 5 Senses are connectors to the world.   We use our sensors (vision, audio, …) to

experience the world.   Sensors are placed for ‘detecting events’.

  Would you put a sensor if nothing interesting ever happens at a place?

  Sensors are the bridge connecting the Cyberspace and the Real World.


Creating a Sentient EventWeb   Imagine Sensors Tweeting – in addition to

human sensors.   All video cameras   Your refrigerator   Each Phone   Many more …

  Infrastructure to create and link all such events.   Filter – in Real Time   Detect Situations   Create proper Situation Indexes   Create alerts for relevant recipients.


Social Pixels

  Traditional Pixels   Photons aggregating at locations

  Social Pixels   User interest at geo-locations

  Create social Image, social Video…   Image Processing operators Situation

Detection operators


Spatio-temporal aggregation using social pixels

  Intuitive visualization   Dashboards

  Effective representation for spatio-temporal data   Acts as common format for representing any geo-spatial

data e.g. weather, demographics, laws, interest levels …   Applicability of media processing inspired tools

e.g. Convolution   Image processing inspired, but semantics are geo-

spatial


Architecture

Situation Detection Engine


Micro-blogs: Twitter

  Each Tweet:   Used to be: What are you doing   Now it is: What’s happening

  Really an event reported by human sensors   Can associate experiential data along with

information.   Time and location associated with each

Tweet I consider Twitter as a phenomena – have no investment in the company. Also, this concept can be further extended to more interesting cases.


iPhone

Expression of interest in iPhone release.


Operators


Aggregation of operators


Swine flu social image and operation on it (segmentation)


Situation awareness: Swine Flu


Recommendation System: Swine Flu


Network/ Transmis

sion

Environment Model

Environment Server

Situation Detection and

Control

Actuator / Presentation

Model

MMDB

Sensors

Actuators / Presentation

Devices

Physical Environment

EventBase

From Micro-events to Environments and Situations


Env. 1

Env. 4

JSM 2

JSM 1

Env. 5 Env. 3

Env. 2 Connecting Environments


Extending Environments

TCS mKrishi: If experts can not go to a farm; lets bring the farm to experts.


Sensors in the field

© Ramesh Jain Irvine TLG October 30, 2009 60

Query No. 276 In text: I think because of salt in water, the leaves are getting damages. I am sending pictures of leaves. Please advise.

Expert’s Advise

Meaning: Please use compost in the soil in large quantity for next two years


Expert Console

Voice query

Picture

Info specific to crop

From public website

Ready references for expert

Sensor data – if applicable


Building an ‘AVIARY’

  Humans Tweeting   Sensors Tweeting   Environments detecting Situations and

reporting   Aggregating and analyzing   Indexing   Accessing the Web

  For every day use.


Goal: Person-context-sensitive just-in-time Information

  Data from various sensors, including human sensors, to understand situations.   Sensor Networks (including Billions of cameras)   Social Networks

  Building (evolving) personal profile from social graph and status updates for filtering immense amount of data.

  Crowd-source knowledge to improve social and environmental conditions


Thanks for your time and attention.

For questions: [email protected]

Nsf iii events and experiences 100422

Documents

Transcript of Nsf iii events and experiences 100422