SixthSense:RFID-based Enterprise

Intelligence

Lenin Ravindranath, Venkat Padmanabhan (MSR India)

Piyush Agrawal (IIT Kanpur)

RFID Radio Frequency Identification Components

RFID Reader with Antennas Tags (Active and Passive)

Electromagnetic waves induce current Tag responds

Globally unique ID Data

UHF (865-956 MHz) Range – up to 7m

Applications Tracking, Inventory, Supply Chain, Authentication, … Novel Research Applications

Motivating Scenario

Lenin missed an object in the

conference room – 2nd floor Scientia

SixthSense Goal Making people, objects and workspaces, the first class

citizens of the enterprise computing

Components Use RFID to capture the rich interaction between

people and their surroundings Combine with other enterprise systems/sensors to

make automated inferences Enable Useful Services

Setting

People and objects tagged

Camera

Calendar

Presence

RFID Antennas

Assumptions Widespread coverage of RFID readers in the workspace Users are free to pick up new tags and affix them to

objects Can put multiple tags on an object

No dependence on cataloging Cataloging is an overhead

TagID Entity Antenna Workspace Error Prone Tags are fragile – may have to be replaced Readers/Antenna could be moved

Start with an undifferentiated mass of tags and infer everything

Lenin missed an object in the

conference room – 2nd floor Scientia

13548234 – Ant 115574523 – Ant 113548234 – Ant 615574523 – Ant 1

Architecture

People and objects tagged

Camera

Calendar

Presence

RFID Antennas

SixthSense

SixthSense

SixthSense

Automated Inference Platform

Programming Model Applications

Inference Engine Person-Object Differentiation Object Ownership Inference Zone Identification Person Identification Person-Object Interaction

Person-Object Differentiation

People can move on their own Objects move only when carried by a person

Co-movement based heuristic Relative Movement (RM)

Zone 1 Zone 2 Zone 3

Object Ownership Inference Co-Presence

Calculate the amount of time the object is concurrently present in the same zone as a person

Owner is the person with which the object is co-present the most and greater than a threshold

Person Identification

1

xyz abc

1

2

Workspace Entrance

Event

Log-in event

Coincidence count

1xyz

1

time

t1

t2

1

Object Interaction (only in zones of interest) Intra zone Identify interaction in zones of interest

A person lifted an object A person turned the orientation an object

Signal Strength of tag varies Change in distance Change in orientation Contact

Monitor variation in RSSI

Object Interaction Sample the RSSI of each object tag every 200ms Sliding 4-second wide window

Difference between the 10th percentile and 90th percentile of the RSSI

Object is said to be interacted - If the difference > threshold

Minimizing spurious detections Use multiple antennas

Object Interaction

Antenna 1

Antenna 2

Interacted

Interacted

Ensuring Privacy Enterprise will deploy and manage the system

Expose appropriate set of information Trust - Analogous to the enterprise e-mail system

Defend against rogue readers Relabeling approach [A. Juels, 2006]

EPC code rewritten at random times SixthSense will be aware of the mapping

between the old and new tag IDs

Implementation

Simulator(Trace Generation)

Experimental Setup(Real-time feed)

Experimental Setup

Results Inter-zone movement detection

Object Reliability (1m) Reliability (2m)Badge on belt clip 100% 96%Small box in hand 94% 88%

Object Interaction

Testbed deployment To make correct inferences

Average inter-zone movements needed – 4 Average log-ins required - 3

Distance between antennas

Detection time

1.5m 2.39s2m 3.4s

2.5m 5.03s

Simulation Probabilistic model to generate artificial traces Simulated

Inter-zone movement (walk) People carrying multiple objects Log-in events Untagged people

Zone 1 Zone 2 Zone 3

Results Person-object differentiation Person Identification Varying average walk length Effects of untagged people

Person-Object differentiation and ownership

20 people, 100 tags, probability of walk – 0.1, walk length - 5

Person Identification

10% of users entering workspace simultaneously

Programming Model Callbacks

InterZoneMovementEvent (tagID, startZone, endZone, Time)

ObjectInteractedEvent(tadID, Zone, Time) Lookups

GetTagList() GetPersonTags() GetOwnedObjects(tagID) GetTagType(tagID) GetTagOwner(tagID) GetPersonTagIdentity(tagID) GetZoneType(zone) GetTagsInZone(zone) GetTagWorkspaceZone(tagID) GetCurrentTagZone(tagID) GetCalendarEntry(ID, Time)

Example Misplaced Object Alert

personTags = GetPersonTags()For each ownerTagID in personTags

ObjTags = GetOwnedObject(ownerTagID)OwnerZone = GetCurrentTagZone(ownerTagId)OwnerWorkspace = GetTagWorkspaceZone(ownerTagId)For each obj in ObjTags

objZone = GetCurrentTagZone(ownerTagId)if (objZone != OwnerZone && objZone !=

OwnerWorkspace)Raise Alert

Applications Annotated video Semi-automated image catalog Misplaced object alert Automatic conference room booking

Annotating video with physical events Events

Inter-zone movements Object Interaction

Tag video feed with events Person X interacted an object Y

Rich video database Support rich queries

Give me all videos where Person A interacted with Object B

Application: Integrate with enterprise security camera system

Semi-automated Image catalog TagIDs are not user friendly Catalog tagID with its Image

User picks up an object Shows before the camera and takes a photo Automatic cataloging (TagID, Image)

Annotating video with physical events

Related Work Localization

LANDMARC Indoor Location Sensing Using Active RFID

Sherlock (UMass) Automatically locating objects for humans

Ferret (UMass) RFID Localization for Pervasive Multimedia

Platform Cascadia (UWashington)

Specifying, detecting and managing RFID events

Object Interaction I sense a disturbance in the force (Intel Research, Seattle)

Unobtrusive detection of Interactions with RFID-tagged Objects

With other sensors Fusion of RFID and Computer Vision (MSR)

Summary SixthSense

Enterprise Setting People and Objects tagged RFID with other enterprise sensors

Components Automated Inference Platform Applications

http://research.microsoft.com/research/mns/projects/SixthSense/

Questions?

Backup

Semi-Automated Image Catalog TagID-Image Cataloging

User picks up a tagged object Hold it in front of the camera Clicks a picture Automatically identify the tagID of the object

SixthSense SystemInference Engine, Database, Applications

RFID Reader

RFID Antennas

Calendar Data Presence Data

ApplicationsQueries

Industry Tracking, Inventory, Supply Chain, Authentication

Research Measurements Improving reliability, security Localization RFID + Computer Vision Interaction detection RFID + other sensors

RFID Applications

Person-Object Differentiation People can move on their own Objects move only when carried by a person

Co-movement based heuristic For every tag T, find co-movement tag set {T1, T2..Tn}

m – total inter-zone movement of T mi – total inter-zone movement of Ti ci – amount of co-movement exhibited by Ti with T

Declare the tag with the highest RM as person Eliminate this tags movements Repeat the algorithm till RM is positive Tags with negative RM are objects

Zone Identification Individual workspace

If there is one person predominantly present in a zone Workspace of that person

Shared workspace If no one person is predominantly present in a zone Length of time from a person entry to exit < threshold

Reserved shared workspace Length of time people are present > threshold Common meeting entries in their calendars

Common areas Any space that is not classified as one of the above

Challenges – Improving Reliability Multi-tagging scheme

Affix multiple tags in different orientation Increases the probability that atleast one of the tags being

detected

Automatic Inference Initially assume all tags belong to one giant super object

Fully connected graph When two tags are detected simultaneously in different

zones Tags belong to different objects Delete edges between them

Connected components Set of tags attached to the same object

Evaluation – with untagged people

Automatic Conference Room Booking Conference Room Zone is automatically

identified Reserved Space

Automatically book conference room If it is not reserved And bunch of people go into the conference room And spend say 5 minutes

Discussion Privacy

Deployed and managed by enterprise Limited access to users Relabeling approach

Economic Feasibility Passive Tags are cheap Prices are RFID readers expected to drop (Intel

R1000) Health Implications

Transmitted RF power (up to 2W) is well within safe limits

this question will undoubtedly continue to receive much attention and study