Iit kgp workshop

Experience certainty.

Copyright © 2011 Tata Consultancy Services Limited

Human-in-loop Cyber-physical Systems- Intelligent Infrastructure using Internet-of-Things

Arpan PalHead of ResearchInnovation Lab, Kolkata

Research Theme

3Experience certainty.

Human-in-Loop Cyber-physical Systems

Humans

Physical

Objects and

Infrastructure

Computing

Infrastructure

Physical

Context

Discovery


Signal

Processing

Internet-of-Things - towards Intelligent Infrastructure

Sense

Extract

Analyze

Respond

Learn

Monitor

Intelligent

Infra@Home

@Building

@Vehicle@Utility

@Mobile

@Store

@Road

“Intelligent” (Cyber) “Infrastructure” (Physical)

APPLICATION SERVICES

BACK-END PLATFORM

INTERNET

GATEWAY

Sense

Extract

Analyze

Respond

Communication

Computing

5

Challenges and Solution Approach

Systems• Making Sensing Unobtrusive and Pervasive• Sensing the Human Context

Cloud-based Sensor Data Analytics Platform Use of Mobile phones and Cameras as sensing devices New Sensors for human context – EEG, 5 senses computing

Signal Processing• Preserving the Battery power of edge devices• Extracting information from ambient-noise-corrupted sensor data

Low-power computing algorithms Adaptive Signal Processing for Ambient Noise Removal

Communication• Reducing the cost of Communication while preserving reliability and security• Preserving the Privacy

Secure, Lightweight yet Reliable communication protocols over Internet Quantifying Privacy vs. Utility measures

Computing• Knowledge and Model Driven Analytics in Real-time

Distributed Computing - Bringing Edge into the Grid

Semantics driven Analytics - Reasoning and Semantic Sensor Web

Model driven Analytics - Cyber-physical system modeling / Human behavior modeling

6

Integrated Platform for Intelligent Infrastructure

People Feedback & Emotions

Social Media

Integrated Services

Sensors & IoTPlatform

Traditional Monitoring & Control Systems

Citizen Data

Smart Integration Platform

Transportation Healthcare Electricity

WaterPublic Safety Tourism

Smart Integrated Services

Sense

Analyze

Extract

Respond

Intelligence

Smart Domain Services

Community

etc.

Sense: People Activity, Appliances, Vehicles , Road, Home/Bldg, Utility Infrastructure

Detect gas leakage/water contamination : mobilize rescue team, suggest optimum route

Divert Road Traffic in case of Water Pipeline Burst

Correlate Electricity/Water /Gas consumption patterns

Intelligent Integration Platform

Integrated Intelligent Services

RIPSAC – Real-time Integrated Platform Services & Analytics for Cyber-physical Systems

7

Platform Architectural Layers

Internet

End Users &

Renderers

Administrators

Device Integration & Management Services

Analytics Services

Application Services

Storage

Messaging & Event Distribution Services

Ap

plic

atio

n S

erv

ice

s

Presentation Services

Application Support Services

OS & Device Drivers

Sensor Signal Processing

Middleware

Mid

dle

ware

(S

ecurity

/Privacy a

nd P

roto

cols

)

Edge Gateway

Sensors

Internet

Back-end


Cyber Physical System – Research Theme

RIPSAC

Infrastructure

Simulation

New Sensing

Platforms,

5 Senses

Computing

Energy

Water

Transportation

Security

& Privacy

Distributed

Computing

Facilities

Data Integration

& Visualization

Safety

Buildings

HomeCity

Vehicles

Non revenue

water Long term network

expansion

Renewable

sources

Energy

conservations

Intelligent

Infrastructures

Sensor Informatics

for Home

Sensor Informatics for

Vehicular Monitoring

Transport planning

Electricity Grid

Optimization

algorithms

Big data

Communications &

Sensor Informatics


Human-in-Loop Cyber-physical System – Research Theme

RIPSAC

2D/3d Surveillance -

Kinect

Physiological

Sensing –

ECG, EEG, …

Physiology

Identity

Smartphone

Sensors –

GPS,

Accelerometer,

Magentometer,

Gyropscope,

Camera, Mic

Healthcare• Elderly People Monitoring• Chronic Disease People

Monitoring

Personalized learning• Cognitive Load

Web and

Social Media

Smart

Meters

Wellness• Fitness• Stress, Fatigue

Organizational Behavior• Personal Physical Network

Discovery• Team Efficiency / Best Practice

Analysis

Retail• Customer

Behavior Study

Driving• Stress• Fatigue

Research Projects – Ubiquitous Healthcare

11

Intelligent Healthcare – Elderly / Chronic Patient Monitoring

ECG

Blood PressureMonitor

Pulse OxyMeter

Mobile phone as medical

gateway

PatientRecords

Health Center / Home

Expert Doctor

• 2 papers in SMART 2012 and MobiHoc 2012

• Requirement from NUH (Peritonial Dialysis) and

SingHealth (Elderly Care)

12

Intelligent Healthcare – Elderly / Chronic Patient Monitoring

Mobile phone as medical

Sensor

PatientRecords

Health Center / Home

Expert Doctor

• Replace Medical Sensors with Mobile Phone Sensing

• Activity and Localization using Mobile Phones

• Activity and Identification using Kinect Camera

• Social Media as a Soft sensor

• Multimodal Fusion

Identification, Localization and

Activity Detection

• 1 paper in UbiComp 2013

13

PPG based Pulse Measurement using Phone Camera

Subject1 Subject2 Subject3

Actual Detected Actual Detected Actual Detected

68 66 66 63 85 84

2.9% 4.5% 1.1%

Subject Actual Pulse

Rate

Detected Pulse Rate (20 sec

skip)

Detected Pulse Rate (automatic

detection)

PSNR (automatic

detection)

A 54 53.542 53.542 22.887

B 66 63.404 63.404 15.351

C 84 84.539 84.539 19.831

D 106 98.629 98.629 21.741

E 80 88.062 88.062 14.788

F 105 102.15 102.15 16.705

G 105 38.747 0 0

H 80 42.27 0 0

• 1 paper in Mobihoc 2013

• 1 patent filed TCS Fit4LIfe Pilot

14

Blood Pressure and ECG Monitoring from PPG

Extract PPG

Features

BP Ground

TruthCreate BP Model

Extract PPG

Features

Predict BP levels

(SP, DP, PP)

Extract ECG

Features

Create ECG

Parameters’ Model

Predict ECG

ParametersECG Parameters

BP levels

(SP, DP, PP)

Training Phase

Testing Phase

Data set Pd Ps PP-diff < 15

Standard dataset with 14 PPG features 92.9% 74.7% 77.9%

TCS recorded dataset with 14 PPG features

+ height, weight, age

99.3% 82.7% 85.5%

BP Level Pd Ps

Very Low < 50 < 70

Low 50-65 70-100

Normal 65-90 100-

135

High 90-100 135-

160

Very High > 100 >160

15

Mobile Phone based Activity Detection and Localization

Activity Detection– Uses Accelerometer Data

– Gyroscope and Magnetometer for orientation

correction

– Step Count, Stride Length Estimation

– Walking, Brisk Walking, Running Classification

Indoor Localization– Initial Referencing through GPS and

Magnetometer

– Inertial Navigation through accelerometer,

gyro, magnetometer

– Improved accuracy through Stride

Length, Kalman Filter / Particle Filter

– Tracking of non-smartphones via

Bluetooth

Location ID LOC I LOC II LOC III

Actual (ft) 2 4 6

Estimated

(ft)1.95 4.16 6.27

% Error 2.5 4.12 4.4


• 3 patents filedActivity Detection for TCS Fit4Life

Generalized indoor localization – Colleague Finder,

Retail, Emergency Evacuation

16

Kinect Based Identification and Activity Detection

Human Identification

– Skeleton Model Based / Depth based

– External Stimulus based refinement

– Accuracy of 90% for less than 10 people

database

• 2D Camera with IR

depth sensor

• Excitation by IR light

pattern

• Directional Mic.

Activity Detection

– Enter/leave a room

– Sitting / Standing / Walking / Lying down

– Going to Bathroom

– Taking MedicineLIRIS human activities dataset

• 4 papers in Fuzz IEEE 2013 , CEC 2013, UbiComp 2013, SMC 2013

• 2 patents filed

17

Social Media as a Soft-sensor for Healthcare

Support Community Discovery

– Use hidden community detection by applying NLP on

posts to create the social graph to identify the

undeclared community for a given disease

Disease onset discovery like dementia or Alzheimer's

disease or psychological disorders from social

network posts

– Search for patterns in posts to detect possible

symptoms to diseases

– E.g. - sentiment analysis on posts will give whether the

given post’s emotion is positive or negative. If the

emotions are cycling between positive and negative

extremes with some periodicity, probably the person

has bipolar disorder


Research Projects – New Sensing Techniques


Bio-sensing for Cognitive Load

User Study

Biased and Indirect

Biological Response using EEG

Unbiased & more reliable method

• 2 papers in Fuzz IEEE 2013 , SMC 2013

• 2 patents filed

• Can be used for Usability Testing of GUI


5 senses Computing - 3D Reconstruction with 2D images from mobiles

• Low cost solution for 3D reconstruction from multiple 2D images captured from

mobile device.

• Derive the motion information from the inbuilt sensors of the mobile phone and then

aid in increasing the accuracy of the 3D reconstruction.

• Dynamic 3D reconstruction in outdoor environment which is not possible using a

Kinect.

• 3D reconstruction of homogeneous objects

• Future research focus on the multi-modal fusion of the 3D information and the other

sensors’ data for a given object or environment

Applications

• Agro-advisory Service

• Remote Diagnostics of Machines

• Remote Healthcare

• 1 paper in VISAPP 2012

Research Projects – Communication and

Computing


Grid with an Edge

• CPU utilization better in Condor

• Condor performs better with more nodes

• Further advantages w.r.t Heterogeneity, Versatility, Matchmaking & scheduling

• 3 papers in NGMAST 2012, PDPTA 2013, WiMob 2013

• 2 patents filed

• Flavio Bonomi et.al. MCC2012, Helsinki, Finland


Communication Aspect- Replace HTTP

Constrained Object Access Protocol (CoAP)

Improved version of CoAP using dynamic network condition sensing

Lightweight security protocols sensor authentication and data delivery

Use suitable lightweight application protocol between edge devices and core network

• 3 papers in ICDCN 2013, ICNC 2013, WPMC 2013

• 2 patents filed


Knowledge-driven Analytics

Ontology and Description for Sensors

• Non-standard / Multi-standard sensor descriptions across multi-domains

• Bringing all sensor data in common semantic form

• Semantic Sensor Web (OGC-SWE)

• Will provide ease of sensor discovery

Ontology and Knowledge based Reasoning

• Reasoning over continuously flowing (streaming) facts

• Reasoning over streaming data is integrated with “background knowledge” (static unchanging or slowly changing knowledge

• Spatio-temporal Queries

Semantic Analytics

• Analytics-as-a-Service

• Query language to specify processing workflow on Time series data

• 5 papers in ICST 2012 , ICNC 2013, ICUIMC 2013, ODBASE 2013, KES 2013

• 4 patents filed


Privacy Preservation

• Information-theoretic approach for sensitivity analysis of sensor data

• Requirement based application utility measurement

• Balancing of Privacy vs. Utility

Privacy

Utility

• 1 paper in SecurIT 2012

• 2 patents filed

Privacy

Preservation

Tool

26

Lab-at-a-Glance

Research Areas• Sensor Signal Processing• 2D/ 3D Image / Video Processing • Protocols, Security and Privacy• Parallel and Distributed Computing• Stream Processing and Reasoning• System Modeling and Identification• Sematic Sensor Web• Social Media Analytics

Academic Collaborations• Singapore Management University (iCity Platform)• Indian Statistical Institute (Protocol/Privacy/Security,

Image / Video Processing)• IIT Kharagpur (Analytics, Semantic Processing)• IIT Bombay (Energy and Utilities)• Jadavpur University (Signal Processing)

Higher Studies• PhD - 4• Masters - 4

Total Researchers - 37

Thank You

[email protected]

Iit kgp workshop

Technology

Transcript of Iit kgp workshop