Leveraging Complex Event Processing for Dog Behavior Monitoring through Wireless Wearable Sensors When Home Alone

THESIS DEFENSE

DIANA OVIEDO QUEVEDO

SOFTWARE ENG. LAB.

DECEMBER 7, 2016

Contents

Introduction

Methods

Results Analysis

Conclusions

Demo

2

Why Monitoring Pets?

3

[9] Video analysis of adult dogs when left home alone

The overall Pet Monitoring Project

4

Pet Monitoring System Components

5

What is Complex Event Processing?

Computing that performs operations on events.

Operations on events such as: Filtering out certain events,

Changing an event instance from one form to another,

Examining a collection of events to find a particular pattern.

6

CEP

?

Event Consumers

Event Producers

Methods

7

Pattern Rules Hierarchical Structure

Leve

l 2

Leve

l 3

Leve

l 1

Loud Vocalization

Climbing

(Unwanted Behavior)

Destructive Behavior

(Separation Anxiety)

Vocalization(Activity)

Jump

Up

Jump

Down

Stand in

2 legsWalking

Engage

Object

Food Stealing

(Unwanted Behavior)

Sniffing(Activity)

Head

DownBarking Howling Whining Digging

Urination/

Defecation

1. Scope for the dog’s behavior events and its hierarchical level classification

Classification process

How to Detect the Higher-Level Behavior?

8

Element Declarations

Variables String symbol,String event,Calendar timestamp, int id

Event typesBasicBehavior(id, symbol, timestamp),

HigherEvent(event,timestamp,id,level)

Patternevery (Level2stream(event in('CL','SF')) and (BasicBehavior(symbol in

('EO','UD')) or [3]BasicBehavior(symbol='DI'))

Context

Conditiontimer:within(3 min)

Action notifies destructiveBehavior(event,timestamp,id,3)

The Implementation: Class Diagram

9

The Implementation: Sequence Diagram

10

How one Level 3 Event is Generated

11

Level3

Level 1

Level 2

JU

HD

WA

Sniffing

HD

WA

Sniffing

EO

JD

Climbing

Food Stealing

Time in mm:ss.SS

FOOD STEALING EVENT GENERATION (IN 01:45.20 )

Results for Dog 2

12

Climbing

Sniffing

Food Stealing

0123456789

10111213141516171819202122232425262728293031323334

BEH

AV

IOR

ID

TIME ( MM:SS.00 )Level 1 Level 2 Level 3

Total time: 9 min 22 sec

L3

L1

L2

Results for Dog 3

13

0123456789

10111213141516171819202122232425262728293031323334

BEH

AV

IOR

ID

TIME ( MM:SS.00 )Level 1 Level 2 Level 3

Loud Vocalization

Destructive Behavior

Food StealingL3

L1

L2

Total time: 30 min 46 sec

Monitoring Reports Web App

14

PetMonitoringDB

Results Analysis

Performance

CEP in ESPER has shown in previous tests that it supports up to 500.000 events/s. The purpose of CEP for this research is mainly to detect behavior patterns in real time, and so far we only have a maximum of 3 events per second.

Concurrency and overhead

Were handled through Inbound threading. It was implemented in the configuration of the ESPER engine, which allows to handle it in an engine-level manner, instead of the (system) time-based processing by default.

Accuracy

15

Dog 1 Dog 2 Dog 3

Total detected events 12 20 152

Expected events 13 20 151

Accuracy 92.31% 100 % 100 %

Results Analysis (2)

Latency

In Level 3 events compared to the last previous event needed to match the pattern

16

Food Stealing Loud VocalizationDestructive

Behavior

Behavior ID 30 31 32

Max 1.160E-06 7.990E-06 1.273E-05

Average 1.160E-06 2.420E-06 2.080E-06

Min 1.160E-06 0.000E+00 0.000E+00

0.000E+00

2.000E-06

4.000E-06

6.000E-06

8.000E-06

1.000E-05

1.200E-05

1.400E-05

30 31 32

Max Average Min

Demo…

17

Conclusions and Future Work

Complex Event Processing can represent a significant contribution to the monitoring

of dog’s behavior when left home alone, parting from basic behavior inputs, higher-

level behavior events can be detected in order to produce only the adequate

amount of notifications to the owner.

The application of CEP for the detection of behavior events can be interpreted as a

kind of middleware application within a bigger IoT system. When integrated

provides a full service for taking care of dogs home alone.

The behavior monitoring can be further extended to a broader range of higher-level

pattern rules, including the prediction of unwanted behavior or diagnosis of

separation anxiety problems. It also can be extended for other animals or more than

one animal simultaneously.

18

References1. Konok V, Dóka A, Miklósi Á. The behavior of the domestic dog (Canis familiaris) during separation from and reunion with the owner: A questionnaire and an experimental study. Applied Animal Behaviour Science, 2011, 135(4): 300-308.

2. Frank D., Minero M., Cannas S., Palestrini C. Puppy behaviours when left home alone: A pilot study. Applied Animal Behaviour Science, 2007, 104, 61-701.

3. Lund J D, Jørgensen M C. Behaviour patterns and time course of activity in dogs with separation problems. Applied Animal Behaviour Science, 1999, 63(3): 219-236.

4. Lukham, David. 2002. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison Wesley.

5. Bhargavi.R, Vaidehi. Semantic intrusion detection with multisensor data fusion using complex event processing. Sadhana Indian Academy of Science, 2013, 38(2), 169-185.

6. Hänninen L1, Pastell M.CowLog: open-source software for coding behaviors from digital video. Behav Res Methods, 2009 May;41(2):472-6

7. Etzion, Opher and Niblett, Peter. 2011. Event Processing in Action. Manning.

8. Vaidehi V, Bhargavi R, Ganapathy K, et al. Multi-sensor based in-home health monitoring using complex event processing. Recent Trends in Information Technology (ICRTIT), 2011 International Conference on. IEEE, 2011: 570-575.

19

References

9. Scaglia E, Cannas S, Minero M, et al. Video analysis of adult dogs when left home alone. Journal of Veterinary Behavior: Clinical Applications and Research, 2013, 8(6): 412-417.

10. Tae-ho Chung, Chul Park, Yong-man Kwon, and Seong-chan Yeon, “Prevalence of canine behavior problems related to dog-human relationship in South Korea – A pilot study”, Journal of Veterinary Behavior, 11 (2016) pp. 26-30.

11. Software Engineering Lab, Korea University, 2016. Pets Management System Based on IoT for Improvement of Living Conditions.

12. Moshnyaga V, Osamu T, Ryu T, Hashimoto K “Identification of Basic Behavioral Activities by Heterogeneous Sensors of In-Home Monitoring System”. 6th International Workshop, HBU 2015. 160-174.

13. Martiskainen, Paula, et al. "Cow behaviour pattern recognition using a three-dimensional accelerometer and support vector machines." Applied Animal Behaviour Science 119.1(2009):32-38.

14. Lianli Gao, Hamish A. Campbell, Owen R. Bidder, Jane Hunter. Corrigendum to “A web-based semantic tagging and activity recognition system for species' accelerometry data” [Ecol. Inf. 13 (2013) 47–56]. Ecological Informatics, Volume 22, July 2014, Page 81.

15. Esper Team and EsperTech Inc. “Esper reference”. Version 5.2.0

20

Thank you.

21