Sakurai Lab.

MiningandForecastingofBigTime-seriesData

Yasushi Sakurai

Sakurai Lab. © 2016 Yasushi Sakurai 1

Sakurai Lab.@ Kumamoto University

BigData

Big time-series data


climate

EconomyWeb

Epidemic

Social/naturalphenomena

IoT,sensors

BigData



climate

EconomyWeb

Epidemic


Environmental sensors

- Temperature,- Air pressure, etc.Application

- Agriculture

IoT,sensors

BigData



climate

EconomyWeb

Epidemic


IoT,sensors

Online user activities

- Web clicks, - Online reviewsApplication

- Market analysis

BigData



climate

EconomyWeb

Epidemic


IoT,sensors

Sensor streams

- Vibration,AccelerationApplication

- Self-driving car- Structural health

monitoring

Motivation

• Given: Big time-series data– Sensor data– Web, online activities– Medical data

• Goal: Real-time forecasting• At-work: – Online marketing– Sensor monitoring– Forecasting future epidemics


New challenges


Indexing,Similaritysearch

Featureextraction

Datastream

Linear-modeling

Time-seriesdataanalysis

PCA ICAModel(M)Data(X)

ED,DTW,SAM

AR,LDS,ARIMA

Correlationmonitoring

etc.

DFT,DWT,SVD,ICA

Y

X

DTW

ED,DTW,SAM

AR,LDS,ARIMA

Correlationmonitoring

etc.

DFT,DWT,SVD,ICA

New challenges



Featureextraction

Datastream

Linear-modeling

Theycannothandle“Big”data,i.e.,

Y

X

PCA ICADTWModel(M)Data(X)

Automaticparameter-tuning

Non-linearphenomena

Complexevents Real-time

forecastingX

New challenges



Featureextraction

Datastream

Linear-modeling

Theycannothandle“Big”data,i.e.,

Y

X

PCA ICADTWModel(M)Data(X)

ED,DTWCorrelation

AR,ARIMA,LDS

StatStreametc…

DFT,DWT,SVD,ICA

Automaticparameter-tuning

Non-linearphenomena

Complexevents Real-time

forecastingX

• Time-series data analysis– Indexing and fast searching– Sequence matching– Clustering– etc.

• New research directions1. Automatic mining2. Non-linear modeling3. Large-scale tensor analysis


X ≈ + ...

Time

New research directions

NOmagic#

(R1) Automatic mining

No magic numbers! ... because,

Big data mining: -> we cannot afford human intervention!!


Manual- sensitive to the parameter tuning- long tuning steps (hours, days, …)

Automatic (no magic numbers)- no expert tuning required



“Automatic” mining algorithm

Find

Given

beaks

wings

tailfeathers

clapsbeaks

wings

tailfeathers

claps

Find: compact description of data X

Chicken dance

Sakurai Lab. 13


Idea (1): Multi-level chain model –HMM-based probabilistic model–with “across-regime” transitions

© 2016 Yasushi Sakurai

Model

Sequences

beaks

wings

claps

Regimes



Idea(2): Minimize encoding cost!


Good compression

Good description

1 2 3 4 5 6 7 8 9 10

CostMCostCCostT

(# of r, m)

CostM(M) + Costc(X|M)

Model cost Coding cost

min ( )



Sakurai Lab. 15© 2016 Yasushi Sakurai

AutoPlait (NO magic numbers)


(R2) Non-linear (gray-box) modeling

• Gray-box mining– If we know the equations

• Non-linear (differential) equations– Epidemic– Biology– Physics, Economics, etc.

• Modeling non-linear social phenomena– Non-linear analysis for Web online activities(e.g., information diffusion, interaction)


Gray-box



News spread in social mediaNumber of mentions in blogs/Twitter

20 40 60 80 100 120 140 1600

100

200

Time (hours)

# of m

entio

ns

20 40 60 80 100 120 140 1600

50

100

Time (hours)

# of m

entio

ns

Breaking news Decay

(power law)

News spread

(perhour,1week)



News spread in social media

Timen=0 Timen=nb Timen=nb+1

β

1. Un-informed bloggers

2. External shock

3. Infection (word-of-mouth)



Timen=0 Timen=nb Timen=nb+1

β

1. Un-informed bloggers

2. External shock

3. Infection (word-of-mouth)


Decayfunction: f (n) = β *n−1.5

f (n)

n

Linear f (n)

n

Log

Powerlaw

Memes in social media/blogs




20 40 60 80 100 1200

50

100

Time

Val

ue

OriginalSpikeM

20 40 60 80 100 1200

50

100

Time

Val

ue

OriginalSpikeM

20 40 60 80 100 1200

50

100

Time

Val

ue

OriginalSpikeM

20 40 60 80 100 1200

50

100

Time

Val

ue

OriginalSpikeM

20 40 60 80 100 1200

50

100

Time

Val

ue

OriginalSpikeM

20 40 60 80 100 1200

50

100

TimeV

alue

OriginalSpikeM


KDD 2012 21

Forecast tail-part and rise-part!

Given (1) first spike,(2) release date of two sequel movies (3) access volume before the release date

(1)Firstspike (2)Releasedate (3)Twoweeksbeforerelease

Y. Matsubara et al.



Android

Xbox

PlayStationWii

Online competitionInteractions

between keywords



Online competition(Google Search volume)

Android

Xbox

PlayStation Wii



The Web as a Jungle !

Ecosystem in the

Jungle

Ecosystem on the

Web




Ecosystem in the

Jungle

Ecosystem on the

Web--- Non-Linear equations ---

Pi (t +1) = Pi (t) 1+ ri 1−aijPi (t)j=1

d∑

Ki

#

$

%%%

&

'

(((

)

*

+++

,

-

.

.

.

Ecosystem on the Web


Online activities

Userresources

Biological species

Foodresources

PopularityPopulation

Annual events(e.g., Xmas)

Climate /season

Jungle Web



Android

Xbox

PlayStationWii

The Web as a Jungle !Interactions

between keywords



Android

Xbox

PlayStationWii


Seasonality

(R3) Large-scale tensor analysis


Time URL User

08-01-12:00 CNN.com Smith

08-02-15:00 YouTube.com Brown

08-02-19:00 CNET.com Smith

08-03-11:00 CNN.com Johnson

… … …

RepresentasMth ordertensor(M=3)

URL

user

Time

x

u

v

n

Elementx:#ofevents

e.g.,‘Smith’,‘CNN.com’,‘Aug1,10pm’;21times

• Time-stamped events– e.g., web clicks

X



Complex time-stamped events

Timestamp URL User Device2012-08-01-12:00 CNN.com Smith iphone2012-08-02-15:00 YouTube.com Brown iphone2012-08-02-19:00 CNET.com Smith mac2012-08-03-11:00 CNN.com Johnson ipad

… … … …2012-08-05-12:00 CNN.com Smith iphone2012-08-05-19:00 CNET.com Smith iphone

Forecast

{time,URL,userID,accessdevices,httpreferrer,…}




Object

Actor

Time

(business) (news) (media)

…

Object/URLActor/user Time

Webclicks




Object

Actor

Time

(business) (news) (media)

…

Object/URLActor/user Time

Webclickse.g.,businesstopicvectorsObject/URL

Money.comCNN.com

SmithJohnson

Actor/user

Time

Mon-Fri Sat-Sun

Higher value:Highly related

topic


Sakurai Lab. © 2016 Yasushi Sakurai 33URLmatrix Usermatrix Timematrix

NoisyL SparseL

Complex time-stamped events(Originaldata)

New research directions1. Automatic mining2. Non-linear (gray-box) modeling3. Large-scale tensor analysis

X ≈ + ...

NOmagic!

New challenge: MANT analysisMulti-Aspect Non-linear Time-series

Putalltogether

M� A� N� T�M� A� N� T�

Putting it all together 3h-TutorialsSIGMOD’15WWW’16



w/Sakurai,Panhuis,Faloutsos

Local patternsP3

External shocksP4

Mistakes, errorsP5

P2

SeasonalityP1

Vaccination

Non-Linear tensor analysis

New challenge: MANT analysis



X

=

P1 P2 P3 P4 P5

B RN ME

FUNNEL

X

Given: Tensor X (disease x location x time)

Find: Compact description of X, “automatically”




X

=

P1 P2 P3 P4 P5

B RN ME

FUNNEL

X

Given: Tensor X (disease x location x time)

Find: Compact description of X, “automatically”


DiscontinuitiesSeasonality


Non-Linear mining & forecasting of local competition



Time-stamped events: {activity, location, time}

© 2016 Yasushi Sakurai 39Sakurai Lab.

m

d

n

X

Activity

Time(weekly)


Given: Tensor X(activity x location x time)

Find: Compact description of X


X

X=

CompCube

B C S D


Given: Tensor X(activity x location x time)

Find: Compact description of X


X

X=

CompCube

B C S D

Basics Competition Seasonality Deltas

Modeling power of CompCube

Google search for


Kindle, Nexus

US

CA

JPCNBRAU

ZA

IT

Weak/Average/Strong

Local Competition strength


Google search for


Kindle, Nexus

US

CA

JPCNBRAU

ZA

IT

Local Competition strength

Weak/Average/Strong


Local seasonality for


iPod

Component #1 Component #2


Local seasonality for


iPod

Component #1 Component #2

XmasDec.

ChineseNew Year

Feb.


Fitting for


News resources

2004 2006 2008 2010 2012 20140

0.2

0.4

0.6

0.8

1

Time (weekly)

# of

clic

ks @

tim

e

Fitting result − RMSE=0.056

1CNN 2Fox_News 3TIME 4Google_News 5BBC 6Yahoo_News 7AP 8Huffington_Post 9MSN_News10Al_Jazeera

Detected!

Wikipedia


Fitting for


US electionNov. 2008

Local attention toUS election

Weak/Strong

Wikipedia

News resources



3.Forecastfutureactivities

?Future

Time

Non-Linear mining & forecasting of local competition

X ≈ + ...

NOmagic!New research directions

1. Automatic mining2. Non-linear (gray-box) modeling3. Large-scale tensor analysis

New challenge: MANT analysisMulti-Aspect Non-linear Time-series

Putalltogether

M� A� N� T�M� A� N� T�

Putting it all together 3h-TutorialsSIGMOD’15WWW’16


Industrial contribution

• Bridge structural health monitoring– Structural monitoring using vibration/shock sensors– Keep track of lag correlations for sensor data

streams

BRAID@SIGMODʼ05



• Bridge structural health monitoring– Goal: real-time anomaly detection for disaster

prevention– Several thousands readings (per sec) from

several hundreds sensor nodes

Structural health monitoringVibration/shock sensor

• Uses BRAID• Metropolitan Expressway

(Tokyo, Japan)

BRAID@SIGMOD’05



• Bridge structural health monitoring with BRAID

Metropolitan Expressway(Tokyo, Japan)

Tokyo Gate Bridge(Tokyo, Japan)

Can Tho Bridge (Vietnam)

BRAID@SIGMOD’05



– Automobile sensor data{ trip ID, zone ID, object}

TrailMarker


velocity

Object（sensors）

Longitudinal/lateral acceleration

w

d

n

Trip

Zone

Original tensor X

WebDBFʼ15Best paper



TrailMarker



Trip 1

w

d

n

Trip

Zone

Original tensor X

Trip 2

Trip 3

zone

zone

zone



TrailMarker



Thank you

M� A� N� T�M� A� N� T�

X ≈ + ...

NOmagic!

Code/Data/Tutorial: http://www.cs.kumamoto-u.ac.jp/~yasushi/

Sakurai Lab.


Sakurai Lab.

Documents

Transcript of Sakurai Lab.