Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS...
Transcript of Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS...
![Page 1: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/1.jpg)
Armazenando Séries de Dados Temporais em Bases de Dados Alternativas
André CarlucciDiretor de Tecnologia – Way2
.Net MVP, Intel Innovator, Intel Black Belt
@andrecarlucci
![Page 2: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/2.jpg)
O que são séries temporais?
![Page 3: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/3.jpg)
“Uma série temporal é uma sequência de medições coletadas normalmente em ordem cronológica com um intervalo fixo.”
Mr. Wikipedia, (2004), TimeSeries
![Page 4: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/4.jpg)
A long time ago in a galaxy far far away…
![Page 5: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/5.jpg)
Get the data!
![Page 6: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/6.jpg)
250 Medidores
154 Grandezas (5, 15 ou 60 minutos)
2.5M/registros dia
![Page 7: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/7.jpg)
sNot that much
![Page 8: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/8.jpg)
TB_MEDIDOR_ENE
Id Medidor
Timestamp
Energia Ativa In
Energia Ativa Out
Energia Reativa In
Energia Reativa Out
TB_MEDIDOR
Id Medidor
Nome
Serial
…
Mesmo para engenharia, qualidade, etc...
![Page 9: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/9.jpg)
![Page 10: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/10.jpg)
Elipse E3 Server Oracle 8i
Vida em 2005
Asp.Net Clássico
![Page 11: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/11.jpg)
Fast Forward 2015
![Page 12: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/12.jpg)
10 anos depois
![Page 13: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/13.jpg)
![Page 14: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/14.jpg)
10 anos depois
![Page 15: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/15.jpg)
200K Medidores
14 Grandezas (5 minutos)
806M/registros dia
![Page 16: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/16.jpg)
sConsiderável
![Page 17: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/17.jpg)
Leituras a cada 6 horas:
201M
![Page 18: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/18.jpg)
A forma atual aguenta?
![Page 19: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/19.jpg)
Way2 PdC Validador
Quebra em Microservices
DataIn
Bulk Insert
![Page 20: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/20.jpg)
Repensando a forma de
armazenar
![Page 21: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/21.jpg)
Timestamp Id Energia
Ativa In
Energia
Ativa Out
Energia
Reativa In
Energia
Reativa
Out
1/1/2005
10:00
13 134.1 0 12.2 0
1/1/2005
10:05
13 132.7 0 11.2 0
1/1/2005
10:10
13 132.6 0 14.4 0
… … … … …
Modelo original (tall storage):
![Page 22: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/22.jpg)
Timestamp Id Energia
Ativa In
Energia
Ativa Out
Energia
Reativa In
Energia
Reativa
Out
1/1/2005
10:00
13 134.1 0 12.2 0
1/1/2005
10:05
13 132.7 0 11.2 0
1/1/2005
10:10
13 132.6 0 14.4 0
… … … … …
Modelo original (tall storage):The Bad:
Locks com inserção paralela de grandezas
Insert or update
Ruim para bulk insert
![Page 23: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/23.jpg)
Id Day 00:05 00:10 00:15…
1 1/1/2016 0 12.2 0
1 2/1/2016 0 11.2 0
1 3/1/2016 0 14.4 0
… … … … …
Wide Storage, por grandeza:
![Page 24: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/24.jpg)
Id Day 00:05 00:10 00:15…
1 1/1/2016 0 12.2 0
1 2/1/2016 0 11.2 0
1 3/1/2016 0 14.4 0
… … … … …
Wide Storage, por grandeza:
Timestamp Id Energia
Ativa In
Energia
Ativa Out
Energia
Reativa In
Energia
Reativa
Out
1/1/2005
10:00
13 134.1 0 12.2 0
1/1/2005
10:05
13 132.7 0 11.2 0
1/1/2005
10:10
13 132.6 0 14.4 0
… … … … …
Modelo original (tall storage):The Bad:
Complexidade de inserts, queries e group by.
Não suporta integralizações diferentes.
Ruim para bulk insert
![Page 25: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/25.jpg)
Timestamp Id Valor
1/1/2005 10:00 1 134.1
1/1/2005 10:05 1 132.7
1/1/2005 10:10 1 132.6
… … …
Tall Storage, por grandeza, por ano:
Tb_EnergiaAtiva_2015, Tb_EnergiaAtiva_2016…
![Page 26: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/26.jpg)
Timestamp Id Valor
1/1/2005 10:00 1 134.1
1/1/2005 10:05 1 132.7
1/1/2005 10:10 1 132.6
… … …
Tall Storage, por grandeza, por ano:
Id Day 00:05 00:10 00:15…
1 1/1/2016 0 12.2 0
1 2/1/2016 0 11.2 0
1 3/1/2016 0 14.4 0
… … … … …
Wide Storage, por grandeza:
Timestamp Id Energia
Ativa In
Energia
Ativa Out
Energia
Reativa In
Energia
Reativa
Out
1/1/2005
10:00
13 134.1 0 12.2 0
1/1/2005
10:05
13 132.7 0 11.2 0
1/1/2005
10:10
13 132.6 0 14.4 0
… … … … …
Modelo original (tall storage):Nossa Escolha em termos de sql:
Simples
Bulk Insert sem medo
Sem locks nos inserts
Tabelas auto-criadas
Backup e restore super simples
Tamanho das tabelas controlados ($)
![Page 27: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/27.jpg)
TB_EnergiaAtiva
Id Medidor
Timestamp
Valor
TB_Medidor
Id Medidor
Serial
…
Relacionamento entre Tabelas
![Page 28: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/28.jpg)
28.7%Ganho de
TB_EnergiaAtiva
Id Medidor
Timestamp
Valor
TB_Medidor
Id Medidor
Serial
…
![Page 29: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/29.jpg)
Dados Relacionais Séries de Dados
Separação das Bases
Sql?
![Page 30: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/30.jpg)
Restrições…
![Page 31: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/31.jpg)
Bases de Dados Analisadas
![Page 32: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/32.jpg)
![Page 33: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/33.jpg)
4th base mais popular*
Apache License
Orientada a documentos
*http://db-engines.com/en/ranking
![Page 34: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/34.jpg)
Estratégia 1:
Um documento por medição
![Page 35: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/35.jpg)
{timestamp: ISODate("2015-11-10T23:05:00.000Z"),id: 100type: "energiaAtiva",value: 12
},{
timestamp: ISODate("2015-11-10T23:10:00.000Z"),id: 100type: "energiaAtiva",value: 14
}somente inserts
![Page 36: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/36.jpg)
Estratégia 2:
Schema orientado a documentos
![Page 37: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/37.jpg)
{day: ISODate("2015-10-10T00:00:00.000Z"),type: "energiaAtiva",id : 100,values: {
1: { 0: 200, 1: 200, …, 12: 100 },…,23: { 0: 160, 1: 120, …, 12: 110 },24: { 0: 130, 1: 140, …, 12: 150 }
}}
![Page 38: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/38.jpg)
db.metrics.update({ day: ISODate("2013-10-10T00:00:00.000Z"),id: 100,type: "energiaAtiva"
}, {$set: {"values.10.01": 193 }
})
update preciso
![Page 39: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/39.jpg)
Reads:
Estratégia 1: 288 reads (dia)
Estratégia 2: 1 read (dia)
![Page 40: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/40.jpg)
Otimizações:
Ter um job criando documentos vazios
Definir ID como {timestamp}_{id} para range
scans
![Page 41: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/41.jpg)
Nossas impressões:
Developer Friendly
Instalação Simples
Sharding
Maduro
![Page 42: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/42.jpg)
![Page 43: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/43.jpg)
Base de dados feita para Séries
de Dados
![Page 44: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/44.jpg)
API - Write
HTTP POST: http://localhost:8086/write?db=mydb'
EneatDel,Med=102 value=632.0 1420070400EneatDel,Med=102 value=632.0 1420070700…
![Page 45: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/45.jpg)
API - Read
HTTP GET: http://localhost:8086/query?db=mydb&q=
SELECT value FROM EneatDel WHERE Med=102
![Page 46: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/46.jpg)
Regras para retenção de dados
Dados
Delete dados
com mais de
um ano
![Page 47: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/47.jpg)
Séries contínuas
5 MIN
Agrupe dados
por hora
1 DIA
![Page 48: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/48.jpg)
Nossas impressões
API Simples
Queries poderosas
Funcionalidades específicas
Sharding (free até v0.11, depois pago)
Imaturo ainda (v0.11, windows)
![Page 49: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/49.jpg)
Outras alternativas
Graphite
OpenTSDB
KairosDB
Druid
Riak TS
TempoIQ
Blueflood
Cityzen Data
Hawkular Metrics
Newts
TimeSeries.Guru
SiteWhere
Axibase
Infiniflux Yanza
Kdb+
Prometheus
![Page 50: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/50.jpg)
![Page 51: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/51.jpg)
Descrição Bytes
DateStart 8
DateEnd 8
Current 8
Interval 4
Valor 1 8
Valor 2 8
Valor 3 8
… …
28 bytes
N * 8 bytes
28 + (288 * 365d * 8) bytes = 821kB
![Page 52: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/52.jpg)
API - Read
GET: http://localhost/list?{id}&{inicio}&{fim}
![Page 53: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/53.jpg)
API - Write
POST: http://localhost/add/{id}{
Data: 2015-01-01T00:05,Value: 100
},…
![Page 54: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/54.jpg)
Id: med/12/energiaAtivaInicio: 30/03/2016Fim: 31/03/2016
Query:
![Page 55: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/55.jpg)
Id:med/12/energiaAtiva
Query: ache o arquivo
Arquivo: c:\neutrino\db\med\12\energiaAtiva.ts
![Page 56: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/56.jpg)
Query: ache o primeiro registro
PesquisaInício
InícioArquivo
-
IntervaloArquivo
Registrosa pular=
![Page 57: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/57.jpg)
Query: ache o primeiro registro
Header
(28 bytes)8B 8B 8B 8B 8B 8B 8B 8B 8B… 8B 8B …
Registros a pular
![Page 58: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/58.jpg)
Query: ache o número de registros
(Fim – início) / Intervalo Arquivo:
31/03/2016 - 30/01/2016 / 5min = 288
![Page 59: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/59.jpg)
var data = new byte[registros * sizeof(double)];
var fs = new FileStream(path);
fs.Seek(offset);
fs.ReadAsync(data, 0, data.Length);
return Deserialize(data);
![Page 60: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/60.jpg)
Nossas impressões
Customizado
Backup ultra simples
Pouco espaço de armazenamento
Pode ser incorporado a aplicação (C#)
Ainda um experimento!!!
![Page 61: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/61.jpg)
Storage
Oracle
Antigo
Oracle Mongodb Influxdb Neutrino
Medidor/Ano 8 MB 3.5 MB 1.9 MB 1.3 MB 0.8 MB
200K, 14
séries106 TB 47 TB 25 TB 17 TB 10 TB
![Page 62: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/62.jpg)
Conclusões
Trate series de dados como series de dados.
![Page 63: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/63.jpg)
Conclusões
Influxdb nos pareceu a melhor opção, mas
há muitas incertezas ainda.
![Page 64: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/64.jpg)
Conclusões
Apesar de tudo, um banco relacional
continua sendo uma excelente opção, pois
nem sempre a decisão é somente técnica.
![Page 65: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/65.jpg)
Perguntas?
André CarlucciDiretor de Tecnologia – Way2
.Net MVP, Intel Innovator, Intel Black Belt
@andrecarlucci
![Page 66: Armazenando Séries de Dados Temporais em Bases de · PDF fileKairosDB Druid Riak TS TempoIQ Blueflood Cityzen Data Hawkular Metrics Newts TimeSeries.Guru SiteWhere Axibase Infiniflux](https://reader034.fdocuments.net/reader034/viewer/2022051406/5aa5563c7f8b9a2f048d4535/html5/thumbnails/66.jpg)
Obrigado!
André CarlucciDiretor de Tecnologia – Way2
.Net MVP, Intel Innovator, Intel Black Belt
@andrecarlucci