Introduction to influx db
24
INTRODUCTION TO
-
Upload
roberto-gaudenzi -
Category
Software
-
view
303 -
download
5
Transcript of Introduction to influx db
INTRODUCTIONTO
Hello!
I am Roberto GaudenziYou can find me [email protected]
MS in Engineering in Computer Science
Pervasive Systemsa.y. 2015-16
Prof. Chatzigiannakis
TIME-SERIES DATA
WHAT IS TIME-SERIES DATA?
A time-series is a sequence of data points, consisting of successive measurements made over a time interval.
Two major types: Regular and Irregular
They are used in statistics, mathematical finance, pattern recognition, IoT, …
4
WHAT IS TIME-SERIES DATA? (2)
Examples of time-series data:◦ Stock price evolution
◦ Bank transactions
◦ ...
Example of non time-series data:◦ Facebook relationships
5
INFLUXDB
WHAT IS INFLUXDB?
InfluxDB is an open-source database specifically designed to handle time-series data.
Ideal for any use case involving large amounts of timestamped data, including IoT sensor data and real-time analytics.
7
Points are written to InfluxDB according to the Line Protocol.Three parts:
◦ The key, composed by the measurement
name and a set of tags
◦ The fields
◦ The timestamp
Example(CLI):
DATA FORMAT: THE LINE PROTOCOL
8
INSERT cpu,host=server02,region=uswest value=31434055562000010000
DATA FORMAT: THE LINE PROTOCOL (2)
Example(InfluxDB-Python):
client = InfluxDBClient(host, port, user, password, dbname)json_body = [
{ "measurement": "cpu", "tags": {
"host": "server02","region": "uswest"
},"time": "2009-11-10T23:00:00Z","fields": {
"value": 3}
} ]client.write_points(json_body)
9
InfluxQL is an SQL-like query language for interacting with data in InfluxDB.
Example (CLI):
QUERY LANGUAGE: INFLUXQL
SELECT mean(water_level) AS mean_levelFROM h2o_feetWHERE time >= '2015-08-18'GROUP BY time(10d)ORDER BY time DESC
10
QUERY LANGUAGE: INFLUXQL (2)
11
Example(InfluxDB-Python):
query = “””select mean(water_level) as mean_levelfrom h2o_feet where time >= \’2015-08-18\’group by time(10d)order by time DESC”””
client = InfluxDBClient(host, port, user, pwd, dbname)# Returns an array of JSONs containing results# from all matching seriesresult = client.query(query)print("Result: {0}".format(result))
Working with much data over a long period of time can create storage concerns.Solutions: downsample or expire the data.
InfluxDB offers two mechanisms for dealing with this issue: Continuous Queries and Retention Policies.
DOWNSAMPLING AND DATA RETENTION
12
A retention policy (RP) describes:
◦ for how long InfluxDB keeps data
◦ how many copies of those data are stored
RETENTION POLICIES
13
CREATE RETENTION POLICY two_hoursON food_dataDURATION 2hREPLICATION 1
Example(InfluxDB-Python):
client.create_retention_policy('two_hours', '2h', 1, 'food_data',default=False)
Example (CLI):
A continuous query (CQ) is an InfluxQL query that runs automatically and periodically within a database.
Example (CLI):
CONTINUOUS QUERIES
14
CREATE CONTINUOUS QUERY cq_30mON food_dataBEGIN SELECT mean(website) AS mean_website,
mean(phone) AS mean_phoneINTO food_data."default".downsampled_orders FROM ordersGROUP BY time(30m)
END
CONTINUOUS QUERIES (2)
15
Example(InfluxDB-Python):
query = “””create continuous query cq_30mon food_databeginselect mean(website) as mean_website,
mean(phone) as mean_phoneinto food_data."default".downsampled_ordersfrom ordersgroup by time(30m)
end”””# An InfluxDBClientError will be returned in case of errortry:
result = client.query(query)except:
print("Note: continuous query already exists.")
The built-in web admin interface is a simple way to interact with InfluxDB.
MANAGE YOUR DATA: THE WEB INTERFACE
16
◦ Write Data
◦ Query Data
◦ Access remote
instances
INFLUXDB IN A SLIDE
“SQL-like”
HTTP APIs andBuilt-in interface
Choose your language
SQL
<\>
New
Designed for time series
Flexible and scalable
17
DEMO
INFLUXDB EXAMPLE: INFLUXCHAT
◦ Simple room-chat application in Python
◦ When a user types a message, this is sent to
the server, that forwards it to all connected
users
◦ The server also stores messages into InfluxDB
◦ Continuous Query and Retention Policy for
collecting statistics about message traffic
19
INFLUXDB EXAMPLE: INFLUXCHAT (2)
20
# Set up the InfluxDB client
DB_NAME = "PervSystPers“DB_ADDRESS = "localhost“DB_PORT = 8086MEASUREMENT_NAME ="influxchat“influxdb_client = InfluxDBClient(host=DB_ADDRESS,
port=DB_PORT,database=DB_NAME)
INFLUXDB EXAMPLE: INFLUXCHAT (3)
21
# Set up the continuous queryCONT_QUERY = “””create continuous query cq_30m on PervSystPers
begin select count(value) as num_msginto PervSystPers.\"default\".downsampled_msgfrom influxchatgroup by time(30m)
end”””try:
result_db = influxdb_client.query(CONT_QUERY)except:
print("Note: continuous query already exists.")
# Set up the retention policyRET_POL_NAME = 'del_4_weeks‘RET_POL_PERIOD = ‘4w’RET_POL_N_COPY = 1resultdb = influxdb_client.create_retention_policy(RET_POL_NAME,
RET_POL_PERIOD,RET_POL_N_COPYDB_NAME, default=TRUE)
INFLUXDB EXAMPLE: INFLUXCHAT (4)
22
# Send the incoming message to InfluxDB
json_body = [{
"measurement": MEASUREMENT_NAME,"tags": {
"username": sender_name},"fields": {
"value": msg_content}
}]influxdb_client.write_points(json_body)
CREDITS AND LINKS
◦ Presentation template by SlidesCarnival
◦ Code available on Github
◦ Presentation available on Slideshare
◦ You can find me also on LinkedIn
24
