Big Data, Big Quality? by Irene Gonzálvez at Big Data Spain 2017
32
-
Upload
big-data-spain -
Category
Technology
-
view
531 -
download
0
Transcript of Big Data, Big Quality? by Irene Gonzálvez at Big Data Spain 2017
TC4D: Test Certified for DataLevel 1: Set-up, monitoring, alerting and documentation
Level 2: Data management and Unit tests
Level 3: Build your defenses
What’s next?Build an algorithm library for anomaly detection (ML4ALL)
Provide the infrastructure to ‘plug&play’ more algorithms
Provide parameter recommendations to tweak the algorithms
What’s next?Spotify-wide strategy
● Have metrics to understand when a dataset qualifies as ‘good’ quality.
● Identify which datasets are critical/ central to Spotify and make them of ‘good’ quality