Big Data = Bigger Metadata
Transcript of Big Data = Bigger Metadata
![Page 1: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/1.jpg)
Big Data = O’Reilly Strata Conference
February 29 2012
Bigger Metadata
![Page 2: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/2.jpg)
Pivot/Skate, etc…
Refounded 2006
Neighborhood boundaries
Mass transit data
Refocused 2009
SaaS for mapping + on-demand data
Founded 2003
Poor man’s GIS
Panamap
![Page 3: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/3.jpg)
Achtung!
NoSQL is no panacea
Big Data isn’t about data
Big Data isn’t new
Big Data doesn’t present a Boolean quandary
With power comes responsibility
AWS bills
Lady Gaga tweets
Innumeracy (correlation v causation)
![Page 4: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/4.jpg)
Big v Important
Big
Heterogeneous
Raw
Distributed
Streaming/real time
Search for meaning
Time-sensitive
Philosophical
Important
Well-defined schema
High value (not free)
Test-driven
Relational
Historical
Enterprise-focused
![Page 5: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/5.jpg)
Data Exhaust
Analytics Probes
Gov 2.0Social Media
![Page 6: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/6.jpg)
Platforms
Commoditization of compute and storage
![Page 7: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/7.jpg)
A Brief History of Metadata
Callimachus Library of Alexandria, Egypt
![Page 8: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/8.jpg)
A Brief History of Metadata
“Pinakes” (lists)
Title
Category
Author
Author birthplace
Father
Word count
Callimachus
![Page 9: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/9.jpg)
A Brief History of Metadata
![Page 10: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/10.jpg)
A Brief History of Metadata
![Page 11: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/11.jpg)
A Brief History of Metadata
Card catalog room,
Library of Congress c. 1920
![Page 12: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/12.jpg)
A Brief History of Metadata
Dewey Decimal System goes electronic in 1967
![Page 13: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/13.jpg)
Out with the Old, in with the New
Archiving card catalogs
after digitization
![Page 14: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/14.jpg)
Why Can’t We Be Together?
Metadata Data
![Page 15: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/15.jpg)
Exponential Growth in Data
1876
TaxonomyPinakes
300 BC
Database
1970
Catalog
1595 AD
Data
Unprecedented rate of data creation, 1995-today
![Page 16: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/16.jpg)
Oh, How I’ve Missed You
The reunification of metadata
and the artifact
![Page 17: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/17.jpg)
Together At Last
![Page 18: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/18.jpg)
GIS Data is Unevolved
+ =
![Page 19: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/19.jpg)
Enter the Data Curator
Part social scientist, part librarian,
part statistician, part RDBMS wiz
![Page 20: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/20.jpg)
DIKW Model
Data
Fact, Signal, Symbol
Information
Structural v Functional
Symbolic v Subjective
Knowledge
Processed
Procedural
Propositional
![Page 21: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/21.jpg)
Popularity (Google Trends)
![Page 22: Big Data = Bigger Metadata](https://reader034.fdocuments.net/reader034/viewer/2022052116/58f02bd71a28ab21248b4617/html5/thumbnails/22.jpg)
Words to Live By
dxdt/