Big Data* *small print
1
Tyler SingletaryDirector of Platform,
Klout@harmophone
My Perspective
• Klout uses GNIP,Twitter, Facebook, Foursquare, LinkedIn and about a dozen other APIs. Consume terabytes/day.
• Distribute own data to partners (Salesforce, CrimsonHexagon) & channels (Gnip, Datasift)
• Over 1,700 monthly active developers
2
Finding a great APIis only the first step
• The data is what you want!
• You get to it easily, it’s well organized!
• This is awesome! I’m going to do X,Y and Z!
• Did you read the TOS?
3
Constraints breed Creativity
• Knowing what you can’t do opens doors
• Legalese is a programming language
• Bend rules, but build a rollback
• Respect the data of others as if it were your own
4
Did you know?• Facebook forbids you from distributing
FBIDs
• Also from giving their data, or a derivative, to ad networks, etc.
• Twitter restricts you from separating location data from a tweet
• Nearly all require attribution, have cache and storage restrictions
A Rant About Scraping
• If a service has data you want, but doesn’t offer it in an API, do you have a right to it?
• Does this serve a long term business relationship?
• Whenever possible: have a relationship
• Robots.txt
Ubiquity• If APIs are going to be the defacto way businesses communicate, then:
• The rules are important.
• They need to apply to innovators as well as established business
• Terms must scale and be programmatically delivered and executed
• When you judge an API you discover, think beyond your own need:
• If I’m taking this data, and 50 other companies are, what benefit do I have? They have? The provider has?
• You serve at the provider’s pleasure.
• How can APIs be industry-wide if industry is afraid of abuse?
Distributing Your Data
• What do you want to encourage?
• What do you want discourage?
• Contingencies!
• Be upfront and clear: cucumber stories.
• legalese: thar be dragons! Test coverage.
• What do you give exceptions for?
8
DRM
• No, not that DRM.
• Okay, you probably still think it’s that DRM.
• Data Rights Management
Robots.json• A standard way to describe
data rights
• Permissible use
• Data storage, caching
• Rate limits
• Copyrights, data stewardship
• License term, financial details
Alternatives
• Hypermedia?
• Requiring everyone to read and understand TOS and implement
• Trust? ignorance?
• Headers
Thank you!
12
Tyler SingletaryDirector of Platform,
Klout@harmophone
{can: republish,must: attribute,
retention: unlimited}
Top Related