YQL talk at OHD Jakarta
-
Upload
michael-smith -
Category
Technology
-
view
976 -
download
1
description
Transcript of YQL talk at OHD Jakarta
![Page 1: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/1.jpg)
Standardizing web querying - Using YQL for accessing and exposing APIs
![Page 2: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/2.jpg)
Agenda
• Introduction to YQL• How to get started• What’s supported through YQL• Theme of the day – mim.yahoo.com• Lots of examples using mim API • Mashup using YQL• Introduction to more YQL goodies
-Server side javascript execution-YQL hosted storage
• Concluding remarks
..lets keep it interactive
![Page 3: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/3.jpg)
YQL – Yahoo Query Language
• Enables you to access Internet data with SQL-like commands• YQL hides the complexity of Web service APIs by presenting data as
simple tables, rows, and columns.• YQL includes pre-defined tables for popular Yahoo! Web services such
as Flickr, Search, Geo and Mim.• YQL can access services on the Internet that output data in the
following formats: HTML, XML, JSON, RSS, Atom, and microformat.• Create fast mashups reducing latency, code
![Page 4: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/4.jpg)
How to get started
• Explore the APIs - http://developer.yahoo.com/everything.html• Get a developer key - https://developer.apps.yahoo.com/wsregapp/• Read the documentation - http://developer.yahoo.com/yql/guide/• Visit the YQL Console - http://developer.yahoo.com/yql/console/
-The YQL Console enables you to run YQL statements interactively from your browser.
• Hack away..
![Page 5: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/5.jpg)
Y! APIs supported by YQL
-Show tables-Flickr-Y! geo-Y! local search-Y! maps-Y! mim-Y! music-Y! MyBlogLog-Y! search-Y! social-Y! upcoming
![Page 6: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/6.jpg)
External APIs supported by YQL
- Amazon ECS- AWS- Bit.ly- Comicvine- Craigslist- Del.icio.us- Dopplr- Facebook- Friendfeed- Github- Google (News, Search,
Translate)- Guardian
- IMDB- LastFm- MediaWikiAPI- Microsoft (Bing)- Twitter- Zillow
Through datatables.orgShow tables
![Page 7: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/7.jpg)
Theme of the day – mim.yahoo.com
• Mim has an YQL based API for everything – reading, searching, updating, deleting and so on.-Profile information of mim users-Followers/following information-Query for users-Listing mim posts of a user-Popular posts-Search for posts- Inserting (posting to mim) through YQL
Reposting, commenting deleting
![Page 8: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/8.jpg)
Getting information about self (logged in)
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="1” yahoo:lang="en-US” yahoo:uri="http://query.yahooapis.com/v1/yql?q=select+*+from+meme.info+where+owner_guid%3Dme">
<results> <meme> <guid>NKADUBBZPUQQG6GF6AHTDTABSQ</guid> <name>shreeni</name> <title>Shreeni&#39;s Mim</title> <description/> <url>http://meme.yahoo.com/shreeni/</url> <avatar_url>http://a.com/b.jpeg</avatar_url> <language>en</language> <followers>18</followers> </meme> </results></query>
select * from meme.info where owner_guid=me
![Page 9: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/9.jpg)
In JSON
{"query": { "count": "1", "created": "2009-11-19T10:33:11Z", "lang": "en-US", "updated": "2009-11-19T10:33:11Z", "uri": "http://query.yahooapis.com/v1/yql?q=select+*+from+meme.info+where+owner_guid
%3Dme", "results": { "meme": { "guid": "NKADUBBZPUQQG6GF6AHTDTABSQ", "name": "shreeni", "title": "Shreeni's Mim", "description": null, "url": "http://mim.yahoo.com/shreeni/", "avatar_url":
"http://d.yimg.com/gg/shreeni/avatars/7a0d55f7d0aab9e326a7906a110f8027d46bd49b.jpeg",
"language": "en", "followers": "18"}}}}
select * from meme.info where owner_guid=me
![Page 10: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/10.jpg)
How to run YQL queries
•Convert to URL: http://query.yahooapis.com/v1/public/yql?q=<yql_query>&format=<json/xml>
•Use the YQL Console - http://developer.yahoo.com/yql/console/•Use a PHP SDK provided by Y! - http://developer.yahoo.com/social/sdk/#php
![Page 11: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/11.jpg)
Selecting specific fields
<results> <meme> <title>Shreeni&#39;s Mim</title> <url>http://mim.yahoo.com/shreeni/</url>
<avatar_url>http://d.yimg.com/gg/shreeni/avatars/7a0d55f7d0aab9e326a7906a110f8027d46bd49b.jpeg</avatar_url>
<language>en</language> </meme> </results>
select title, url, avatar_url, language from meme.info where name='shreeni'
![Page 12: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/12.jpg)
Selecting specific fields (also Followers API)
<results> <meme>
<name>herry</name> </meme> <meme> <name>grahamhills</name> </meme> <meme> <name>fizzycitrus</name> </meme> </results>
select name from meme.followers where owner_guid = “NKADUBBZPUQQG6GF6AHTDTABSQ”
![Page 13: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/13.jpg)
Sub-select and Limits
<results> <meme>
<name>herry</name> </meme> <meme> <name>grahamhills</name> </meme></results>
select name from meme.followers where owner_guid in (select guid from meme.info where name='shreeni’) limit 2
![Page 14: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/14.jpg)
Functions
select name from meme.followers where owner_guid in (select guid from meme.info where name='shreeni') | sort(field="name", descending="true”)
![Page 15: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/15.jpg)
Functions
- Sort- Tail- Truncate-Reverse-unique- Sanitize
![Page 16: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/16.jpg)
Following API in mim
select name from meme.following where owner_guid in (select guid from meme.info where name='bangwinissimo')
![Page 17: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/17.jpg)
Searching for people
select guid, name, title, language from meme.people where query="Fajar”
select guid, name, title, language from meme.people where query="Fajar" and language=”id"
![Page 18: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/18.jpg)
Finding specific Posts of a mim user
select * from meme.posts where owner_guid in (select guid from meme.info where name='bangwinissimo')
select * from meme.posts where owner_guid in (select guid from meme.info where name='bangwinissimo') and type='text'
![Page 19: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/19.jpg)
Popular Posts
select * from meme.popularselect * from meme.popular where locale="id"
![Page 20: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/20.jpg)
Search in mim
SELECT * FROM meme.search WHERE query="jakarta”
![Page 21: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/21.jpg)
Inserting/Deleting mim posts
INSERT INTO meme.user.posts (type, content) VALUES ("text", "This is a text post from YQL!”)
DELETE FROM meme.user.posts where pubid = "Nswwwmv"
![Page 22: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/22.jpg)
Reposting, Commenting
INSERT INTO meme.user.posts (guid, pubid, comment) VALUES ("S5R44PGJRBLKNEE5GYSRQPTXLQ", "rGCOBCK", "this is the repost comment”)
INSERT INTO meme.user.comments (guid, pubid, comment) VALUES ("S5R44PGJRBLKNEE5GYSRQPTXLQ", "rGCOBCK", "meme rocks")
![Page 24: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/24.jpg)
Exposing own API through YQL
<?xml version="1.0" encoding="UTF-8"?> <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta> <sampleQuery>select * from {table} where query=’q'</sampleQuery> </meta> <bindings> <select itemPath="results" produces="JSON" > <urls> <url env="all">http://query-url</url> </urls> <inputs> <key id=”q" type="xs:string" paramType="query" required="true" /> </inputs> </select> </bindings> </table>
![Page 25: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/25.jpg)
Now access it through YQL
USE "http://definition-file.xml" AS mytable;SELECT * FROM mytable WHERE query="whatever”
![Page 26: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/26.jpg)
Mashup - introduction
•Finding the Sunrise/Sunset time for Jakarta•Earthtools api for sunrise-sunset information
•XML based api•Needs latitude-longitude of the location
•Yahoo Geo API to convert location to latitute-longitude
•Would need 2 different API calls and effort from your side•Assuming Yahoo API was in JSON would have to deal with 2 formats
![Page 27: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/27.jpg)
Solution – Exposing 3rd party API in YQL
<?xml version="1.0" encoding="UTF-8"?> <table xmlns="http://query.yahooapis.com/v1/schema/table.xsd"> <meta> <author>Earthtools.org</author> <documentationURL>http://www.earthtools.org/</documentationURL> <sampleQuery>select * from {table} where lat='lat' and lon='lon' and day='day' and
month='month' </sampleQuery> </meta> <bindings> <select itemPath="sun" produces="XML" > <urls> <url env="all">http://www.earthtools.org/sun/{lat}/{lon}/{day}/{month}/99/1</url> </urls> <inputs> <key id="lat" type="xs:string" paramType="path" required="true" /><!– 3 others -->
</inputs> </select></bindings> </table>
![Page 28: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/28.jpg)
Querying for the data
# Get the Lat Long information for Jakartaselect * from geo.places where text="Jakarta" limit 1;
# Now get the sunrise/sunset from earthtools using a # YQL queryUSE "http://rahukalamcalculator.appspot.com/yql/earthtools-definition.xml" AS earthtools;SELECT morning, evening FROM earthtools WHERE lat="1.365580" and lon="103.827713" and day="21" and month="11”
![Page 29: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/29.jpg)
Combine the queries
# One awesome call!select * from mytable where (lat,lon) in (select centroid.latitude, centroid.longitude from geo.places where text="jakarta") and day="21" and month="11”
![Page 30: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/30.jpg)
Mashup – solution highlights
•Exposed a third party API through YQL•Expressed business logic in Query – readable, maintainable•Used server side for the actual join•Less latency•Use the format you like
![Page 31: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/31.jpg)
More goodies – YQL server side execution
•Executing JavaScript allows you to use conditional logic and to format data in a granular manner.
•Better data shaping and parsing•Better support for calling external Web service•Better support for adding, modifying, and deleting data using external Web
services•Lots of examples and detailed documentation at
http://developer.yahoo.com/yql/guide/yql-execute-examples.html
![Page 32: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/32.jpg)
Server Side execution - example
<bindings> <select itemPath="" produces="XML"> <urls>
…</inputs>
<execute><![CDATA[ // Your javascript goes here. We will run it on our servers response.object = <item> <url>{request.url}</url> <a>{a}</a> <b>{b}</b> </item>; ]]></execute> </select> </bindings>
![Page 33: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/33.jpg)
More goodies – YQL Hosted Storage
•Size Limit: You can have up to 1000 records, with each record being up to 100KB.
•Retention Limit: Records not read, updated, or executed at least once every thirty days may be removed.
•Record Format: Records must be in a text-based format. Examples include JavaScript code, XML files, Open Data Tables, or YQL environment files.
•Authentication for New Records: All connections to yql.storage.admin must be authorized using two-legged OAuth. Alternatively, you can create new records using the YQL console. However, connections to yql.storage do not require authentication.
![Page 34: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/34.jpg)
Storing into YQL
# Just the valueinsert into yql.storage.admin (value) values ("example text content")
# Contents of a URLinsert into yql.storage.admin (name,url) values ("newrecord","http://hostingdomain.com/mytable.xml")
# response looks like <results> <inserted> <execute>store://35ad2c72-e353-41c4-9d21-4a7e5a1ec92</execute>
<select>store://08fd2c74-d393-48c4-9ee1-3bf7e5a1ec92</select> <update>store://3cc85a99-6a89-4600-ade1-f7f83ecc4b83</update> </inserted> </results>
![Page 35: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/35.jpg)
Querying from hosted storage
# Uses select-idselect * from yql.storage where name="store://08fd2c74”
# Uses update-iddelete from yql.storage where name="store://3cc85a99”update yql.storage set value="new value" where name="store://3cc85a99”
# Use contents as env file (uses execute-id)use "store://35ad2c72" as mytable; select * from mytable;
# Use contents as JS for server side execute (uses execute-id)y.include('store://35ad2c72'); response.object = <success>{success}</success>;
![Page 36: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/36.jpg)
Concluding remarks
•YQL is an extensible gateway to use APIs across and beyond Yahoo!•Allows standardization of APIs across the web•Familiar and concise language for expressing business and data logic •Powerful tool to build mashups•You can build upon existing APIs and expose third party APIs•Sound server side support, including javascript execution and storage to enable
easy development
![Page 37: YQL talk at OHD Jakarta](https://reader038.fdocuments.net/reader038/viewer/2022103111/54bd639d4a7959a9278b460e/html5/thumbnails/37.jpg)
Questions?