Chris&an HeilmannFron0rends, Warsaw, Poland, October 2010

Using web services in JavaScript.

What is the web?

Was ist das Web?

Data + Interfaces

Lots of yummy, yummy data.

If a company has a bit of brains these days then they build on and release an API for their services.

Company benefits:

People bring your data and services into environments you could never reach.

Our benefits:

solve problems and get information without buying data or writing code.

Solving 2 problems.

Problem #1: Distance between two places on Earth.

Sometimes it would be good to know the distance between two places on Earth.

For this you could use any mapping service and navigate from one to the other.

But what if you want to have that as plain information?

Battle Plan:1. Find the location of the two places

on Earth

2. Calculate the distance.

Yahoo GeoPlanet is a data set that has information about the location of places on Earth.

Yahoo GeoPlanet is a data set that has information about the location of places on Earth.

http://developer.yahoo.com/geo/geoplanet/

http://where.yahooapis.com/v1/places.q('warsaw')?appid={appid}&format=json

This gives us the latitude and longitude of both places...

...but how do we calculate the distance?

http://www.movable-type.co.uk/scripts/latlong-vincenty.html

Putting it all together...

Putting it all together...

Problem #1: Foreign Tweets.

Twitter supports different languages, but has no translation service.

Google has a translation service though.

Battle Plan:1. Investigate Twitter’s search API and

Google’s translation API and if needed, get keys.

2. Get the results from Twitter for a certain search.

3. Loop over the results, see which ones are not in English, and then translate them with the Google Translation API.

The solution is very similar to the other solution.

It also suffers from the same issues.

Not fun with JS:1. Asynchronous lookups with

generated script nodes are a pain to get right - what if one breaks?

2. Depending on how many Tweets are not in English, you have to hammer Google’s translation API which slows down your overall app.

Simplifying access.

Using several APIs in conjunction in JavaScript is not fun.

It doesn’t matter if that is internally or externally in a company.

Yahoo had a lot of these problems...

...which is why they built YQL to work around that.

Using YQL, web services become databases you can access like any other DB.

YQL http://developer.yahoo.com/yql/console/

YQL http://developer.yahoo.com/yql/console/

select {what} from {where} where {conditions}

Foreign Tweets?

select text from twitter.search where q=”ft2010” and iso_language_code=”pl”

select * from google.translate where q in (select text from twitter.search where q=”ft2010” and iso_language_code=”pl”) and target=”en”

Keywords for Warsaw?

select * from search.web where query="warsaw"

select abstract from search.web where query="warsaw"

select abstract from search.web(100) where query="warsaw"

select * from search.termextract where context in (select abstract from search.web(100) where query="warsaw")

select * from search.termextract where context in (select abstract from search.web(100) where query="warsaw") | sort(field="Result") | unique(field="Result")

Photos of Warsaw?

select * from flickr.photos.search(50) where woe_id in (select woeid from geo.places where text="warsaw") and min_taken_date = "2009"

Instead of going crazy filtering and sorting in JS...

...use the YQL server and then have a very simple JS for displaying.

Using web services with YQL in JS.

YQL is a web service endpoint on its own...

https://query.yahooapis.com/v1/public/yql?q={uri-encoded-query}&format={xml|json}&diagnostics={true|false}&callback={function}&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys

As it supports JSON-P you can use it with dynamically created script nodes.

Special case: Scraping

http://www.flickr.com/photos/fdtate/4426760544/

http://www.slideshare.net/cheilmann/reasons-to-be-cheerful-fronteers-2010

select * from html where

url="http://www.slideshare.net/cheilmann/reasons-to-be-cheerful-fronteers-2010"

and

xpath="//ol/li/p[contains(.,'http')]"

http://y.ahoo.it/r/ENSPGm

http://lanyrd.com/people/codepo8/

HTML as JSON is not fun.

JSON-P-X = HTML as a string in a JSON-P container!

Using YQL re-use of web content is very easy indeed.

Be safe, be good...

Using JSON-P in JavaScript is easy.

Don’t rely on data arriving - test for it!

XML to JSON?

XML to JSON?

Using JSON is easy with libraries.

$.getJSON(url+'&callback=?', function(data){});

JSON-P and jQuery:

$.ajax({ url: url, dataType: 'jsonp', jsonp: 'callback', jsonpCallback: 'ohyeah'});function ohyeah(data){}

JSON-P and jQuery:

Which one to use?

getJSON() is dangerous with other people’s data.

http://{...}&format=json&callback=ohyeah

$.ajax():

$.getJSON():http://{...}&format=json&callback=jsonp1282497813335

Random number

Cachebreaking is not a good idea.

Local caching is a good idea though.

Cookies suck, though.

Would be good to have a better solution for that.

localStorage = cookies on steroids.

if(('localStorage' in window) && window['localStorage'] !== null){localStorage.setItem( 'cake', 'much better than cookies')

}

if(('localStorage' in window) && window['localStorage'] !== null){var what = localStorage.getItem( 'cake')// what -> 'much better than cookies'

}

localStorage only stores Strings - use JSON to work around that.

if(('localStorage' in window) && window['localStorage'] !== null){localStorage.setItem( 'cake', JSON.stringify( {yummy:‘yes’,candles:5} ));

}

if(('localStorage' in window) && window['localStorage'] !== null){var what = JSON.parse( localStorage.getItem('cake'));// what -> Object{...} // and not [Object object]

}

Let’s wrap this up in a function.

yql - the queryid - storage key namecacheage - how long to cachecallback - obvious, isn’t it?

Browsers supporting localStorage fetch the data every hour.

Others still work, but load the data every time.

OK, dann machen wir das halt..

OK, dann machen wir das halt..

OK, dann machen wir das halt..

There are a few libraries offering fallbacks to other browers via Flash.

You can of course also use those.

Offering your own API.

To get your own API into YQL all you need to do is write an XML schema and put it on GitHub.

http://github.com/yql/yql-tables

YQL allows you to write “executable tables”...

...which means you can convert data with JavaScript that will be executed server-side.

Twitter translate example:

Offering your own API.

Offering your own API.

SELECT * FROM twitter.translate WHERE language="en" and search="warsaw" and amount="20"

Distance example:

SELECT * FROM geo.distance WHERE place1=”london” and place2="warsaw"

http://isithackday.com/hacks/geo/distance/

Using your JS tables.

Write your schema, put it on the web and use, err... USE to, err... use it.

use “http://app.pl/distance.xml” as distance; SELECT * FROM distance WHERE place1=”london” and place2="warsaw"

Both problems solved and released as an API - in JS!

In summaryUse YQL instead of wasting time reading API docs for a simple taskFilter data in the service and get the info back in formats you need.Use the fast YQL server instead of doing lots of requests.Write your own JS APIs using execute.Use local storage and don’t break caching.Go and use the web.

Christian Heilmann http://wait-till-i.com http://developer-evangelism.com @codepo8

Cheers