Restful Patterns
-
Upload
patrice-cappelaere -
Category
Documents
-
view
238 -
download
0
Transcript of Restful Patterns
-
8/4/2019 Restful Patterns
1/71
Hosted and Sponsored by
Gold Sponsor
Copyright 2011 Open Geospatial Consortium
OGC RESTful Services
Common Patterns78th OGC Technical CommitteeBoulder, Colorado (USA)
Pat Cappelaere
September 22d, 2011
-
8/4/2019 Restful Patterns
2/71
OGC RESTful ServicesCommon PatternsPat Cappelaere
Panagiotis (Peter) A. Vretanos
Volker Mische
Version 14
9/21/2011
With feedback from:
Arnulf ChristiSimon CoxJohn HerringCarl ReedRaj SinghPeter SchutDon Sullivan
-
8/4/2019 Restful Patterns
3/71
OGC TC Boulder Sept 22, 2011
3
This presentation is Not about WhatRESTis
The current standards need to evolveand meet the needs of a large andgrowing community,
This is What We are Hoping RESTcanDoforOGCServices
Not about SOA vs. ROA either
-
8/4/2019 Restful Patterns
4/71
OGC TC Boulder Sept 22, 2011
Agenda
Part 1
Business Case
ROA
CRUD
Custom REST Actions
Resources, Patterns & Facades
Benefits
4
Part 2
Multi-faceted Example
Asynchronous Transactions Resource Representation
Computational Processes
Discovery
Caching (Etags)
Pubsub & Notifications Data & Activity Feeds
RESTful SecurityBackups Languages & Genomics WFS Queries, SPS, WPS, WCPS, WCS, WMS, SOS examples
-
8/4/2019 Restful Patterns
5/71
OGC TC Boulder Sept 22, 2011
What we want to see are very simple standards,and some of the OGC standards are verycomplicated. If I had one plea to that community itwould be to simplify standards, otherwise people
wont adopt and deploy them and put them intopractice.
5
Jacqueline McGlade
Executive Director European Environment Agency
And Newest OGC Board Member
Business Case: What Some Customers Want
-
8/4/2019 Restful Patterns
6/71
-
8/4/2019 Restful Patterns
7/71
OGC TC Boulder Sept 22, 2011
Whats in Demand Right Now? 7
Node.js = Javascript on server-side (using asynchronous eventing)
This will have a big impact to new web service development
Upcoming NewDevelopments
Act as Enablers
for Web Services
We need to pay attention to this!
Javascript -> JSON and -> Async Services
-
8/4/2019 Restful Patterns
8/71
OGC TC Boulder Sept 22, 2011
OGC ROA Business Case
OGC Language Has Naturally Evolved TowardsIncreased Complexity Over The Years
And Entering A Period Of Modernization Of LanguageTowards Simplification
We Can Take A Step Back And Look Across All Services
Because We Have Learned Something
We Have A Unique Chance At Harmonization Of Services
And Make It Simpler, Faster To Learn And Cheaper To Use
8
-
8/4/2019 Restful Patterns
9/71
Opportunity to Use Consistent Patterns Across ALLServices
No New Operations or Verbs/Words
(such asGetCapabilities, SubmitTask, ExecuteProcess,
ProcessCoverage, DescribeProcess, DescribeData,GetStatus)
Focus on Core Concerns
OGC TC Boulder Sept 22, 2011
Resource Oriented Architecture9
There are still some semantics issues and that terms
and definitions for OGC usage need to be
finalized but not a topic for this presentation
And Cross Cutting Concerns
Caching (Etags) Pubsub and Notifications
Discovery
Search
Security...
-
8/4/2019 Restful Patterns
10/71
OGC TC Boulder Sept 22, 2011
Resource Oriented Architecture
Everything is a Resource (and has a URL)
What is a Resource?
Something that a user can create[, read,
update, delete] Use Simple HTTP methods ONLY: POST, GET,
PUT, DELETE
Focus on User and its access to the data(not operations)
Keep it Simple
GOAL: Client Implementation in a few hours
(Or NeoGeographers and other datamashers will move on to something else)
10
OBVIOUS
This is not ROA vs. SOA but given ROA
-
8/4/2019 Restful Patterns
11/71
OGC TC Boulder Sept 22, 2011
Custom REST Actions
Still possible
At collection level
/sps/tasks/completed
At resource level
/sps/tasks/1/complete
/sps/tasks/1/edit or new
Could be used for
Halt, resume, abort
11
Lets start thinking in URLs, resources, and APIs Geoffrey Grosenbach, peepcode.com
Theres good URL design, theres bad URL design, and theres everything in
between its subjective -- Kyle Neath, warspire.com
NOT OBVIOUS
-
8/4/2019 Restful Patterns
12/71
OGC TC Boulder Sept 22, 2011
OGC Resource Patterns
WPS
Processes
SPS
Tasks
Feasibilities
WFS
Features
Queries
WCPS
Processes
Algorithms
WfCS
Workflows
WCS Coverages
WMS Maps
We need common patterns to build our services
SOS
Observations
Pubsub/WNS
Notifications
Subscriptions
12
Core Concerns
-
8/4/2019 Restful Patterns
13/71
OGC TC Boulder Sept 22, 2011
From Simple OGC Services To Multi-facetedApplications
13
WFS
SPS
WMS WPS
WCS WCPS
WNS
WCS
SPS
WMS
WMS WfCS
WCS
WPS
WNS SOS
WMS
WFS WCTS
Users can now look at data different
ways using Facades
We need harmonization and common patterns..
-
8/4/2019 Restful Patterns
14/71
OGC TC Boulder Sept 22, 2011
Benefits
Faade APIs can be used for OGC services (WFS,WPS, WCPS, WfCS, SPS)
NO New WORDS/VERBS to be defined
(No GetCapabilities, SubmitTask, ExecuteProcess,ProcessCoverage, DescribeProcess, DescribeData,GetStatus)
Consistent across services
Simpler To Learn
Faster to Use
Therefore Cheaper for this growing market
14
A Simplified Experience!
-
8/4/2019 Restful Patterns
15/71
OGC TC Boulder Sept 22, 2011
What we want to see are very simple standards,and some of the OGC standards are verycomplicated. If I had one plea to that community itwould be to simplify standards, otherwise people
wont adopt and deploy them and put them intopractice.
15
Jacqueline McGlade
Executive Director European Environment Agency
And Newest OGC Board Member
Simpler To Meet Some Customer Expectation
-
8/4/2019 Restful Patterns
16/71
OGC TC Boulder Sept 22, 2011
Agenda
Part 1
State of the World
ROA CRUD
Custom REST Actions
Resources, Patterns & Facets
Benefits
16
Part 2
Multi-faceted Example
Asynchronous Transactions Resource Representation
Computational Processes
Discovery
Caching (Etags)
Pubsub & Notifications Data & Activity Feeds
RESTful SecurityBackups Languages & Genomics WFS Queries, SPS, WPS, WCPS, WCS, WMS, SOS examples
ROA has to address:
-
8/4/2019 Restful Patterns
17/71
OGC TC Boulder Sept 22, 2011
From Stovepipe Services to
Facades
17
WFS
SPS
WMS WPS
WCS WCPS
WNS
WCS
SPS
WMS
WMS WfCS
WCS
WPS
WNS SOS
WMS
WFS WCTS
Users can now look at data different
ways using Facades
Leveraging harmonization and common patterns..
http://en.wikipedia.org/wiki/Facade_pattern
-
8/4/2019 Restful Patterns
18/71
OGC TC Boulder Sept 22, 2011
Modis Flood Extent Server
User Stories
User requests flood extent for tomorrow(coincidental with an EO-1 imaging)
SPS Facade
User requests flood extent overlay
WFS Facade
User request flood extent from last 15 days
WCS Time series
User requests prediction for next 8 days
WPS Facade
18
Facade: Provide a unified
interface to a set ofinterfaces in a
subsystem. Facade
defines a higher-level
interface that makes
the subsystem easier
to use. [GoF, p185] Wrap a complicated
subsystem with a
simpler interface.
API Must Be Consistent
-
8/4/2019 Restful Patterns
19/71
OGC TC Boulder Sept 22, 2011
Facade Implementation
Separation of Concerns
Core Concerns
Specialized Resources of Interest to Users Optional but Recommended Cross-Cutting Concerns
Security, Logging, DIscovery, Search, Pubsub
Use HTTP Operations
Result as an Atom/JSON Feed with links to Data
19
-
8/4/2019 Restful Patterns
20/71
OGC TC Boulder Sept 22, 2011
WCPS Faade Design Example 20
Core Concerns Cross Cutting Concerns
Optional But Recommended
WCPS Faade API
Logging Security
/
/WCPS
-
8/4/2019 Restful Patterns
21/71
OGC TC Boulder Sept 22, 2011
WFS Faade Design Example 21
Core Concerns Cross Cutting Concerns
Optional But Recommended
WFS Faade API
Logging Security
FeaturesFeatureType1FeatureType2..
/
/WFS
-
8/4/2019 Restful Patterns
22/71
OGC TC Boulder Sept 22, 2011
Data Interchange Protocol
HTML (of course, if we deal with users directly)
XML (May not be the easiest approach in many cases)
JSON (Emerging as de-facto standard)
Smaller footprint & easier to work with than XML
Javascript Native on Client Side
Javascript is becoming a language of choice on the serverside as well (Node, V8 engine)
Atom Because of browser native support and readers availability
Also Used for Notifications
Easy to extend -> Common Wrapper for Data Products
OGC Examples: GeoSync
22
About JSON feeds and XML feeds
Common Patterns
-
8/4/2019 Restful Patterns
23/71
OGC TC Boulder Sept 22, 2011
Alternate Representations Pattern
Using request headers (if you are a machine)
Accept
Accept-Encoding
Implicit using uri extension /geoactivities.json
/tasks.atom
Explicit using parameter
/geoactivities?fmt=json
/geoactivities?output=json
/geoactivities?alt=json
23Common Patterns
-
8/4/2019 Restful Patterns
24/71
OGC TC Boulder Sept 22, 2011
We seem to have gotten bogged down in queries, which arejust a simple example of the more general case of information
created by a computational process, where the computationalprocess is written/created by the client.
Raj Singh 9/9/2011
24
Computational processes created by client could be as a result of
executing user-defined WCPS algorithms, WPS-T processes, WfCS
workflows and/or WFS stored queries.
[Note: A simpler case exists when processes are hard-coded in the server.
WPS processes and SPS Tasks for example. No need for user uploads]
OGC Computational Processes
-
8/4/2019 Restful Patterns
25/71
-
8/4/2019 Restful Patterns
26/71
-
8/4/2019 Restful Patterns
27/71
OGC TC Boulder Sept 22, 2011
Step 2 - Execute
User creates a Handle by POSTing to relevantcollection
>POST /Handles
{ name: myHndl,
what: myThing,
Param1: xyz}
System returns 201, created and location URL
27
Handle CollectionsReplace with:
tasks
processes
transactions
jobs
This pattern applies to simpler case when processes are hard-coded (WPS
and SPS)
User wants to execute desired query/algo/process/task
What I want to execute
With opt parms
-
8/4/2019 Restful Patterns
28/71
OGC TC Boulder Sept 22, 2011
Step 3 Poll / Notify
User can then get a representation of thatresource in some format and poll for status
>GET /Handles/2345.json
{ name: myHndl,
what: myThing,
param1: xyz
status: started }
Or get notified via Pubsub when data isavailable
28
Not Recommended
Recommended
-
8/4/2019 Restful Patterns
29/71
OGC TC Boulder Sept 22, 2011
Step 4 Get Data
User can then get access to data when complete
>GET /Handles/2345.json
{ name: myHndl,
what: myThing,
param1: xyz
status: completed,
link: http://example/product.atom }
29
Resource Representation in
JSON format (but could be XML,
GML)
Link to actual data products
with additonal metadata,
mime-typesWARNING: This is the case where resource
representation is not the data itself
-
8/4/2019 Restful Patterns
30/71
OGC TC Boulder Sept 22, 2011
WFS Queries, WCPS Algorithms, WfCS
Workflows and Transaction Pattern30
Query or
Algorithm or
WorkflowPOST
GET1
Handle (or Transaction)
POST
GET2StatusLink
Atom feed
LinkLink
PNG
KML
Attrs
Create
Submit
For Execution
3 Wait for NotificationsGet Results4
This is NOTjust a User Interface
Only 2 resources involved!
Examples
-
8/4/2019 Restful Patterns
31/71
OGC TC Boulder Sept 22, 2011
So, Complex TransactionalProcesses
2 POST and 1 GET
[First POST may be optional for hardcoded processes]
31
-
8/4/2019 Restful Patterns
32/71
OGC TC Boulder Sept 22, 2011
Simple Queries.
1 GET[See OpenSearch]
32
More OGC Specific Examples in Backups
-
8/4/2019 Restful Patterns
33/71
OGC TC Boulder Sept 22, 2011
Common Discovery Pattern
Discovery Options
http OPTIONS / HEAD methods are weak (and require
many roundtrips) The Approach that the Google Discovery Service uses
is a better option to tie methods and representations
Client Libraries exist for many languages
JSON Schema necessary for Discovery & Validation
Natural to Javascript Client/Server sides (See Slide7)
Machine Readable as well
Examples
33
What I POST What I GET or PUT
-
8/4/2019 Restful Patterns
34/71
OGC TC Boulder Sept 22, 2011
Auto-Discovering the Discovery
Documents
Auto-Discovering the OpenSearch Description Documents Example What Description Document?
Value: Custom Search Within the Browser
Common Browser Experience
Auto-Discovering the Discovery Service Use HTML Page Headers!
https://portal.opengeospatial.org/files/?artifact_id=44919
Auto-Discovering PuSH Servers (See Annotations) Use Feed Links
Examples
34
-
8/4/2019 Restful Patterns
35/71
OGC TC Boulder Sept 22, 2011
ETags
For Efficiency / Cache Control
Dont send data that has not changed
For Data Integrity
Dont try to update data that has changed
Examples:
HTTP Header
Feed
35
HTTP/1.1 200 OK
Date: Fri, 30 Oct 1998 13:19:41 GMTServer: Apache/1.3.3 (Unix)
Cache-Control: max-age=3600, must-revalidate
Expires: Fri, 30 Oct 1998 14:19:41 GMT
Last-Modified: Mon, 29 Jun 1998 02:28:12 GMT
ETag: "3e86-410-3596fbbc"
Content-Length: 1040
Content-Type: text/html
Foo2006-01-23T16:25:00-08:00http://www.example.com/myFeed
Jo March
-
8/4/2019 Restful Patterns
36/71
-
8/4/2019 Restful Patterns
37/71
-
8/4/2019 Restful Patterns
38/71
OGC TC Boulder Sept 19th 2011
38
Send an POST request to http://geobliki.superfeedr.com
hub.mode:! publishhub.url: ! http://geobpms.geobliki.com/geoactivities.atom
PING for OGC Services
Send an POST request to http://superfeedr.com/hubbub, with HTTP Basic
Authentication and the following params :
hub.mode: ! subscribe or unsubscribehub.verify: ! sync or asynchub.callback: ! http://domain.tld/your/callbackhub.topic: ! http//feed.you.want.to/subscribe/tosuperfeedr.digest: true if you want to receive daily digest notifications.
SUBSCRIBE/UNSUBSCRIBE for Clients
Simple API
-
8/4/2019 Restful Patterns
39/71
-
8/4/2019 Restful Patterns
40/71
OGC TC Boulder Sept 19th 2011
40Notifications via Email via Chat
-
8/4/2019 Restful Patterns
41/71
-
8/4/2019 Restful Patterns
42/71
OGC TC Boulder Sept 22, 2011
[Geo]Activity Feeds
We need an acceptable Geo Extension for location
Activity = Subject + Verb + Object/Target + Location
For Microblogging and Notifications
The Activity Streams format has already been adopted by
BBC, Gnip, Google Buzz, Gowalla, IBM, MySpace, Opera,
Socialcast, Superfeedr, TypePad, Windows Live, YIID, and
many others.
42
Links:
http://activitystrea.ms/
http://groups.google.com/group/activity-streams
Lets go beyond data feeds
-
8/4/2019 Restful Patterns
43/71
OGC TC Boulder Sept 22, 2011
Activity Example
43
-
8/4/2019 Restful Patterns
44/71
OGC TC Boulder Sept 22, 2011
Security
You cannot design web services withoutaddressing security issues
It has to be built-in!
But not obtrusive
Using concepts that the user is familiar with
44
-
8/4/2019 Restful Patterns
45/71
OGC TC Boulder Sept 22, 2011
Distributed Identity We may need to access users from other
organizations
You may need to establish trusted relationships
with other Identity Providers
Users need to be authenticated (sometimes withtwo or three factor authentication) and havevarious access control rights
Users need to be able to delegate their authoritiesto workflows
Consumer Applications have their Identity
45
-
8/4/2019 Restful Patterns
46/71
OGC TC Boulder Sept 22, 2011
46
Organization 1
Thats
-
8/4/2019 Restful Patterns
47/71
OGC TC Boulder Sept 22, 2011
47
Organization 1
Organization 2
Trust
Relationship
Organization 3
More Realistic Scenario!
-
8/4/2019 Restful Patterns
48/71
OGC TC Boulder Sept 22, 2011
OpenID / OAuth Hybrid
Users and Consumers Have OpenIDs
Consumers need to register with the OP (and
publish their Public Key)
User Information, Access Control Privileges andKeys are managed by the OP using SimpleRegistration and Attribute Exchange
48
-
8/4/2019 Restful Patterns
49/71
OGC TC Boulder Sept 22, 2011
THANK YOU!
Pat Cappelaere
Panagiotis (Peter) A. Vretanos
Volker Mische
49
-
8/4/2019 Restful Patterns
50/71
OGC TC Boulder Sept 19th 2011
BACKUPS
50
Languages & Genomics WFS Queries, SPS, WPS, WCPS, WCS,
WMS, SOS examples
-
8/4/2019 Restful Patterns
51/71
OGC TC Boulder Sept 22, 2011
How Many Specifications does OGC have?
How Are Specifications Being Developed?
What is The Total Number of Pages?
What is The Total Number of SOAP Verbs?
How Has This Evolved Over The Years?
How Many Schemas Do We need to Master For ASignificant Deployment?
51
Status Check
-
8/4/2019 Restful Patterns
52/71
OGC TC Boulder Sept 22, 2011
52Similar Language Evolution or
Genomes for that matter
http://www.nature.com/embor/journal/v7/n8/full/7400756.html#B4
One would need to know about 22,000 characters to attain 99.99%
coverage of ancient Chinese
-
8/4/2019 Restful Patterns
53/71
OGC TC Boulder Sept 22, 2011
Evolution Phase 2
Modern Chinese
Only 4,600 characters are needed to attain 99.99% coverage
53
The current trajectory of OGC is unsustainableToo many schemas operationsspecs
We need to modernize the approach
This is What I am Hoping RESTcanDoforOGCServices
-
8/4/2019 Restful Patterns
54/71
-
8/4/2019 Restful Patterns
55/71
OGC TC Boulder Sept 22, 2011
55
Se curi ty
Client
WCPS
FacadeGeoActivitiesAlgori thms
Push
Serve rProcesses
POST
201
Client
GET
200
WCPS POST / GET Processes (to execute specific algorithm and get data)
Subscri ber
Client
Pro cess Comp letedPING
Notification
Security Transactions with OpenID Server ignored
Text
-
8/4/2019 Restful Patterns
56/71
OGC TC Boulder Sept 22, 2011
56
Se curi ty
Client
WCPS
FacadeGeoActivitiesQueri es PublisherFeatures
POST
201
Client
GET
200
WFS POST / GET Stored Queries
Su bscri ber
Security Transactions with OpenID Server ignored
WFSFacade
WFS POST / GET J b ( ifi d d )
-
8/4/2019 Restful Patterns
57/71
OGC TC Boulder Sept 22, 2011
57
Se curi ty
Client
WCPS
FacadeGeoActivitiesFeatures
Push
Serve rJobs
POST
201
Client
GET
200
WFS POST / GET Job (to execute specific query and get data)
Su bscri ber
Client
Job Comp leted
PING
Notification
Security Transactions with OpenID Server ignored
WFSFacade
-
8/4/2019 Restful Patterns
58/71
OGC TC Boulder Sept 22, 2011
WFS Query Example
User creates the query >POST /wfs/queries
{ name: myQuery,
query=SELECT ?parcel ?hwy
WHERE { ?parcel rdf:type my:CommercialParcel .?hwy rdf:type my:Street .?parcel ogc:touches ?hwy
}
}
System returns 201 created, location /wfs/query/fbs4769
>GET /wfs/query/fbs4769.json to get it back in json format (or usePUT/DELETE to edit/delete)
{ name: myQuery,
query=SELECT ?parcel ?hwyWHERE { ?parcel rdf:type my:CommercialParcel .
?hwy rdf:type my:Street .?parcel ogc:touches ?hwy }
}
58
GeoSPARQL
example
But could be WFSQuery, SQL, XQL,
XQUERY
-
8/4/2019 Restful Patterns
59/71
OGC TC Boulder Sept 22, 2011
Next steps
Execute the query
>POST /wfs/jobs
{ query: myQuery
output: kml }
System returns 201 created, location /wfs/jobs/169
Poll for status (or rather wait for notification)
>GET /wfs/jobs/169.json
{ query: myQuery
output: kml,
created_at: 9/10 2011
status: processing }
59
-
8/4/2019 Restful Patterns
60/71
-
8/4/2019 Restful Patterns
61/71
-
8/4/2019 Restful Patterns
62/71
-
8/4/2019 Restful Patterns
63/71
OGC TC Boulder Sept 22, 2011
SPS Feasibilities (Story #1)
Submit the feasibilities request
>POST /sps/feasibilities
{
latitude: -89.1,
longitude: 112.34 }
System returns 201, created, location: /sps/feasibilities/539 (for example)
Poll for status and eventually get the data (or get notified)
>GET /sps/feasibilities/539.json
{ ..
latitude: -89.1, longitude: 112.34,created_at: 9/10 2011 10:30:02,
updated_at: 9/10 2011 10:50:02
status: completed,
link: http://www.example.com/sps/feasibilities/products/2456.atom}
63
a 1 POST, 1 GET as well
Example for illustration purpose. Will
depend on actual asset to task
-
8/4/2019 Restful Patterns
64/71
OGC TC Boulder Sept 22, 2011
Feasibilities feed
Feed could return
Not feasible if it cant be done given inputs
Or N entries Each entry could contain:
Link to prepopulated form with additional
parameters that might be needed for a task
request.
User could then simply click on chosen
feasibility to submit task (HATEOAS)
Machine could just get parameters back and
use micro-template to POST request
64
-
8/4/2019 Restful Patterns
65/71
-
8/4/2019 Restful Patterns
66/71
-
8/4/2019 Restful Patterns
67/71
OGC TC Boulder Sept 22, 2011
WCPS Examples
User Stories User adds/edits custom algorithms using some representation
User deletes algorithms
User executes algorithms User gets data
>POST /wcps/algorithms Some resource entry in some format (JSON, XML)
>DELETE /wcps/algorithms/:id
>POST /wcps/processes Some resource entry in some format
>GET /wcps/processes/:id.:fmt?
67
-
8/4/2019 Restful Patterns
68/71
OGC TC Boulder Sept 22, 2011
WCS Examples
User Stories
User lists existing coverages
User retrieves specific coverage User search for specific coverage
>GET /wcs/coverages
>GET /wcs/coverages/:id
>GET /wcs/coverages/:filter [OpenSearch pattern]
68
-
8/4/2019 Restful Patterns
69/71
OGC TC Boulder Sept 22, 2011
WfCS Examples
User Stories User adds/edits workflows using some representation
User deletes workflows
User executes workflows
User gets data
>POST /wcps/workflows Some resource entry in some format (JSON, XML)
>DELETE /wcps/workflows/:id
>POST /wcps/processes Some resource entry in some format
>GET /wcps/processes/:id.:fmt?
69
-
8/4/2019 Restful Patterns
70/71
OGC TC Boulder Sept 22, 2011
WMS Example
User Stories:
User retrieves a particular map for location and zoom
level
> GET /wms/map/:mapid/:filter?
70
OpenSearch Pattern
-
8/4/2019 Restful Patterns
71/71
OGC TC Boulder Sept 22, 2011
SOS Example
User Stories
User retrieves list of observation
User retrieve specific observation
User searches for an observation
>GET /sos/observtions
>GET /sos/observations/:id
>GET/sos/observations/:filter [OpenSearch Pattern]
71