Building an aws sdk for Perl - Granada Perl Workshop 2014
-
Upload
jose-luis-martinez -
Category
Technology
-
view
385 -
download
3
description
Transcript of Building an aws sdk for Perl - Granada Perl Workshop 2014
![Page 1: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/1.jpg)
Building an AWS SDK
Granada Perl Workshop 2014
Jose Luis Martinez
JLMARTIN on CPAN
Twitter: @pplu_io
![Page 2: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/2.jpg)
About me
CTO @CAPSiDE
Training Partner for AWS
Delivering AWS official training
All four AWS certifications
Real experience with real platforms
Deliver consulting and continuous operation
Writing Perl for +10 years
And still having fun with it!
![Page 3: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/3.jpg)
About AWS
Mostly known for EC2 and S3
But lot's more services! (+30)
Everything is an API
“Programmable datacenter”
Software Defined Everything
![Page 4: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/4.jpg)
About AWS
Mostly known for EC2 and S3 But lot's more services! (+30) Everything is an API “Programmable datacenter”
Software Defined Everything
It’s the your wet dream
![Page 5: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/5.jpg)
Perl & AWS
Bad news
No official SDK
Official ones: Ruby, PHP, Python, JS, .Net, Java
Note: python wasn't official initially!
![Page 6: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/6.jpg)
Perl & AWS
Good news
CPAN
And an active community
Lots of modules for different services (sometimes more than one)
![Page 7: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/7.jpg)
Perl & AWS: The CPAN
Inconsistencies
In method naming
Arbitrary defaults ('eu-west-1', 'us-east-1')
Not up to date with latest APIs
Keeping up with AWS is hard!!!
“Not so popular” services not covered
IAM, CloudFormation, SWF, RedShift, CloudSearch, etc
Almost no Role support
AWS::CLIWrapper is a good generic solution. Not native
![Page 8: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/8.jpg)
So lets write an entire SDK!
![Page 9: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/9.jpg)
![Page 10: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/10.jpg)
Hard enough
![Page 11: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/11.jpg)
You can end up with…
or
![Page 12: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/12.jpg)
Challenges: # of services
30+ services
Some with 100+ API calls
Each call with it’s parameters
Want to return objects
Evolution
![Page 13: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/13.jpg)
Challenges: # of services
![Page 14: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/14.jpg)
So lets start classifying
![Page 15: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/15.jpg)
Challenges: Endpoints
Endpoint Type Services
Regional Services available in a región
the rest
Single Global services 6
![Page 16: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/16.jpg)
Challenges: WebService Types
One URL REST
Params: QueryResponse: XML
17 3
Params: JSONResponse: JSON
10 1
![Page 17: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/17.jpg)
Challenges: Signatures
Type Services
v2 2
v3https 1
cloudfront 1
v4 the rest
![Page 18: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/18.jpg)
Challenges: API versions
More tan one API version is live
That’s why not-up-to-date callers still work
RDS
4 versions
CloudFront
3 versions
EC2
4 versions
![Page 19: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/19.jpg)
AWS Pace of innovation
Every month there are changes
Changes mean APIs change (slightly)
Very hard to keep up!
![Page 20: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/20.jpg)
AWS Pace of innovation
Every week there are announcements
Changes mean APIs change (slightly)
Very hard to keep up!
![Page 21: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/21.jpg)
Lets start writing!
![Page 22: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/22.jpg)
Hand write classes and methods, and parameters
![Page 23: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/23.jpg)
Hand write classes and methods, and parameters
![Page 24: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/24.jpg)
Parse docs
After all… you would have to read them to write the classes
![Page 25: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/25.jpg)
Parse docs
After all… you would have to read them to write the classes
![Page 26: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/26.jpg)
No published spec
Introspect an official SDK!
After all AWS already has read the docs for you
![Page 27: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/27.jpg)
No published spec Introspect an official SDK!
That’s your spec!
After all AWS already has read the docs for you
![Page 28: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/28.jpg)
Data driven SDKs!
JS SDK has datastructures that define calls
Proof of concept
Win++! Boto and PHP2 too!
Later migrate to using Boto
![Page 29: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/29.jpg)
Data driven SDKs!
JS SDK has datastructures that define calls
Proof of concept
Win++! Boto and PHP2 too!
Later migrate to using Boto
![Page 30: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/30.jpg)
The solutions
![Page 31: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/31.jpg)
Modern Perl to the rescue
Try to be very lazy Parameter validation: Create Moose classes that validate
the parameters passed to the call. May change in the future.
Got very quick results!
Hide all possible implementation from the user Try not to leak implementation
That way you can change it at will
Meta lets you inspect the objects! Call objects get marshalled into datastructured via meta
Result datastructures get marshalled into objects via meta
![Page 32: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/32.jpg)
Roles
![Page 33: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/33.jpg)
Modern Perl: Roles
All functionality gets pulled in via Roles
Code is basically in different roles, Classes are “serialized configurations” of how to call APIs
Makes it easy to
Generate service classes from datastructures
Change behaviour for tests
![Page 34: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/34.jpg)
Roles are used for
One global endpoint vs endpoints per region
Caller (Query, JSON)
Response (XML, JSON)
Signatures (V2, V3, V4)
IO
![Page 35: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/35.jpg)
Aws is a class factory
![Page 36: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/36.jpg)
Dynamic class construction
User never constructs service classes directly
Aws class method uses default config
my $ec2 = Aws->service(‘EC2’)->new( . . . )
![Page 37: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/37.jpg)
Dynamic class construction
User wants custom functionality
my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::TestCaller‘ ) );
my $ec2 = $aws->service(‘EC2’)->new( . . . )
service loads Aws::EC2, creates a new class with TestCaller
![Page 38: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/38.jpg)
Win: Hello to the async world
Still playing around in examples
my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::MojoCaller‘ ) );
my $ec2 = $aws->service(‘EC2’)->new( . . . )
my $asgs = $ec2->DescribeAutoScalingGroups;
See examples/asyncAutoScaling.pl
![Page 39: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/39.jpg)
Win: Also going “fully async”
my $aws = Aws->new( config => AWS::SDK::Config->new(caller => 'Net::AWS::MojoAsyncCaller'));
my $delay = Mojo::IOLoop->delay(sub { my ($delay, @responses) = @_; Dumper($_) for @responses;});
foreach my $region ( "us-east-1", "ap-northeast-1", "sa-east-1", "ap-southeast-1", "ap-southeast-2", "us-west-2", "us-west-1", "eu-west-1", 'invented-region') { my $ctrail = $aws->service('CloudTrail')->new( region => $region, delay => $delay, ); print "Doing a call for $region\n"; my $list = $ctrail->DescribeTrails;}
$delay->wait;
![Page 40: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/40.jpg)
![Page 41: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/41.jpg)
Lessons Learned
![Page 42: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/42.jpg)
Lessons learned
First shot with MooseX::Declare
Error messages are HORRIBLE!
Change to Moops didn't go well
Finally plain old Moose
![Page 43: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/43.jpg)
Lessons learned
Namespace finding (still in the works)
AWS vs Aws
Thanks @clintongormley for the article!
![Page 44: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/44.jpg)
Lessons learned
Enums (still in the works)
APIs sometimes return unexpected stuff
![Page 45: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/45.jpg)
Lessons learned
Abstract $request object passed around
Finally marshalled into the UserAgents’ taste
![Page 46: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/46.jpg)
Still open stuff
Load time for big services (EC2)
Not all classes will be used all the time
Generate classes on demand
Calling convention
Now CamelCased
Not so perlish
Calling from objects (not service objects)
$queue->DeleteQueue(); from a queue object
Attribute Traits
Access to Arrays and HashRefs via nicer methods
Lots more!!!
![Page 47: Building an aws sdk for Perl - Granada Perl Workshop 2014](https://reader033.fdocuments.net/reader033/viewer/2022061214/549bf6a6b47959ca318b45e7/html5/thumbnails/47.jpg)
Use AWS? Use Perl? Want to help?
Fork your heart out:
https://github.com/pplu/aws-sdk-perl/