Build Great Networked APIs with Swift, OpenAPI, and gRPC
-
Upload
tim-burks -
Category
Technology
-
view
509 -
download
3
Transcript of Build Great Networked APIs with Swift, OpenAPI, and gRPC
![Page 1: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/1.jpg)
Build Great Networked APIs with Swift, OpenAPI, and gRPCTim Burks, Google
Swift Cloud Workshop No. 2
September 30, 2017
![Page 2: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/2.jpg)
What do we do?
![Page 3: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/3.jpg)
WE WRITE CODE
![Page 4: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/4.jpg)
![Page 5: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/5.jpg)
What do we want?
![Page 6: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/6.jpg)
QUALTIY!
![Page 7: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/7.jpg)
Protocol Buffersa language-neutral, platform-neutral, extensible mechanism for serializing structured data.
![Page 8: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/8.jpg)
Interface Builder for Data
message Person {
string name = 1;
int32 id = 2;
string email = 3;
message PhoneNumber {
string number = 1;
}
repeated PhoneNumber phone = 4;
}
Interface Builder: Developers specify their interfaces using a special tool, tooling compiles and integrates that into their apps.
Protocol Buffers: Developers specify their data structures using a special language, tooling compiles and integrates that into their apps.
![Page 10: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/10.jpg)
Discovery Format Code Generators
![Page 12: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/12.jpg)
github.com/google/google-api-python-client
![Page 13: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/13.jpg)
Idiosyncracies and Lessons
![Page 14: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/14.jpg)
Toolkit and the next generation of Google APIs
![Page 15: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/15.jpg)
So far, so good.
REST (Discovery Format)
● 232 API descriptions at https://www.googleapis.com/discovery/v1/apis● 10 generators listed at https://developers.google.com/api-client-library/
gRPC (Toolkit)
● 30+ gRPC-based APIs● 7 target languages
![Page 16: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/16.jpg)
OpenAPI 3.0
Industry standard format for describing for REST APIs
Originally designed for documentation, now with many other applications: API Authoring, Validation, Documentation, Analysis, Search, Testing, Mocking, Management, Code Generation
![Page 17: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/17.jpg)
Consensus
![Page 18: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/18.jpg)
OpenAPI-based code generators
● swagger-codegen (Open source, Smartbear + community)○ 70+ targets○ First commit: July 6, 2011○ Used by Lyft and Square to generate SDKs
● AutoRest (Open source, Microsoft)● oas-nodegen (Open source, Capital One)● APIMatic (Proprietary, APIMatic)
more?
![Page 19: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/19.jpg)
![Page 20: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/20.jpg)
Problems we’ve seen so far in open source code generators
● Missing or weak build system integration.● Invalid service addresses (error in OpenAPI description).● No auth.● No documentation (not even a README!).● No samples.● No test harness.● No assurance that the called service even works.● Ugly generated code.
○ Machine-generated operation names.○ Machine-generated struct names.
● Unwanted dependencies.
![Page 21: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/21.jpg)
swagger-codegen often isn’t used “out of the box”:
“Generating client libraries involves customizing the provided language-specific templates…
The amount of modification each template needs varies by language and we’re looking forward to working with the Swagger Codegen community to share our refinements.” Val Polouchkine, Lyft
“...Swagger Codegen is a pretty active project. If you don’t check in your templates, things are gonna break unexpectedly because Swagger Codegen just uses the latest and greatest templates that are out there. So if you don’t sort of manually make sure that those things work, you’re gonna have an issue there.” Tristan Sokol, Square
![Page 22: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/22.jpg)
Code generation pipeline
API Description GeneratableAPI Description
Language-specific model
GeneratedAPI support
code
render idiomatic API support code according
to user preferences (re. build systems,
dependencies)
verify model, define structures and
entry points,name everything
filter language-specific reserved words,
(optionally) define file structure for generated code
![Page 23: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/23.jpg)
typical monolithiccode generation pipeline
API Description(OpenAPI)
GeneratableAPI Description(internal data
structures)
Language-specific model(internal data
structures)
GeneratedAPI support
code
All-in-one repo and package
![Page 24: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/24.jpg)
Problems with monolithic code generators
● Long build times: changing one target requires rebuilding everything.● Long test times: new builds must be tested for every target language.● For stability, teams may prefer to archive their own generator builds.● Forks will abound.● Quality is uneven.● Versioning is hard.● Complexity and potentially unfamiliar build systems deter contributors.
![Page 25: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/25.jpg)
protoccode generation pipeline
API Description(.proto)
GeneratableAPI Description
(binary FileDescriptors)
Language-specific model (internal data
structures)
GeneratedAPI support
code
language target-specific plugins
protoc
![Page 26: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/26.jpg)
protoc plugin definition
![Page 27: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/27.jpg)
Why does protoc have a plug-in architecture?
● Fast build times: changing one target only requires rebuilding its plugin.● Fast test times: new builds need only be tested for the affected targets.● For stability, teams can archive their own protoc and plugin builds.● New plugins can abound.● Separately-maintained plugins can offer different maturity levels.● Separately-maintained plugins can be appropriately versioned.● Separately-maintained plugins can be in languages that contributors
prefer.
![Page 28: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/28.jpg)
What’s the catch?
Plugins require a well-defined interchange format.
Fortunately, we have two great tools for that.
![Page 29: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/29.jpg)
gnostic
gnostic processed and verified protobuf representation of
OpenAPI description
protoc + pluginsOpenAPI
.proto
reusable data structures and reader for protobuf OpenAPI descriptions
gnostic apps and plugins
OpenAPIdescription
gnostic-generator
OpenAPI.proto and
compiler code
OpenAPIJSON
schema
![Page 30: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/30.jpg)
Kubernetes OpenAPI: .json vs .pb
Format Size Deserialization time Download time (at 80 Mbps)
Json 1653 KB >500 ms 165.3 ms
Proto binary 914 KB 9.3 ms 91.4 ms
Proto binary compressed 96 KB 13.5 ms 1.3 ms
Source: [email protected]
![Page 31: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/31.jpg)
gnosticcode generation pipeline
API Description(OpenAPI v3)
GeneratableAPI Description
(Normalized/Annotated
OpenAPI v3?)
Language-specific model(Normalized/
Annotated OpenAPI v3?)
GeneratedAPI support
code
target-specificplugins
gnostic +linter
![Page 32: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/32.jpg)
let anyone write code generation plugins...
● in their own repositories● with their own versioning● in whatever implementation language they choose
API code generation is a community problem that needs community-based solutions...
![Page 33: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/33.jpg)
Code generators should belong to their communities.
![Page 34: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/34.jpg)
gRPC
![Page 35: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/35.jpg)
@grpcio
RPC: Use Cases
Direct RPCs:Microservices
RPCs to access APIs
Google APIs
OSS APIs
MobileWeb
DesktopRPCs
Datacenters Cloud
Service 1
Service 2
Service 3
Service 4Containers
![Page 36: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/36.jpg)
@grpcio
What is gRPC?● HTTP/2 based RPC framework developed by Google● Open, Multiplatform, Secure, Performant
Multiplatform
● Idiomatic APIs in popular languages (C++, Go, Java, Python, Node.js, C#, Ruby, PHP)● Supports mobile devices (Android Java, iOS Obj-C, Swift)● Linux, Windows, Mac OS X● (web browser support in development)
OpenSource
● developed fully in open on GitHub: https://github.com/grpc/
![Page 37: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/37.jpg)
@grpcio
● Builds on Apple’s swift-protobuf and grpc-core.● Includes:
○ gRPC framework (C and Swift components)○ generated code surface○ protoc plugin for code generation
● Full-service gRPC:○ All four gRPC API styles are supported.○ gRPC framework supports both clients and servers.○ Plugin generates client and server code in separate files.○ Testing on MacOS and Ubuntu.
● Audiences: client and server developers, inside/outside Google
gRPC for Swift (https://github.com/grpc/grpc-swift)
![Page 38: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/38.jpg)
@grpcio
package echo;
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
// Splits a request into words and returns each word in a stream of messages.
rpc Expand(EchoRequest) returns (stream EchoResponse) {}
// Collects a stream of messages and returns them concatenated when the caller closes.
rpc Collect(stream EchoRequest) returns (EchoResponse) {}
// Streams back messages as they are received in an input stream.
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
// The text of a message to be echoed.
string text = 1;
}
message EchoResponse {
// The text of an echo response.
string text = 1;
}
gRPC Swift sample service
![Page 39: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/39.jpg)
@grpcio
/// To build a server, implement a class that conforms to this protocol.
public protocol Echo_EchoProvider {
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse
func expand(request : Echo_EchoRequest, session : Echo_EchoExpandSession) throws
func collect(session : Echo_EchoCollectSession) throws
func update(session : Echo_EchoUpdateSession) throws
}
gRPC Swift server protocol
![Page 40: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/40.jpg)
@grpcio
// get returns requests as they were received.
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse {
return Echo_EchoResponse(text:"Swift echo get: " + request.text)
}
...
// update streams back messages as they are received in an input stream.
func update(session : Echo_EchoUpdateSession) throws -> Void {
while true {
do {
let request = try session.Receive()
try session.Send(Echo_EchoResponse(text:"Swift echo update: \(request.text)"))
} catch Echo_EchoServerError.endOfStream {
break
}
}
try session.Close()
}
}
gRPC Swift server sample
![Page 41: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/41.jpg)
@grpcio
gRPC Swift unary client sample // Unary
if client == "get" {
var requestMessage = Echo_EchoRequest(text:message)
let responseMessage = try service.get(requestMessage) // blocking
print("get received: " + responseMessage.text)
}
![Page 42: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/42.jpg)
@grpcio
gRPC Swift bidirectional streaming client sample (1/2) // Bidirectional streaming
if client == "update" {
let sem = DispatchSemaphore(value: 0)
let updateCall = try service.update() // blocking
DispatchQueue.global().async {
while true {
do {
let responseMessage = try updateCall.Receive() // blocking
print("Received: \(responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
sem.signal()
break
}
}
}
...
![Page 43: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/43.jpg)
@grpcio
gRPC Swift bidirectional streaming client sample (2/2)...
let parts = message.components(separatedBy:" ")
for part in parts {
let requestMessage = Echo_EchoRequest(text:part)
try updateCall.Send(requestMessage)
sleep(1)
}
try updateCall.CloseSend()
// Wait for the call to complete.
sem.wait()
}
![Page 44: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/44.jpg)
![Page 45: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/45.jpg)
gRPC-Swift TODO
● Build system integration○ Package Manager○ Cocoapods?○ Carthage?
● gRPC interoperability tests● Samples that wrap Google APIs
○ Google Cloud Speech API○ Google Datastore API
github.com/grpc/grpc-swift/issues
![Page 46: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/46.jpg)
More gRPC Information
Website: http://grpc.io
Sources: https://github.com/grpc/grpc
Mailing list: https://groups.google.com/d/forum/grpc-io
Ecosystem: https://github.com/grpc-ecosystem
![Page 47: Build Great Networked APIs with Swift, OpenAPI, and gRPC](https://reader031.fdocuments.net/reader031/viewer/2022021422/5a6478db7f8b9a40568b4617/html5/thumbnails/47.jpg)
github.com/googleapis/gnosticplugins for OpenAPI-based code generation
github.com/grpc/grpc-swiftFast streaming APIs in Swift
github.com/google/auth-library-swiftOAuth support for Google Cloud
http://twitter.com/[email protected]