How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on...
Transcript of How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on...
![Page 1: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/1.jpg)
![Page 2: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/2.jpg)
How we run GraphQL APIs in production on our (own) Kubernetes cluster
@
![Page 3: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/3.jpg)
@ultrabug
Gentoo Linux developerPSF contributing memberCTO at Numberly
![Page 4: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/4.jpg)
Couldn’t you have more buzz words in your talk title?
![Page 5: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/5.jpg)
Previous workflow and its limitations
![Page 6: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/6.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
![Page 7: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/7.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
![Page 8: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/8.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
![Page 9: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/9.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
![Page 10: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/10.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
ansible
![Page 11: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/11.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
ansible
SSL offloading
![Page 12: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/12.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviews
YAML configuration file
ansible
SSL offloading
![Page 13: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/13.jpg)
Why Kubernetes?
![Page 14: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/14.jpg)
ur own bare-metal Kubernetes cluster
![Page 15: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/15.jpg)
54321 3 4 5
Methodology
Build a b
are cl
uster
Decide o
n the to
oling
Writ
e docu
menta
tion
Foster a
nd support
adoption
Distrib
ute expert
ise
![Page 16: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/16.jpg)
A bare-metal Kubernetes cluster?
- Package it to deeply know what’s it’s made of and how it works
- Automate installation, configuration, provisioning… everything!
![Page 17: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/17.jpg)
Developer-driven
![Page 18: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/18.jpg)
OpenID authentication
● Developer goes to internal kubeconfig URL
● Login using usual Google Suite account (openID) + free MFA (Yubikey)
● Download Kubeconfig
● Welcome to Kubernetes!
![Page 19: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/19.jpg)
Gitlab based authorization
- Gitlab based RBAC + Pod Security Policy since day 1- 1 namespace = 1 team
- Open sourced gitlab2rbac: https://github.com/numberly/gitlab2rbac
![Page 20: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/20.jpg)
Cluster capabilities and choices
- Gitlab registry for our Docker containers- Ensure only whitelisted images can be deployed
- runAsNonRoot + strict Network Policies enforced
- Ingress using nginx-ingress with fully automated LetsEncrypt certificate lifecycle
- Multi-tenant cluster supporting all environments (production, staging, development)
- Special “sandbox” namespace to test things:
- No distributed persistent storage yet
-
![Page 21: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/21.jpg)
A workflow-oriented documentation
![Page 22: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/22.jpg)
Foster and scale Kubernetes adoption
We created an internal Kubernetes Certification
● To make sure that in every team someone can help with Kubernetes● To help everyone identify who can support them when they need a Kubernetes expert● To value the expertise of members of our teams
![Page 23: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/23.jpg)
T ke Away
![Page 24: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/24.jpg)
T ke Away
● Gitlab for RBAC and image registry + Kubernetes = gitlab2rbac
● Balance security vs freedom: not opposed all the time!
● Enforce security and QA rules from the start○ TODO: work on admission controller to enforce whitelisted images only
● Ops concentrate on features that are immediately available to all devs○ TODO: automate F5 ingress SSL setup for public services
● Practical and useful docs are key
● Spread expertise to foster and scale adoption○ TODO: create more certification levels
![Page 25: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/25.jpg)
Our Kubernetized workflow
![Page 26: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/26.jpg)
Code repositoriesConfiguration repositoriesContinuous IntegrationCode reviewsUsers roles = k8s RBACGroups = k8s namespacesDocker image registry
YAML kubernetes deployment
SSL offloading
ingress-
Moved to k8s secrets
Needs Dockerfile
![Page 27: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/27.jpg)
Let’s build a GraphQL app on Kubernetes!
![Page 28: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/28.jpg)
Demo app: Trello REST API to GraphQL
GraphQL
REST
![Page 29: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/29.jpg)
Demo app: Trello REST API to GraphQL
GraphQL
REST
+ = ?
![Page 30: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/30.jpg)
![Page 31: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/31.jpg)
Demo app: Trello REST API to GraphQL
GraphQL
REST
+
![Page 32: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/32.jpg)
Tartiflette main features
● Python 3.6+
● Schema First (SDL)
● Built on AsyncIO
● aiohttp integration
● Embedded GraphiQL development web interface
● Tastes even better than it smells (AKA developer friendly)
![Page 33: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/33.jpg)
Schema Definition Language
![Page 34: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/34.jpg)
1 GraphQL request = x REST requests
These edges will resolve in multiple REST API calls1 GraphQL call = multiple REST calls
![Page 35: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/35.jpg)
'idBoards': ['5d1f33e746ea0a8020560465', '5d1f341e82d5a37d0efb97b1']
1x
2x
resolved edge with full objects
![Page 36: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/36.jpg)
Show me some code: aiohttp app definition
Generic SDL
Resolver functions
![Page 37: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/37.jpg)
Show me some code: GraphQL resolvers
Edge resolver
Root query resolver
![Page 38: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/38.jpg)
#shipit
![Page 39: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/39.jpg)
Dockerfile: multi-stage build
Slim python3.7 run image
Full python3.7 build image
![Page 40: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/40.jpg)
Build + Image tag = git branch + Upload to Gitlab registry
Git branch workflow● development● staging● master + git tag = production
![Page 41: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/41.jpg)
To Kubernetes!
Security
AutomatedLet’s Encrypt SSL
![Page 42: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/42.jpg)
Quick demo
![Page 43: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/43.jpg)
![Page 44: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/44.jpg)
T ke Away
![Page 45: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/45.jpg)
T ke Away
● GraphQL removes friction by normalizing how data is addressed between teams
● Schema Definition Language lets you concentrate on the data, not the code
● Tartiflette is a modern, fast and efficient way of doing Python + GraphQL
● Workflow for environment deployment based on git branches○ TODO: challenge environment multi-tenancy of the cluster later
● Kubernetes secrets + environment variables to store and access secrets○ TODO: generalize vault
● Kubectl is powerful: give that power to developers!○ TODO: allow some abstraction tools when adoption is higher if needed
![Page 46: How we run GraphQL APIs in production on - EuroPython · How we run GraphQL APIs in production on our (own) Kubernetes cluster @ @ultrabug Gentoo Linux developer PSF contributing](https://reader030.fdocuments.net/reader030/viewer/2022041017/5eca7b9603de2c4cd02dc32d/html5/thumbnails/46.jpg)
Thanks!
https://github.com/ultrabug/ep2019
@ultrabug