Operator Framework

Rob SzumskiProduct Manager, OpenShift

1

2

A broad ecosystem of workloadsOperator-backed services allow for a

SaaS experience on your own infrastructure

Relational DBs

NoSQL DBs

Storage

Messaging

Security

Monitoring

AL/ML

Big Data

DevOps

3

Embed ops knowledge from the

experts

Operator v1.1.2 DeploymentsStatefulSetsAutoscalers

SecretsConfig maps

OPERATOR BASICS

4

● Operator SDK - Allows developers to build, package and test an Operator based on your expertise without requiring all the knowledge of Kubernetes API complexities

● Operator Lifecycle Manager - Helps you to deploy, and update, and generally manage the lifecycle of all of the Operators (and their associated services) running across your clusters

● OperatorHub.io - Publishing platform for Kubernetes Operators, allows for easy discovery and install of Operators using a graphical user interface

Build Operators for your appsBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

Ansible SDKHelm SDK Go SDK

Helm Chart Ansible Playbooks,Roles & APBs

Build operators from Helm chart, without any

coding

Build operators from Ansible playbooks and

APBs

Build advanced operators for full lifecycle

management

OPERATORSDK

6

Operator Capability Model

Red Hat Certified OperatorsBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

STORAGE

SECURITY

DATABASE

DATA SERVICES

APM

DEVOPS

OperatorHub data sourcesBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

Operator Metadata from quay.io● Backend for all default sources, cluster needs to be online● Supplies Red Hat Operators, ISV Operators and Community

Operator● Custom sources supported in customer-owned quay.io

namespaces

Operator Metadata in container images● Already used internally used by OLM● Operator package data is served from a SQlite database,

bundled up in a container image● Custom sources supported in customer-owned image registries● Cluster can be disconnected / air-gapped

Package namee.g. prometheus

Channel namee.g. stable

Operator Bundle 1

Operator Bundle 2

Channel namee.g. tech-preview

Operator Bundle 3

...

Operator Package Metadata

Operator SDKBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

Helm-based Operator● Support for Helm 2.14

○ Helm 3 Support under investigation● SDK automatically generates RBAC for your chart

Ansible-based Operator● Support for Prometheus Metrics● Uses UBI base-image● Molecule-based e2e testing

Golang-based Operator● Supporting Kubernetes 1.14● Remove $GOPATH dependency● Go module support● Support for Prometheus Metrics● Generate OpenAPI spec

Framework Integration● Single command to install / uninstall OLM:

operatork-sdk alpha olm [install|status|uninstall]

10

Next 3 months More than 9 monthsNext 3-9 months

OPERATOR SDK ROADMAP

Upstream compatibilityObjectives: - a single upstream effort for Golang Operators

Features: - Kubebuilder CLI support - Kubebuilder project layout compatibility

Stage: Development

Framework integrationObjectives: - OLM / SDK integrationFeatures: - install OLM from SDK CLI - run Operator with OLM from SDK CLIStage: Prototype

Operator SDK 1.0Objectives: - Stable interfaces and high project maturity

Features: - Helm v3 Operator / Ansible 1.0 Operator - Kubebuilder Integration - scorecard v2 / custom functional tests - stable OLM integration

Stage: Planning

Increase Language supportObjectives: - allow more people to write Operators

Features: - Java SDK - Python SDK

Stage: Discovery

SDK Developer ExperienceObjectives: - incrementally mature Operators - quick development feedback loop

Features: - modular Operator (mix helm/ansible/go) - skaffold integration

Stage: Discovery

11

● Official online catalog on catalog.redhat.com● Community catalog on operatorhub.io● OperatorHub on cluster accessible to admins ● Discovery/install of all optional components

and apps● Upstream and downstream content● ISV partners will support their Operators

Red Hat ProductsISV PartnersCommunity

TYPES OF OPERATORS

OPERATOR HUB ROADMAP

12

OPERATOR HUB ROADMAP

CSV Bundle Editor for Operator developers

4.1 Static Dependency ResolutionBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

YourOperator v1.1.2

requires

requires

Jaeger Operatorjaeger.jaegertracing.io/v1

CockroachDB Operatorcockroachdb.charts.helm.k8s.io/v1alpha1

resolves to

resolves to

Operator Framework Dependency Graphs

4.2 Automated Dependency ResolutionBROAD ECOSYSTEM OF WORKLOADS

Generally AvailableProduct Manager: Daniel Messer

YourOperator v1.1.2

requires

requires

Jaeger Operatorjaeger.jaegertracing.io/v1

CockroachDB Operatorcockroachdb.charts.helm.k8s.io/v1alpha1

resolves to

resolves to

Operator Framework Dependency Graphs

OPERATOR

LIFECYCLE MANAGER

installed by

installed by

15

BROAD ECOSYSTEM OF WORKLOADS

Product Manager: Daniel Messer Generally Available

Allow regular users to install Operators

● In 4.1: only users carrying cluster-admin roles are allowed to install Operators

● In 4.2: administrators can delegate install to users

○ cluster-admin select namespaces in which namespace admins can install operators self-sufficiently

○ cluster-admin defines ServiceAccount in this namespace

○ all installed Operators in this namespace get equal or lower permissions of this ServiceAccount

■ RBAC is typically limited to this namespace

OperatorGroup

ServiceAccount

SA Role

OperatorRole

cannot be greater thanNamespace /

Project

16

BROAD ECOSYSTEM OF WORKLOADS

Product Manager: Daniel Messer

Singleton CRs & Auto-create CRs from single click

Useful for: Serverless, Metering, Service Mesh, Pipelines, Logging, Container Storage & more

1. Install into a specific namespace from CSV

2. Automatically create an Operand instance

3. Hooks into OpenShift Console are installed/configured

a. If RH product, navigation shows up

b. Configure custom dashboards

c. Configure external links and banners

d. Register new CLIs in the downloads area

*Single click*

17

BROAD ECOSYSTEM OF WORKLOADS

Product Manager: Daniel Messer

Simplified Object Model

CSV + Subscription + InstallPlanapiVersion: operatorframework.io/v1alpha1kind: Operatormetadata: ...

1. Unlocks ability to install specific version (not latest)2. Directly install Operator outside of OperatorHub

a. bypass catalogs, OperatorGroups, etc3. Easier onboarding and building of Operator releases

single Operator object

Split CSV into new bundle format

Kubernetes objects:Deployment/StatefulSet, Roles, RoleBindings, custom SCCs

Metadata:icon, channels, related images,CR examples,

18

BROAD ECOSYSTEM OF WORKLOADS

Product Manager: Daniel Messer

New Operator Bundle FormatStreamlined developer UX for getting an Operator running without hassle of a central catalog

$ operator-sdk bundle init --type=registry --bundle-folder=0.1.0

$ tree testtest├── 0.1.0│ ├── testbackup.crd.yaml│ ├── testcluster.crd.yaml│ ├── testoperator.v0.1.0.csv.yaml│ └── testrestore.crd.yaml

$ podman build .$ podman push quay.io/test/test-operator:v0.1.0

$ kubectl apply -f -apiVersion: operators.operatorframework.io/v2alpha1kind: Operatormetadata: name: test-operatorspec: bundle:

image:

quay.io/test/test-operator:v0.1.0

1. Build with CLI 2. Push to Registry 3. Pull & start Operator

Working with kubebuilder & others upstream to standardize this format.

Certified/Community catalogs will also use this format.

19

Next 3 months More than 9 months

SimplificationObjectives: - simplify API surface - simplify shipping Operators and catalogs

Features: - new top-level Operator object - singleton CR / auto-create CRs - observability of Operators - Operator bundles as container images - Operator catalogs as container images

Stage: Development

Next 3-9 months

Leverage upstream technologiesObjectives: - get more upstream acceptance - simplify shipping Operator update

Features: - support Helm charts as packaging format for Operators - Operators can ship/customize any k8s object - implicit updates path through semver - allow to depend on Operator by version

Stage: Discovery

OPERATOR LIFECYCLE MANAGER ROADMAP

Platform integrationFeatures: - Platform coordinates with OLM in order to anticipate breaking updates - First-class support for Webhooks

Stage: Design

New OLM interaction modelObjectives: - simplify using Operators - OLM can manage applications

Features: - Operators scale to zero - Fine grained Operator discoverability - OLM participates in application management - kubectl plugin

Stage: Discovery

Console Customization for Clusters

Generally Available

Console Customization built with CRDsCluster admins will be given the ability to customize certain aspects of the web console for all cluster users.

Customizations include:

● Links - ConsoleLinks CRD○ Help Menu, User Menu, Application Menu

● Notifications - ConsoleNotifications CRD○ Top, Bottom, Top and Bottom

● Branding - Console-Config ConfigMap○ Logo, About

● Command Lines - ConsoleCLIDownload CRD○ Add your own Command lines

● External Log Links - ConsoleExternalLogLinks CRD

Product Manager: Ali Mobrem

EXTENDING THE CONSOLE

EXTENDING THE CONSOLE

Expose Third Party App Console for Operator-backed Services

Expose Operator-backed serviceConsole through console CRD

Easily integrate/onboard third-party user interfaces in order to develop, administer, and configure Operator-backed services.

Product Manager: Ali Mobrem

Enhanced Declarative UI for Operator-backed Services

Declarative dynamic UI for creating/mutating the CustomResources managed by the Operators

● Generic creation form is generated/validated based on OpenAPIV3Schema

● Operator Lifecycle Manager (OLM) descriptors override the generic form generated by the OpenAPIV3Schema with advanced custom widgets

● A set of new widgets associated with the specDescriptors:- podAntiAffinity- radioButton- advanced- fieldGroup- arrayFieldGroup

- text- number- password- fieldGroup - checkbox

- k8sResourcePrefix- updateStrategy- imagePullPolicy- nodeAffinity- podAffinity

Product Manager: Tony Wu

EXTENDING THE CONSOLE

Reduce friction for deploying operator backed applications

● Rich UI for creating and mutating the app instances

Reusable UI widgets

● New Operators get UIs without touching openshift/console codebase at all

Extending the Console

● ISVs can built their own UI widgets and contribute back

Create/Edit View for Operator-backed Services

Product Manager: Tony Wu

EXTENDING THE CONSOLE

24

BROAD ECOSYSTEM OF WORKLOADS

Product Manager: Daniel Messer Generally Available

Proxy Support

OpenShift 4.2 Cluster

OPERATORLIFECYCLE MANAGER

Pod

spec: containers: - name: my-container image: ... env: - name: HTTP_PROXY value: "..." - name: HTTPS_PROXY value: "..."

Cluster Proxy Config

Operator

App Binding with Operator-backed services

25

● Inject (bind) credentials and configs into applications.

● Manages the binding between a PodSpec-compliant workload and an operator-backed service.

● Via label selectors or resource references and the ServiceBindingRequest

● Supported by Topology View in Dev Console.

● Available as optional install from OperatorHub

apiVersion: apps.openshift.io/v1alpha1kind: ServiceBindingRequestmetadata: name: binding-request namespace: service-binding-demospec: applicationSelector: matchLabels: connects-to: postgres environment: demo group: apps.openshift.io version: v1 resource: deploymentconfigs backingServiceSelector: group: postgresql.baiju.dev version: v1alpha1 kind: Database

resourceRef: db-demo

26

Operators in Action PanelBuilders, Users and Maintainers

Piyush Nimbalkar (Portworx)Evan Pease (Couchbase)

Simon Croome (StorageOS)Peter Hack (Dynatrace)

Jason Mimick (MongoDB)