Spinnaker and SpELEvolving an Expression Language forContinuous Delivery Pipelines

| Tomas Lin @tomaslin

BakeTrigger Deploy SmokeTest

Approval

BakeTrigger Deploy SmokeTest

Approval

Branchname

BakeTrigger Deploy SmokeTest

Approval

Branchname

myapp-feature- Branchname

BakeTrigger Deploy SmokeTest

Approval

run smoke tests?

BakeTrigger Deploy SmokeTest

Approval

run smoke tests?

Spring Expression Language ( SpEL )

“The Spring Expression Language (SpEL for short) is a powerful expression language that supports querying and manipulating an object graph at runtime. The language syntax is similar to Unified EL but offers additional features, most notably method invocation and basic string templating functionality.”

- Spring documentation Chapter 10

Also in:Spring

Thymeleaf

a. Cluster-${ 7+2 } Cluster-9

b. Cluster-${ 7+2 }-${ 9 + 1 }-${ ‘string’ } Cluster-9-10-string

c. ${‘ some stringaroony ’.trim() } some stringaroony

d. ${ deploymentDetails[0] }

e. ${ trigger[‘parameters’][‘ideal temperature’] }

a. ${ trigger.user ?: ‘britney’ }

b. ${ deploymentDetails?.[ region == ‘us-west-1’ ] }

c. ${ #toJson( deploymentDetails ) }

d. ${ new java.text.SimpleDateFormat('mm-dd-yyyy').format(new java.util.Date()) }

YayWe were already heavily invested in Spring Ecosystem - Spring Boot, Spring Batch.

No external library dependency.

Seem pretty robust and powerful.

Looked to have great documentation.

NayCouldn’t find a lot of examples of usage outside of Spring and Thymeleaf.

Could it work outside of managing bean dependencies?

Unsure of how easy it was to extend and customize.

Did it have a heavy dependency on Java?

Spinnaker Pipelines

Open Source successor to Asgard

Multi-region, multi-account, multi-cloud

Used to deploy around 98% of microservices at Netflix.

http://spinnaker.io

Resize, Disable, Enable, Destroy, Deploy, Clone

Shrink, Scale Down, Disable

Wiring up SpEL

Note: Everything we’re talking about is available in Github.

Easiest way to check it out is search for ‘ContextParameterProcessor’ in orca,

https://github.com/spinnaker/orca/search?q=contextparameterprocessor

Trigger( builds a ami )

Smoke Test Merge PREnableServer Group

DeployServer Group

Customizing the parser

task context

Evaluating SpEL

evaluationContext

${ SpEL expression }

parserContext

result

${ job }

faily-godmother

Adding Properties

Execution - the current pipeline ExecutionTrigger - the pipeline trigger

Parameters - pipeline parameterstrigger - pipeline triggerparameters - pipeline parametersscmInfo - git details in triggerdeployedServerGroups : details from last cluster deployed

execution the current pipeline execution

parameters pipeline parameters

trigger pipeline trigger

scmInfo git details

deploymentDetails results of the findImage or Bake stage

deployedServerGroups last created server group

${ deployedServerGroups?.[region==‘us-east-1’][‘serverGroup’] }

myapp-v001

Adding Methods

Execution - the current pipeline ExecutionTrigger - the pipeline trigger

Parameters - pipeline parameterstrigger - pipeline triggerparameters - pipeline parametersscmInfo - git details in triggerdeployedServerGroups : details from last cluster deployed

#alphanumerical trims all non-alphanumerical characters

#fromJson converts JSON to a map

#fromUrl contents from a url

#jsonFromUrl JSON content from an url

#judgment choice from manual judgment stage

Execution - the current pipeline ExecutionTrigger - the pipeline trigger

Parameters - pipeline parameterstrigger - pipeline triggerparameters - pipeline parametersscmInfo - git details in triggerdeployedServerGroups : details from last cluster deployed

#stage shortcut to stage contents by name

#toBoolean conversion to boolean

#toFloat conversion to float

#toInt conversion to int

#toJson conversion from Map to Json

${ #alphanumerical(‘jenny from the block’) }

jennyfromtheblock

Getting data from Jenkins

Trigger

Smoke Test Merge PR

ami=i-xxxyyyyregion=us-west-1

Property Files

${ trigger.properties[‘ami’] }

Trigger

Smoke Test Merge PR

retries=truenumber of retries=2

${ retries }${ #stage(‘Smoke Test’)[‘context’][‘retries’] }

Example: Jenkins parameter passing

${ #alphanumerical( trigger.properties[‘us-west-1’] ).toUpperCase() }

Global SpEL Evaluation

stage

stage

task

task

task

context

definition

stage

stage

task

task

task

context

definitiondefinition

stage

Not everything is a String…

}

Pipeline parameters

Using SpEL to control execution of stages

}

Using SpEL in Spinnaker UI

Common mistakes around Expressions

Culture of Freedom and Responsibility

You build it, you run it, you support it.

#spinnaker slack channel

Office Hours

Anytime walk-in 1-1 help with first responder

${ parameters.ami ?: ${ ‘ami’ + ‘x’ } }

${ parameters.use ami in us-west-1 }

${ parameters[‘use ami in us-west-1’] }

${ #deploymentDetails?.[ region == parameters.region ] }

${ #deploymentDetails?.[ region == #root.parameters.region ] }

${ ‘a’ + ‘b’ } ${ derp } ${ 1 + 1 }

“${ my escaped expression }”.toString()

Simplifying Stage Executions

read from url

#jsonFromUrl( url )

${ #jsonFromUrl( ‘http://archeius/rest/v1/props/myapp’)[‘propertiesList’][0][‘value’]}

${ deploymentDetails[0].jenkins.number != #jsonFromUrl( ‘https://spinnakerapi/applications/orca/clusters/mgmt/orca-main/aws/us-west-2/serverGroups/target/NEWEST?onlyEnabled=true' )['buildInfo']['jenkins']['number'] }

Expressions can be a nice temporary solution

${ trigger.user ?: ‘myteam@netflix.com’ }

What’s next?

Better support for on-the-fly expression testing and construction.

Helper methods such as #pipeline( ‘xxx’ ) and #stageInPipeline( ‘Wait’, ‘xxx’ ).

Better UI support.

Results

Over 1100 pipelines running in production ( of 5100 + ) use at least one SpEL expression.

Thank you.

spinnaker.io jobs.netflix.com

Try Spinnaker Come work with us!