Senlin deep dive 2016
-
Upload
qiming-teng -
Category
Software
-
view
151 -
download
1
Transcript of Senlin deep dive 2016
Senlin Clustering Service Deep DiveEthan Lynn, IBM, IRC: elynn
Qiming Teng, IBM, IRC: Qiming
Yanyan Hu, IBM, IRC: yanyanhu
Agenda
• Why did we start senlin project
• Senlin overview
• Senlin status and roadmap
Why did we start senlin
Background
• Why Senlin?• CNCP (IBM research project)
• Heat AutoScaling Evaluation
• Heat AutoScaling Rework
• Customer Engagement
Gap Analysis
Heat AutoScaling ReworkTalk on Atlanta Summit
Evaluation Report
Background
• Why Not Heat?• Heat AutoScaling today is based on
AWS design, with gaps
• Heat's mission• Defined in project governance
• It's NOT about autoscaling, NOT about HA, ...
• It's NOT Heat's job to provide new service(s)
To orchestrate composite cloud applications
using a declarative template format through an
OpenStack-native REST API.
Heat Mission Statement
https://github.com/openstack/governance/blob/
master/reference/projects.yaml
• Proposal:
https://wiki.openstack.org/wiki/Heat/AutoScaling
• API Design: http://docs.heatautoscale.apiary.io/
• Targeting at many use cases not covered by Heat
today
AutoScaling: a new design
Heat as a Compiler
YAML
Template
Heat
Nova
Cinder
Neutron
Keystone
• CREATE
• GET
• UPDATE
• DELETE
Blueprints on Reworking Heat AutoScaling
BP Priority Description
autoscaling-
api-resources
high Heat resources invoking AS APIs
as-api-group-
resource
high ScalingGroup resource wrapping AS API's
group functionality
as-api-policy-
resource
high ScalingPolicy resource wrapping AS API's
policy functionality
as-api-
webhook-res
high Webhook resource wrapping AS API's
execution of webhooks
autoscaling-
api-client
high A python client for Heat to interact with AS
API
autoscaling-
api
- A separate service for the implementation
of autoscaling w/ Heat
as-engine - A separate engine/service for autoscaling
support AS API
as-engine-db - A DB dedicated to autoscaling, using
schema created in as-lib-db
as-lib - A separate module to be used by the AS
service
as-lib-db - A DB for autoscaling bookkeeping
What Do We Really Need?
Scalable
Load-Balanced
Highly-Available
Manageable
......
of any (OpenStack) objects
- What is missing from
OpenStack?
A Clustering Service
- Auto-scaling?
Just one of the usage
scenario of a cluster.
- Auto-Healing (HA)?
Just another usage
scenario.
- We can address the
concerns by making policies
orthogonal
Senlin Overview
Senlin Architecture
REST RPC
Profiles
Policies
Senlin Features
• Profiles: A specification for the objects to be managed
• Policies: Rules to be checked/enforced before/after actions are performed
10
(others)
Senlin
Nova
Docker
Heat
Ironic BareMetal
VMs
Stacks
Containersplacement
deletion
scaling
health
load-balance
affinity
Policies as Plugins Profiles as Plugins Cluster/Nodes Managed
Senlin Server Architecture
openstacksdk
identity
compute
orchestration
network
...
engineengine lock
scheduler
actions
nodecluster
service
registry
receiverparser
drivers
openstack
dummy
(others)
dbapi
rpc client
policies
placement
deletion
scaling
health
load-balance
affinity
receiver
webhoook
MsgQueue
extension points
for external
monitoring
services
extension points
facilitating a
smarter cluster
management
extension points to talk to different
endpoints for object CRUD operations
extension points for interfacing
with different services or clouds
profiles
os.heat.stack
(others)
os.nova.server
senlin-api
WSGI
middleware
apiv1
Senlin Operations (Actions)
• Cluster• CREATE• DELETE• UPDATE• LIST• SHOW• ADD_NODES• DEL_NODES• SCALE_OUT• SCALE_IN• RESIZE• POLICY_ATTACH• POLICY_DETACH• POLICY_UPDATE
12
Node• CREATE• DELETE• UPDATE• LIST• SHOW
Policy• CREATE• UPDATE• DELETE• LIST• SHOW
Profile• CREATE• UPDATE• DELETE• LIST• SHOW
Action• LIST• SHOW
Event• LIST• SHOW
Receiver• CREATE• DELETE• LIST• SHOW
profile/cluster/node operations
13
$ senlin profile-create -s <spec> <name>$ senlin profile-list$ senlin profile-show <profile>$ senlin profile-delete <profile>$ senlin profile-update -n <name> <profile>
$ senlin cluster-create -p <profile> -n <size> <name>$ senlin cluster-delete <cluster>$ senlin cluster-list$ senlin cluster-show <cluster>
$ senlin node-create -p <profile> -c <cluster> <name>$ senlin node-delete <node>$ senlin node-show <node>$ senlin node-join -c <cluster> <node>$ senlin node-leave <node>
$ senlin cluster-node-add -n <nodes> <cluster>$ senlin cluster-node-del -n <nodes> <cluster>$ senlin cluster-node-list <cluster>
Profile CRUD
Cluster
CRUD
Node CRUD
Cluster
Membership
policies operations
14
$ senlin policy-create -s <spec> <name>$ senlin policy-list$ senlin policy-show <policy>$ senlin policy-delete <policy>$ senlin policy-update -n <name> ... <policy>
$ senlin cluster-policy-attach <cluster> <policy>$ senlin cluster-policy-detach <cluster> <policy>$ senlin cluster-policy-list <cluster>$ senlin cluster-policy-show <cluster> <policy>$ senlin cluster-policy-enable <cluster> <policy>$ senlin cluster-policy-disable <cluster> <policy>$ senlin cluster-policy-update <cluster> <policy>
Policy
Management
Cluster-Policy
Bindings
Other Useful Operations
15
$ senlin cluster-scale-in [-c <count>] <cluster>$ senlin cluster-scale-out [-c <count>] <cluster>$ senlin cluster-resize <options> <cluster>
$ senlin profile-type-list$ senlin profile-type-show <type>$ senlin policy-type-list$ senlin policy-type-show <type>
$ senlin action-list$ senlin action-show <action-id>
$ senlin event-list$ senlin event-show <event-id>
Cluster
Resize
Profile Type
Policy Type
Action
Event
senlin cluster-resize
16
usage: senlin cluster-resize [-c <CAPACITY>] [-a <ADJUSTMENT>] [-p <PERCENTAGE>][-t <MIN_STEP>] [-s] [-n MIN] [-m MAX] <CLUSTER>
Positional arguments:<CLUSTER> Name or ID of cluster to operate on.
Optional arguments:-c <CAPACITY>, --capacity <CAPACITY> The desired capacity the cluster.-a <ADJUSTMENT>, --adjustment <ADJUSTMENT>
A positive integer meaning the number of nodes to add,or a negative integer indicating the number of nodesto remove.
-p <PERCENTAGE>, --percentage <PERCENTAGE>A value that is interpreted as the percentage of sizeadjustment. This value can be positive or negative.
-t <MIN_STEP>, --min-step <MIN_STEP>An integer specifying the number of nodes foradjustment when <PERCENTAGE> is specified.
-s, --strict A boolean specifying whether the resize should beperformed on a best-effort basis when the new capacitymay go beyond size constraints.
-n MIN, --min-size MIN New lower bound of cluster size.-m MAX, --max-size MAX New upper bound of cluster size. A value of -1
indicates no upper limit on cluster size.
Adjustment Types
• EXACT CAPACITY
• CAPACITY CHANGE
• CAPACITY PERCENTAGE
Optional Modifiers
• Strict vs. Best-Effort
• Minimum Step
Dealing with Constraints
• MIN_SIZE
• MAX_SIZE
Senlin Roadmap
Senlin Journey
Talk on China
OpenStack/Docker
community joint
meetup
2015-04-25
• IBM internal
announce
• Email to OpenStack
Heat Core team
2015-02-10
Project
accepted to
StackForge
2015-03-16
Community
Announcem
ent
2015-03-26
Git repo
migrate to
github.com
2014-12-25
Initial Git
Repository
inside
CRL
2014-12-10
First weekly
IRC
meeting
2015-06-10
Talk on
OPNFV
meeting
2015-07-08
Start work
with TOSCA
std team
2015-07-09
Talk on 2nd China
OpenStack
Hackathon
2015-08-20
Senlin
dashboard
project
initiated
2015-08-24
Talk on Tokyo
Summit -- Scaling
for Containers
2015-10-29
Senlin
docs
went
online
2015-12-03
Project
accepted
into Big
Tent
2015-11-11
Mitaka
Mid-Cycle
Meetup
2016-01-11
Vancouver
Heat design
summit
2015-05-20
2015
Jan2015
Jul2016
Jan
2015
Apr
2015
Feb
2015
Mar
2015
May
2015
Jun
2015
Oct
2015
Aug
2015
Sep
2015
Nov
2015
Dec
2014
Dec2016
Feb
2016
Mar
2016
Apr
Version
1.0
released
2016-04-07
Senlinclient
0.4.0
released
2016-03-19
Mitaka Features
• API Changes and API Reference
• OpenStackClient Support
• Heat Resources
• Health Management primitive
• User/Developer Documentation
• Senlin Dashboard
Newton Outlook
API micro-version
Health management
Container Cluster
Testing:
API/Scenario/Performance...
Additional Clustering Operations
Queue (e.g. Zaqar) type Receiver
Notifications to other services
Batched Operations
User Defined Actions
Access Control
More communications, more collaborations
Relation To Other Projects
21
Senlin
Ceilometer
Heat
Nova Cinder Neutron Swift Keystone
Horizon
Primitive Data Types
Complex Data Types
struct person {int age;char name[0];
}
person team[10]; // Senlin cluster of Heat stacks
// Senlin cluster of nova servers
// Heat stack containing senlin clusters
Mistral MagnumSaharaMonasca Tacker Zaqar
Senlin provides the array data type for cloud programming
We Are Hiring,
Standard
OpenStack
Customer
IBMTacker
Contributors
Zaqar
Questions?
Thank You!