Better value in the NHS - innovate stage, 3.30pm, 2 september 2015
Innovate Better Through Machine data Analytics
-
Upload
hal-rottenberg -
Category
Software
-
view
67 -
download
1
Transcript of Innovate Better Through Machine data Analytics
Copyright © 2015 Splunk Inc.
Innovate Better Through Machine Data AnalyticsHal Rottenberg, Global ITOA PractitionerSplunk
1
Or…
How to Splunk your DevOps
Or…
Hal really didn’t like the first two titles, thought way too long on a third one, then gave up and decided that what really mattered was the next slide anyway
This is what I’m selling
“DEVOPS IS A SUPERSET, NOT A SUBSET”
7
Is this DevOps?
Development Operations
DEVOPS?
8
Is this DevOps?
Development Operations
DEVOPS?NO!
9
This is DevOps!DevOps
Development Operations
10
How Shall We Get There?
Culture
Data
Analysis
Continuous Improvement
Defining DevOps
INTEGRATION
COLLABORATION
COMMUNICATION
BETWEEN DEV AND OPS
METHODS FOR IMPROVING
CAMS – as close to prescriptive as DevOps gets
CultureAutomationMeasurementSharing
BUT WHAT SHOULD
YOU MEASURE?
I’m working super hard!!
That’s my stapler.
Activity?
Yeah, but … … what are you
achieving?
I’m gonna need you to come in Sunday.
Outcomes?
Some DevOps Metrics that Might Matter
Culturee.g.• Retention• Satisfaction• Callouts
Processe.g.• Idea-to-cash• MTTR• Deliver time
Qualitye.g.• Tests passed• Tests failed• Best/worst
Systemse.g.• Throughput• Uptime• Build times
Activitye.g.• Commits• Tests run• Releases
Impacte.g.• Signups• Checkouts• Revenue
From every tool, every process, every component, on-prem or off
Machine Data Is A Critical Source Of DevOps Metrics
Industry Leading Platform for Machine DataAny Machine Data
Online Services Web
Services
ServersSecurity GPS
Location
StorageDesktops
Networks
Packaged Applications
CustomApplicationsMessaging
TelecomsOnline
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
Datacenter
Private Cloud
Public Cloud
Enterprise Scalability
Search and Investigation
Proactive Monitoring
Operational Visibility
Real-time Business Insights
Operational Intelligence
Visibility Across the Ops Environment
APISDKs UI
Server, Storage. N/W
Server Virtualization
Operating Systems
Infrastructure Applications
Mobile Applications Cloud Services
Other ToolsTicketing/Help
Desk
No rigid schemas – add in data from any other source.
Custom Applications API Services
Visibility Across the Dev Lifecycle
APISDKs UI
Other ToolsEscalation/
Collaboration
No rigid schemas – add in data from any other source.
Plan Code Build Test/QA Stage Release Config Monitor
INCREASE VELOCITY
IMPROVE QUALITY
DRIVE IMPACT
Improve the Impact of Application Delivery
What DevOps Data Can You Splunk?
Machine Data for Provisioning and Config
Machine Data from Pre-Prod/Staging
Machine Data From Testing and QA
Machine Data from Release Servers
Machine Data from Infrastructure Systems
Machine Data from Database Servers
Machine Data from Customer-Facing Systems
CI / Build Server
Code Review
Task Tracking
What Data Can You Splunk?
Which code has already been reviewed for this release/sprint? Who has completed the most code reviews? What code has NOT been reviewed?
Who is changing files? What kinds of files are being changed? What branches are most active? What types of activities are occurring for a branch?
Version Control
How many builds completed today/this week/this month? Which check-in kicked off this build? Which tests ran against this failed build?
Which tasks are assigned to which developers? What progress is being made to complete assigned tasks? What tasks remain for this release/sprint?
How Can You Splunk DevOps?
Industry Leading Platform for Machine DataAny Machine Data
Online Services Web
Services
ServersSecurity GPS
Location
StorageDesktops
Networks
Packaged Applications
CustomApplicationsMessaging
TelecomsOnline
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
Datacenter
Private Cloud
Public Cloud
Enterprise Scalability
Search and Investigation
Proactive Monitoring
Operational Visibility
Real-time Business Insights
Operational Intelligence
Splunk Add-On for Jira
Github Modular Input
Puppet Enterprise App for Splunk
Chef App for Splunk
Splunk App for AWS
Splunk Add-on for Google Cloud Platform
Splunk Logging Driver for Docker
curl -k https://<host>:8088/services/collector -H 'Authorization: Splunk <token>' -d '{"event":"Hello Event Collector"}'
Applications IoT Devices
Agentless, direct data onboarding via a standard API
HTTP Event Collector
Scales to Millions of Events/Second
AWS Lambda for HTTP Event Collector
Splunk App for Stream
Enables real-time insights into private,
public and hybrid cloud infrastructures
Delivers rapid deployment, easy
scale out and efficient wire data capture
Capture and analyze critical events not
found in logs or with other collection
methods.
1 2 3
Enhance Operational Intelligence With Wire Data Capture
Splunk MINT for Mobile Data
Deliver Better Performing, More
Reliable Apps
Deliver Real-Time Analytics
Achieve End-to-End Visibility
PagerDuty App for Splunk
Copyright © 2015 Splunk Inc.
Why Use Splunk for DevOps?
Machine Data To Enable Continuous Improvement
Defect Information
CapacityPlanning
Quality Standards
Enhancement Requests
Integration Requirements
Acceptance Metrics
Service Levels and KPIs
Application Development Test and Acceptance Production
BuildCodePlan Test/QA Stage Release Config Monitor
InfrastructureDependencies
Increase Delivery Velocity
DevOps Teams Iterate with Continuous Insights
Product Managers
identify new opportunities
Code Continuously delivered to market
Auditorshave visibility
Customersare happy
Improve Code Quality
Code quality scans Static security scans
White BoxDevelopers check in code
Automated Acceptance Tests
Dynamic Security Scans
Black Box
“Chaos Monkey” tests
Test Fail: Return
Test Fail: Return
X
X
Production
QA Prod Pattern
QA Pattern Library
Test Pass: Promote
Test Pass: Promote to Production
Pattern library used for test and
QA
Align With Business Impact
In Closing…
INCREASE VELOCITY
IMPROVE QUALITY
DRIVE IMPACT
Improve the Impact of Application Delivery
A Reminder
“DEVOPS IS A SUPERSET, NOT A SUBSET”- HAL ROTTENBERG, MAY 2016
Where to go for more Info● DevOps Videos, Customer Stories, Whitepapers
‣ http://splunk.com/DevOps– Developer Tutorials, Code Samples, Downloads– http://dev.splunk.com
● Splunk Apps and Plugins– https://splunkbase.splunk.com
● Blogs for Dev, Ops, and DevOps– http://blogs.splunk.com
53
54
Bonus Content
55
Splunk for Developers
Real-time dashboards show error rate in production and impact of pushing
new builds
Developers can search and visualize web logs, Java logs, eventlogs etc;
trace tx without complex instrumentation
Alerts notify developers as soon as a problem arises
57
Find and Fix Issues Faster
Gain end-to-end visibility to make informed decisions
Analytics insights without the need for additional analytics tools
Ask questions while exploring and collecting data
58
Push Better Code Using Analytics
Powerful Platform for Enterprise Developers
59
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript/CSS Extensions C#JavaScriptPython
RubyJavaPHP
Data Models
Search Extensibility
Modular Inputs
SDKs
KV Store
Splunk Developer Guidance
Splunk Reference AppsComplete, working real-world Splunk solutions built together with partners (Conducive; Auth0)̶U 2 (pseudo-) production releases̶U entire code & test repos on GitHub̶U under Apache 2.0
Associated GuidanceI. Start-to-Finish Journey Documentary II. Essentials
dev.splunk.com/goto/devguide
Splunk in Real World DevOps Use Cases
Successful Businesses Use Splunk for DevOps
63
Improved DevOps Agility
63
Key Customer Benefits
• Increased success rate of deployments• Detect issues before they affect broad production• Monitoring deployment process several times per day
-Robert Gonsalves,Web Operations
“It’s like we were working without peripheral vision before and now we have it.”
64
Deliver Better Code Quality
Key Customer Benefits
-Principal Engineer,Apollo Group
“Developers are now able to look for errors and troubleshoot issues five to ten times faster by having all their event data centralized in Splunk.”
• Provide full visibility into QA sanity and load testing before production
• Exceed SLA thresholds with full visibility and benchmark key infrastructure metrics and errors
• Easier troubleshooting if tests do not contain the expected results
65
Enable Data-driven Continuous Delivery
-Alison Perkins, Senior Systems Engineer
“ Dump all the logs into Splunk, and it starts looking like one big system, instead of a bazillion teeny ones that hate each other.”
Key Customer Benefits • Quickly validate and troubleshoot code pushes to
production• Ensure that new code does not negatively impact
performance or user experience • Reduced one application’s error rate by 2 orders of
magnitude in a matter of weeks