Competing with Software: It Takes a Platform -- Devops @ EMC World
DevOps and Continuous Testing€¦ · · 2018-03-21time it takes to release new features or...
Transcript of DevOps and Continuous Testing€¦ · · 2018-03-21time it takes to release new features or...
DevOps andContinuous Testing
Mark Kachik, Micro Focus Lead Solution Consultant
Shannon Lake, B12, VP, Solutions and Strategy
Robert Linton, B12, VP, Applications Delivery
March 8th, 2018
Modern Software Lifecycle Disruptions
Overview End-to-End DevOps
Approaches to DevOps
Tools Discussion
Example Customer User Cases
Questions
2
Agenda
Modern software lifecycle disruptions
DevOps AdoptionChanges everything
Software ComplexityGrowing to the point of chaos
Cloud TransformationShift to cloud / utility models
Waterfall Iterative Agile DevOps
+
Client-ServerMonolithic N-Tier Microservices + APIs
Open SystemsMainframe Virtualization/Cloud Cloud Native
MET
HO
DO
LOG
YA
PP
LIC
ATI
ON
SIN
FRA
STR
UC
TUR
E
ORGANIZATIONShift from IT centric to LOB projects with self-service
REQUIREMENTSShift from elicitation and sign-off to dynamic backlog
QUALITYShift from quality COEs to continuous quality in every part of the lifecycle
DEPLOYMENTShift from release trains to DevOps deployment automation
TECHNOLOGYShift from enterprise architecture to developer dominance
CONTROL POINT SHIFTS
Trends do not guarantee business outcomes
0%
29%
52%
5%
12%
4%
29%
40%
15% 14%
0%
10%
20%
30%
40%
50%
60%
Very satisfied Somewhatsatisfied
Somewhatdissatisfied
Verydissatisfied
Don't know /no
measurement
How satisfied do you think your business is with the time it takes to release new features or changes to customer-facing business services or applications?
2014 2016
https://www.forrester.com/report/Six+Trends+That+Will+Shape+DevOps+Adoption+In+2016+And+Beyond/-/E-RES120481
87% are moving toward a DevOps process
Source: Dimensional Research – It Is Flying Blind In Hybrid Clouds – April 2016
86% are considering containers
85% will have a continuous delivery process
Customer
value?
Business
demandsPlan Develop Test Release
Is your entire value chain agile?
Deploy Operate
Manual testing & # number of defects
Rapidly increasing WIP
Manual build and integration processes
Insufficient test data for decision
making
Outages from undetected defects
Outages due to change related
incidents
Lack of timely customer insight
7 days 2 days 15 days 10 days1 day
Cycle Time = 70 daysLong wait time for build/test environments Manual and error prone app deploymentsIT and business release cycles misaligned
7 days12 days
Overview End-to-End Enterprise DevOps Solution
Enterprise DevOps
7
Continuousplanning
Plan
Define
Continuousintegrationand testing
Dev & Build
Test
Continuousdeployment and
release
Release
Deploy
Continuousoperations
Operate
Monitor
Continuous Security
Continuous assessment
DemandCustomer Experienc
e
Continuous Collaboration
Micro Focus Enterprise DevOps Operating Model: your transformation blueprint
Leadership
• Lean-Agile leaders as a baseline• Lead the change• Enable collaboration & motivate• Decentralize decision making
Strategy & Goals
• Implement strategy aligned to customer centric value delivery in tune with business strategy
• Align vision, roadmap, portfolio, budgets & metricsto value streams
Culture
•
• Accepting DevOps into culture & mindset• Management of organizational change for
individuals, teams and the organization
Learning & Information
• Enable for agile, lean & DevOps thinking
• Manage & coach adoption • Foster knowledge with communities
ofpractice, pairing roles & piloting
Processes
• Align to support agile, waterfall & outsourcing
•
• Standardize using best practices and align
to IT4IT value streams
Organization & Structure
• Focused on delivering value• De-silo: Foster collaboration &
strong relationships• Optimize with Scaled Agile
Framework
Mission & Vision
LeadershipLearning & Information
Strategy & Goals
PerformanceMeasures &
Rewards
Process
Systems
OrganizationalStructure
CultureSystems
•
industry standard software and open source• Automate & integrate process & people
bottlenecks• Span across the DevOps pipeline
Performance, Measures & Rewards
• KPIs & measures aligned to customer value• Measure DevOps processes across quality,
velocity, productivity & security• Reward teams and the organization
The art of the possibleEnabling transformation through end-state visualisation
Fully functional, reference implementation environment, with pre-built integrations across
Micro Focus portfolio and open source tools, showcasing key, common, real-life use cases
based on IT4IT™ and SAFe® industry standardsMicro FocusModel Office
Micro Focus DevOps Model Office Blueprint
Continuous Operations
Scale up/down Auto Failover Service Mgmt App Perf.
Monitoring Detect2Correct Secure Access Secure Data Secure App
Software Tools: Docker & K8S UCMDB Service Manager SiteScope AppPulse BSM Voltage & SDM NetIQ Fortify
AppDefender
Continuous Security
Security Requirement/Design ALM Octane & ALM.Net
Security Static Code Analysis Fortify SCA/SSC & Jenkins & IDE & Octane
Security Dynamic Testing Fortify WebInspect/SSC & ALM & Octane & SDM
Security Monitoring/Protection Fortify AppDefender, ArcSight, Voltage
Continuous Testing
Shift-Left LeanFT Functional Test Performance Test Security Test Service Virtualization Network Virtualization Mobile Testing Test Data Management Business Process Test
Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center SDM & Secure Data Fortify SCA &WebInspect Fortify SSC & ALM & OctaneTesters
Continuous Planning
Demand Management Proposal Management Portfolio Management Program Management Time Management Resource Management Cost Management Req/Defect Management Agile Release Management Agile Time Management Architecture Design Security Design
Project Managers
Business Analysts
Architects
ProgramManagers
Software Tools: PPM ALM Octane ALM.Net 3rd party Agile Tools
Continuous Integration
Developers
Dev Testers
System Team
Web/Mobile/Backend Dev Req/Code Linkage Defect/Code Linkage Code Coverage Shift-Left Testing Container Image Build Build & Packaging Security Static Code Scan Gated Check-in Version Control on Src Version Control on Exec. Status Synchronization
Software Tools: IntelliJ, Eclipse, VS, Borland ALM Octane & Plugins LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry
SIT QAS STG PRDTIS
Continuous Deployment & Release
Application Release Automation Hybrid Cloud based Deployment Docker-based Deployment Mobile App Deployment Configuration Integration IT Service Management
Software Tools: Jenkins & Serena & Chef/OO HCM – Premium K8S / Docker EE / Mesos Mobile Center UCMDB & Service Manager
Continuous Collaboration
PPMBot
ALM OctaneBot
FortifyBot
JenkinsBot
GITLabBot
SitescopeBot
OOBot
SMBot
Software Tools MatterMost, Microsoft Teams, Slack
Continuous Assessment
Software Tools: Vertica, postgresql, IDOL, ETL, PowerBI
Velocity-focusedKPIs
Productivity-focusedKPIs
Security-focusedKPIs
Quality-focusedKPIs
Micro Focus DevOps Model Office Blueprint
Continuous Operations
Scale up/down Auto Failover Service Mgmt App Perf.
Monitoring Detect2Correct Secure Access Secure Data Secure App
Software Tools: Docker & K8S UCMDB Service Manager SiteScope AppPulse BSM Voltage & SDM NetIQ Fortify
AppDefender
Continuous Security
Security Requirement/Design ALM Octane & ALM.Net
Security Static Code Analysis Fortify SCA/SSC & Jenkins & IDE & Octane
Security Dynamic Testing Fortify WebInspect/SSC & ALM & Octane & SDM
Security Monitoring/Protection Fortify AppDefender, ArcSight, Voltage
Continuous Testing
Shift-Left LeanFT Functional Test Performance Test Security Test Service Virtualization Network Virtualization Mobile Testing Test Data Management Business Process Test
Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center SDM & Secure Data Fortify SCA &WebInspect Fortify SSC & ALM & OctaneTesters
Continuous Planning
Demand Management Proposal Management Portfolio Management Program Management Time Management Resource Management Cost Management Req/Defect Management Agile Release Management Agile Time Management Architecture Design Security Design
Project Managers
Business Analysts
Architects
ProgramManagers
Software Tools: PPM ALM Octane ALM.Net 3rd party Agile Tools
Continuous Integration
Developers
Dev Testers
System Team
Web/Mobile/Backend Dev Req/Code Linkage Defect/Code Linkage Code Coverage Shift-Left Testing Container Image Build Build & Packaging Security Static Code Scan Gated Check-in Version Control on Src Version Control on Exec. Status Synchronization
Software Tools: IntelliJ, Eclipse, VS, Borland ALM Octane & Plugins LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry
SIT QA STG ProdTIS
Continuous Deployment & Release
Application Release Automation Hybrid Cloud based Deployment Docker-based Deployment Mobile App Deployment Configuration Integration IT Service Management
Software Tools: Jenkins & Serena & Chef/OO HCM – Premium K8S / Docker EE / Mesos Mobile Center UCMDB & Service Manager
Continuous Collaboration
PPMBot
ALM OctaneBot
FortifyBot
JenkinsBot
GITLabBot
SitescopeBot
OOBot
SMBot
Software Tools MatterMost, Microsoft Teams, Slack
Continuous Assessment
Software Tools: Vertica, postgresql, IDOL, ETL, PowerBI
Velocity-focusedKPIs
Productivity-focusedKPIs
Security-focusedKPIs
Quality-focusedKPIs
Continuous Integration and Testing
Continuous Integration
Web/Mobile/Backend Dev Req/Code Linkage Defect/Code Linkage Code Coverage Shift-Left Testing Container Image Build Build & Packaging Security Static Code Scan Gated Check-in Version Control on Src Version Control on Exec. Status Synchronization
Software Tools: IntelliJ, Eclipse, VS ALM Octane LeanFT, SV, Sprinter Fortify & SSC Maven, Gradle, GITLab Nexus, Jenkins, verigreen Docker, Docker Registry
Continuous Testing
Shift-Left LeanFT Functional Test Performance Test Security Test Service Virtualization Network Virtualization Mobile Testing Test Data Management Business Process Test
Software Tools: UFT/LeanFT Performance Center Service Virtualization Network Virtualization Mobile Center IM&G SDM Fortify SCA &WebInspect Fortify SSC & ALM
GatherResults &Feedback
Code
StaticAnalysisUnit Test
PrivateBuild
Check-In
IntegratedBuild
RunIntegrated
Test
Deployto Test
AutomatedTest
Continuousintegration
Continuoustesting
VirtualizeServices
VirtualizeData
Run System &End to End Tests
RunNon-Functional
Tests
ManageDefects
DefineTests
ContinuousPlanning &
DesignDefineStories
ElaborateStories
DefineFeatures
ContinuousDeployment
& Release
Source CodeRepository Build & deploy
code
Run smoketests
12
Features of continuous integration & testingContinuous integration
Use a single integratedcode repository
Automated, fast, self-testingbuild & deploy
Static analysis of codeincluding security scan
1st
Define tests first beforecode is written
Stories, reqs, code, tests, builds & defects are all linked
Developers pair up withother developers or testers
Automated tests occur inproduction-like environments
Planning, design, coding, testing& deployment are integrated
Continuous testing
Shift all types of testing left
Manage all test scriptin a common repository
Automate testing early,as much as possible
Virtualize service & networks
User targeted change impactregression testing
Automate, simulate &create test data
Apply dynamic app security &performance testing early
Implement mobile & IoTtest automation
Continuous testing flow
13
Dev UnitTest
Deploy toTIS
Team Integration Test
Continuous Testing in Continuous Delivery Flow
Issue/DefectManagement
Deploy toSIT
System Integration Test
Deploy toQA
FunctionalTest
Deploy toSTG
PerformanceTest
Deploy toPRD
Junit/LeanFTTest
HCM deploy K8S Deploy Mesos deploy Mobile Deploy
2
Junit/LeanFT/SCATest
Run Unit Test LeanFT tests SCA Code Scan LeanFT test
cases in GITLab
3
SSC/ALM/OctaneIssues/defects
Defects & test results
HCM deploy K8S Deploy Mesos deploy Mobile Deploy
4
Functional test using UFT
Optionally Security Dynamic Testing
UFT test cases in GITLab
5
UFT/WebInspectTest
Defects & test results
HCM deploy K8S Deploy Mesos deploy Mobile Deploy
6
Regression UFT Functional test
Security Dynamic Testing
Test Data Mgmt
7
UFT/WebInspect/TDMTest
Defects & test results
HCM deploy K8S Deploy Mesos deploy Mobile Deploy
8
ALM/PC/WebInspectTest
Performance Testing
Optionally Security Dynamic Testing
9Defects & test results
Test results Issues/Defects Traceability
from story, code, build, test, result, defect, release
Test & quality metrics
10
Defects & Results
11
Local Unit tests LeanFT test
1
Developer
12
Security SME
QA
LeanFT/UFT Test Cases
GITLab
Speed of DeploymentComparison (VM v Docker)Feature
sentiment
FeatureUsage
MTTRs workaround
Security
Velocity
Quality
Productivity
Measuring continuous integration & continuous testing
Metrics
Mean time to restore service with fix
RequirementsCoverage Ratio
DeploymentSuccess Rate
Speed/Frequency ofBuild Verification
Security TestPass Rate
Speed ofDeployment
Frequency ofDeployment
Lead, Waite &Cycle Times
Why Micro FocusFor DevOps, continuous integration & continuous testing
The Micro Focus software services difference
Model Office
Your reference
Operating Model
Your transformation
Transformation Workshop
Roadmap to success
Education and MoC
Driving your adoption
Containerization
DockerEE, K8S, micro-services
ChatOps
Proven customer success
KPIs and analytics
Your management approach
Mobility and Security
Industry leading solutions
IT4ITThe IT4IT standard provides a vendor-neutral, technology-
agnostic, and industry-agnostic reference architecture for
managing the business of IT, enabling insight for continuous
improvement.
SAFe – Scaled Agile FrameworkScaled Agile Framework (or SAFe) is an agile software
development framework consisting of a knowledge-base of
integrated patterns intended for enterprise-scale Lean-Agile
development.
BEST PRACTICESInsustry best practices that discuss the full lifecycle
STRATEGY TO PORTFOLIODrive IT Portfolio to business innovation
S2P REQUIREMENT TO DEPLOYBuild what the business needs, when it needs it
R2D
REQUEST TO FULFILLCatalog, fulfill, & manage service usage
R2F DETECT TO CORRECTAnticipate & resolve production issues
D2C
IT4IT – YOUR MOSTLY COMPLETE VIEW OF
INFORMATION AND TECHNOLOGY
Provides the strategy to balance and
broker your portfolio1
Improves data quality for decision-
making3
Provides a unified viewpoint across
PMO, enterprise architecture, and
service portfolio
2
Provides KPIs and roadmaps to
improve business communication4
STRATEGY TO PORTFOLIO (S2P)
Provides a framework for creating,
modifying, or sourcing a service1
Enables visibility of the quality,
utility, schedule, and cost of the
services you deliver
3
Supports agile and traditional
development methodologies2
Defines continuous integration and
deployment control points4
REQUIREMENT TO DEPLOY (R2D)
Helps your IT organization transition
to a service broker model1
Efficiently manages subscriptions
and total cost of service3
Presents a single catalog with items
from multiple supplier catalogs2
Manages and measures fulfillments
across multiple suppliers4
REQUEST TO FULFILL (R2F)
Essential SAFeEssential SAFe is most basic configuration of the
framework and it provides the minimal elements
necessary to be successful with SAFe
Large Soliution SAFeLarge Solution SAFe is for enterprises that are
building large and complex solutions, which do not
require the constructs of the portfolio level.
Portfolio SAFePortfolio SAFe provides portfolio strategy and
investment funding, Agile portfolio operations, and
Lean governance
Full SAFeFull SAFe represents the most comprehensive
configuration. It supports building large, integrated
solutions.
SAFe– YOUR MOSTLY COMPLETE VIEW OF
THE AGILE ENTERPRISE
LARGE SOLUTION SAFeLarge Solution SAFe is for enterprises that are building large and complex solutions, which do not require the constructs of
the portfolio level.
FULL SAFeFull SAFe represents the most comprehensive configuration. It supports building large, integrated solutions.
Backlog Elements in SAFe
25
ENTERPRISE DEVSECOPS
PLANDEVELOP
TESTRELEASE
DEPLOY OPERATE
Enterprise DevOps starts with Business Demand and ends with Customer Value
while security is a major part of the automated process
Enterprise DevSecOps
SECURITY
Enterprise DevOps
27
Continuousplanning
PPM, ALM Octane,
ALM.Net
Plan
Define
Continuousintegrationand testing
intelliJ,Eclipse,VSALM Octane
LeanFT, SV, SprinterFortify (1, 2))
Dev & Build
Test
Continuousdeployment and
releaseJenkins& ALM &
Chef, HCM (CODAR)Docker EE (3)
Release
Deploy
Continuousoperations
Docker Mgmt, CMS, SMA-XSiteScope,
AppPulse(4)
Operate
Monitor
Continuous Security
Continuous assessment
DemandCustomer Experienc
e
Continuous Collaboration
(1) SSC,Maven, Cradle, GITLab, Nexus, Jenkins, verigreen, Docker, Docker Registry (2) UFT/LeanFT, Performance Center, Service Virtualization, Network Virtualizatio, Mobile
Center, Fortify, Weblnspect, ALM (3) K8S, Mesos, Mobile Center, UCMDB, SMA(4) BSM, Fortify AppDefender
Managed in a Controlled yet Open Ecosystem(Open Source Solutions are pervasive in Public Sector)
PLAN
OPERATE
BUILD
TEST
RELEASE/DEPLOY
29
Value driven portfolio
optimizationSteering committee
demands
Programs Projects Requests Epics FeaturesPortfolio
Sub Portfolio
Sub Portfolio
Program
Program
Agile Project
Project
Program
Agile Project
Epics
Program
Program
Agile Project
Project
ALM OctaneAgile Project
JIRAAgile Project
ALM OctaneAgile Project
JIRAAgile Project
CA RallyAgile Project
VersionOneAgile Project
Budget
Strategy
Resources
application development vs business goals
governance & industry regulations
Public Sector is still Hybridmultiple toolchains
(open source, commercial), methodologies, frameworks
Application Lifecycle Management at Enterprise Scale
We help customers manage hybrid application lifecycles (Waterfall, Agile, DevOps) and increase velocity and quality of testing.
32
Key Points
• Plan – think it thru • Process – understand it• Procedures – document them• Build It – put it together, piece by piece• Test – make sure it is working, verify • Review• Evaluate• Improve
• Don’t try to accomplish too much, too fast• Move at a speed your organization is comfortable with
33
Civilian Agency
• Issue was –• Dropped code • Manual testing • Experienced too many issues in production• Systems where not reliable
• Desired –• Better Quality • Higher reliability in production
• Solution – created test management,• Created test scripts• Executed automated functional test• Ran regular regression test • Then performance test • Monitored systems in production reusing functional test
• Instituted running SiteScope to monitor
34
Homeland Agency
• Issue was –• Waterfall methodology • Manual testing• Poor Quality
• Desired –• Embrace Agile• Test Earlier – shift left • Improve Quality• Deliver Faster
• Solution –• Leverage MF ALM as test repository • Built UFT scripts for some functionality• Leveraged Selenium tests from Agile sprints • Integrated CI into SDLC• Integrated JIRA defects with ALM • Tested more thoroughly and Increased Quality• Reduced Defects and Risk
35
Defense Organization
• Issue was –• Manual testing• 600+ test plans • 50 + SMEs • 6 month effort
• Desired –• Shorter cycle• More testing via Automation• Find Issues earlier in the SDLC
• Solution –• Leverage MF ALM for repository• Automate test scripts with UFT• Regression test in Dev and Pre Prod • Improve the CD effort • Cloud Hosted solution
36
Civilian Agency • Issue was –
• Poor Requirements management • Manual testing • Weak Defect management
• Desired –• FedRAMP approved cloud hosted solution• Test Management solution• Better Requirements management
• Full lifecycle traceability • Improved Defect resolution
• Solution –• AWS GovCloud Hosted Test Management • Leveraged MF ALM
• Requirements management• Test cases• Test scripts• Defect management
PROVEN LEADERS LEADING w/LEAN AGILITY
Successful introductions of Lean &
Agile methods in Human
Resources, Marketing, Corporate
Strategy, Risk & Compliance,
Legal and FinanceHOLISTIC APPROACH
Successful adoptions at 5 levels of
scale: Team, Train, Value Stream,
Portfolio & Enterprise
INNOVATION w/LEAN
AGILITY
Developed advanced techniques in:
• Agile Metrics
• Balanced Scorecard
• Lean-Agile Portfolio & ProdMgmt
• Team & ART Benchmarking
• Digital Strategy
GOV-WIDE REACH,
PERSONAL TOUCH
Operated in most agencies over
past 21 years , Commercial Div
PL
LE
HA
IN
GR
B12 STRATEGY CONSULTINGB12 is a Full-Service Enterprise Solutions Provider
(2012)
(1998) XP (1998)
On GovCloud
with
B12 Micro and Managed Service Solutions
Small Business Program
Small-Business
Woman-Owned
Minority-Owned
Org TransformationVideo Analytics Platform
with UniqueContent
Micro Focus Center of Excellence
DOD/Intel Cleared
GSA Schedule 70 PrimeAlliant II, EAGLE II, TIPSS, STARS, GITSS, COMMITS
subcontract positions
ADM
IM&GSecurity
ITOM
and legacy
Gov aaMS Offerings:
AM, DevOps, Gov Compliance
ITOM, Docker, TBMGovLean, & PPM
Thank You. Questions?
Shannon Lake – [email protected] - 650-417-1000 (cell)