120908 Cast Forrester Webinar Final
30
-
Upload
alliance-global-services -
Category
Technology
-
view
1.191 -
download
0
Transcript of 120908 Cast Forrester Webinar Final
Managing Risk and Quality In Today’s EconomyMargo VisitacionVice PresidentForrester Research
3Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Software Quality Assurance is as much
about business risk as it is about software
performance
4Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Agenda
• Why quality, why now
• Why should business drive QA?
• Dynamic QA – processes for the 21st century
• Today’s quality assurance organization
• How to prepare for adaptation and adoption
5Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Why Quality, Why Now?
• Poor software quality costs over $60B per year
• Finding, repairing defects = approximately 35% of project budget
• Developers generally find only about 50% of their own bugs
• Typical testing only finds 75% of potential defects
6Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Poor internal quality is a major contributor to high maintenance spending
33%
33%
67%
67%
2007*
2008
Source: Enterprise And SMB Software Survey, North America And Europe, Q3 2007*Source: Forrester Business Technographics September 2006 North American and European Enterprise Software Survey
Base: 680 North American and European enterprises*Base: 451 North American and European enterprises
“Approximately what percent of your software budget will go to new initiatives and projects versus ongoing operations and maintenance?”
7Entire contents © 2008 Forrester Research, Inc. All rights reserved.
•Customer tolerance for defects lower each year – quality can affect spending in the long term
•IT spending forecasts are being recast – and it’s not a pretty picture – absent quality can trend toward budget cuts in the wrong places
Why Quality, Why Now?
8Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Why Should Business Drive QA?
Drivers:
Reduced bandwidth compromises successReduced bandwidth compromises success
Meeting shifting market objectives – Meeting shifting market objectives – lower tolerance for wastelower tolerance for waste
Reduce frustration from IT “not getting it right”Reduce frustration from IT “not getting it right”
Business domain expertise trumps allBusiness domain expertise trumps all
9Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Business Drivers Require Greater Examination of Risk in QA Practices
►Internal quality: The way an application is constructed
►Will the application perform as required?
►Have we done everything to prevent security leaks?
►Have we considered liability?
►External quality: The way an application behaves
►Are we getting expected outputs?
►Is the application usable?
►Are we as defect-free as possible?
10Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Traditional QA Practices
From Quality Assurance Versus Quality Control, December 2004
11Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Agile Practices Take Development to the 21st Century
1 month
12Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Making Quality More Adaptable - Dynamic QA
Business determinesobjectives, sets
requirements
QA determines“testability”
CollaborativeDetermination
QualityCriteria
Continuous BuildsApplication Design and
Test Development
QA advises BAs & DEV on testing criteria
and quality thresholds
Integrated Quality
Processes
UATPost
Mortem
13Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Planning, Strategies and Resource Allocations
• Quality planning is more than determining what you’re going to test
– Business determines highest risks and values
– Include How, Why and Value
– Metrics, acceptance and performance criteria determined at kick off
– Analysis criteria for application lifecycle
• Strategies extend to resources
– Leverage internal IP
– Make best use of outsourcing
– Earlier analysis and inspections to bake quality in and optimize resource usage
14Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Key Actions in Dynamic QA
• Test management
– Test planning/strategies
– Analysis and quality design
– Resource planning and allocation
– Prioritization
– Risk adjusted testing
• Visibility
– Collaboration
– Defect and change management
– Expectations management
15Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Prioritization and Risk Adjusted Quality Processes
Taking a portfolio view of quality• Align emphasis with business objectives, technical
complexity• Prioritize based upon objectives, resources and risks• Ensure that processes include internal and quality • Metrics must show coverage, business
acceptance, value to stakeholders
• Risk measures must be prioritizedbusiness exposure, liability, complexityability to deliver• Business must sign off on risk levels at the test planning stage and validate at theacceptance stage• Test planning must be based upon risks to business and ability to support
Include risk assessments in making decisions about qualityTech Imp
H
H
M
Bus Imp
M
L
L L
M
Requirement
XXXX
XXXX
XXXX
XXXX
16Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Visibility and Collaboration
• Role appropriate dashboards
• Universally understood metrics
• Standard measures
• Relative contextual information
• Easy to access
• Code complexity
• Adherence to standards
• Defect removal efficiency
• Mean time to detect
• Mean time to repair
• Stakeholder quality
17Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Investing in Quality Can Reduce Risk, Cost and Raise Value
Average
• Defect Potential – 1.00 (Requirements)
• Average Defect Removal Efficiency – 85%
• Delivered Defects - .75 (per FP)
Best in Class
• Defect Potential – 0.40 (Requirements)
• Average Defect Removal Efficiency – 96%
• Delivered Defects - .13 (per FP)
• ROI - > $15 for every $1 spent
Capers Jones 2008
18Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Investing in Quality Can Reduce Risk, Cost and Raise Value
0
20
40
60
80
100
120
Requirements Code Operations
Cost to Repair
19Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Today’s QA OrganizationCharacteristics Description
Location
Reporting
Focus
Management
Make – up
Testers
More Mature – Development Organization or Operations/Service Delivery
To CIO or Head of Development Org. Peer to Development Mgmt
Increased Emphasis on Strategic Orientation – varied technical requirements
Shifting Skill Sets – Vendor, Relationship Mgmt
Leadership and IP – internalTesting resources –combination internal/outsourced
Increased technical expertise – testing requirements expanded
20Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Keys to Adoption
• Universal language
– Glossaries
– Process
– Hand-offs
– Internal/External quality
• Business context
– Risk
– Testability
– Usability
– Performance
• Automate the process to open doors
– Eliminate barriers wherever possible
– Test management tools are key
– Leverage integration with other tools to support collaboration
• Educate the executives
– What’s in it for them
– Include metrics
21Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Recommendations
• Make risk assessments part of test planning AND execution
• Encourage business to determine more than just requirements
• Empower QA to be active part of PROJECT lifecycle
• Raise the bar on QA career path to support risk and business driven testing
• Use tools to enable, not hide critical information
22Entire contents © 2008 Forrester Research, Inc. All rights reserved.
Thank you
Margo Visitacion
+1 856-334-8522
http://www.forrester.com/rb/analyst/margo_visitacion
www.forrester.com
Achieve Insight. Deliver Excellence.
CAST Application Intelligence Platform
December 2008
CAST Application Intelligence Platform
Automated analysis of entire applications Immediate, unbiased quality assessment Executive level of synthesis & trending Drill down to root cause in the source code
Manage Risk at Less CostManage Risk at Less Cost
Transparency! Automated.
Analyzing the entire business application
Database
Data Management Layer EJB – Hibernate - Ibatis
DatabasesDatabasesFilesFiles
Web
Services
CICS Connector
Enterprise Applications
Legacy Applications
Middleware
Presentation Tier
Business Logic Tier
Data Tier
Only CAST can analyze thisOnly CAST can analyze this
Web / Client Server Applications ASP/JSP/VB/.NET
Application Logic Java, C++, … Frameworks Struts MVC, Spring
COBOLCOBOL
CICS Monitor (Cobol)
Tuxedo Monitor (C)
BatchShell Scripts
Deep structural analysis of software quality
Transferability
Changeability
Robustness
Performance
Size
Naming Conventions
Documentation
Architecture
Complexity
Package naming Class naming
Interface naming
Package comment
Class comment
Method comment
Package size
Class size (methods)Interface size
Class complexity (Inh. depth)
Class complexity (Inh. width)
Artifacts having recursive calls
Method complexity (control flow)
Maintainability
Security
ProgrammingPractices
File conformity
Dead code
Controled data access
Structuredness
Modularity
Encapsulation conformity
Empty code
Inheritance
Immediate ImpactImmediate Impact
Application Quality
On-Going ImpactOn-Going Impact
Ove
r 80
0+ a
rch
itec
tura
l an
d l
ang
uag
e-sp
ecif
ic c
od
e ch
ecks
Health FactorsQuality IndicatorsQuality Metrics Subset Application Quality
Multiple artifacts inserting data on the same SQL table
Coupling Distribution
SQL Complexity Distribution
Profile, assess, and benchmark applications and teams
Project #nProject #3
Project #2
Major global telecommunications company
230,000 employees, $100 billion revenue, 40 million accounts Billing & OSS Solutions
120 billion call records and 1 billion invoices per year Also, SAP, Siebel, all front end apps that power e-commerce sites
Running CAST one or two times per quarterly release Penalties in contracts based on CAST Aggregation of CAST metrics into C-level management dashboards
Parser Agent
Team #1
Team #2
Team #3
Team #4
Neutral & independent vendor unit is running the CAST AI Center for the customer
Project #1 • 150+ applications• 4 apps silos, with 50+ CAST consumers
plus management in each
CAST Dashboard
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1.1 2.0 2.5
Vendor A
Vendor B
Vendor C
Vendor D
Vendor D facility
Insights for both buyers and vendors
Management visibility
Guidance for developer
• Ensure teams are working efficiently• Manage stability, security & project risks• Better relationships with outsourcers
• Ensure architectural compliance• Ensure projects are not at risk• Metrics – quality, quantity,
technical
• Immediate feedback regarding code qualityInternal and Outsourced Teams
Division CIO and VP, Apps Delivery
Project Managers, Architects and Quality Assurance
Java developers .NET developers DBAs
Solution Information What IT constituents need
Overall team and application KPIs
Overall team and application KPIs
Measure of conformance to standards &
architecture
Measure of conformance to standards &
architecture
Identify specific application
quality issues
Identify specific application
quality issues
Identify code-level style
and quality issues
Identify code-level style
and quality issues
CAST AIP
Q & A
