Software Risk Management:A Primer for IT Executives

Featuring research from

Controlling the Structural Drivers of Business Risk

Software Risk Management: A Primer for IT Executives

2

The following section is excerpted from Predicts 2010: Agile and Cloud Impact Application Development Directions. To view the full note, click anywhere in the section.

Strategic Planning Assumption: Through 2015, the shift toward cloud architecture will create demand for new skills, practices and objectives for software quality.

Analysis By: Thomas Murphy

Key Findings: Software quality is often a poor misnomer for the current practice of risk management applied by most companies when it comes to practices and scheduling in software projects. The focus is not to drive quality, but to mitigate risk. While this is a viable approach, it also goes together with a concept that quality equals the absence of defects. Although this is theoretically true, the application is often too narrow to say that from this, quality software is delivered. The International Organization for Standardization (ISO) produced a standard (9126) that is generally ignored, because quality costs, and often is not seen as providing a return on investment.

However, as organizations seek to drive down maintenance costs and adapt to the shorter project life cycles found in agile practices, there is a need to focus efforts on a broader quality definition. In addition, organizations will need to invest in additional tools and skills to deal with increasingly complex distributed applications. Development frameworks may hide some of the complexity of creating these applications, but it won’t help with the testing of applications.

We are seeing strong growth now in tools that support a more automated test lab environment. This includes:

• Virtuallabmanagement

• Virtualizationofservices

Excerpt from:

Predicts 2010: Agile and Cloud Impact Application Development Directions

Application Structural Quality is the Key to Software Risk Management is published by CAST Inc. Editorial supplied by CAST Inc. is independent of Gartner analysis. All Gartner research is © 2010 by Gartner, Inc. and/or its Affiliates. All rights reserved. All Gartner materials are used with Gartner’s permission and in no way does the use or publication of Gartner research indicate Gartner’s endorsement of Cast’s Software products and/or strategies. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.

2Predicts 2010: Agile and Cloud Impact Application Development Directions

4Software Risk Management

6About CAST

3

3

• Improvedtoolsfortestdatamanagement,includingsubsettingand data masking

• Integrationintolifecycletoolstoimprovetraceabilityandautomation of workflow, and to close gaps in the common bugs that cannot be reproduced in tester/developer interaction

However, these are all just improvements to business as usual. While ALM tools provide better accountability to requirements, quality software has a variety of attributes not directly connected to normal requirements, including:

• Understandability

• Completeness

• Conciseness

• Portability

• Consistency

• Maintainability

• Testability

• Usability

• Reliability

• Structure

• Efficiency

• Security

The ongoing promise of evolving Web application architectures is to deliver applications and services that are customizable by business analysts and end users. Just as many organizations have moved more than 50% of their “development” budgets into packaged implementations, we believe that this trend will continue with increased capabilities for non-developer-targeted development. However, companies that seek to utilize technology to drive business innovation will evolve a more holistic view of software quality, because without it, they will not be able to support the ever-increasing maintenance burden.

Market Implications: The shift first toward SOA, then to rich Internet applications has stressed the ability of testing tools to keep up with technology shifts, and for testing teams to keep up with the pace of technology and application changes. The complexity of testing scenarios requires vendors to also deliver a broader spectrum of tools. This is resulting in a number of new companies and products coming to market, and will also result in increased acquisition activity as existing market leaders look to fill out their solutions.

While many organizations will be attracted to the promise of reuse from SOA, success will be limited because of the lack of skills and structure to support reusable assets. Reuse requires a view toward governance, ownership and quality.

Because software quality can’t be tested at the end, organizations will need to look at facilities and practices that drive quality through the development life cycle. This will include using practices from agile, such as TDD, and using tools that drive repeatable processes, such as continuous integration (CI). This will also create a continued drive for the use of ALM solutions that provide integration across the life cycle.

A great challenge will be dealing with development that happens outside the traditional IT process. Simplified business process management (BPM) and mashup tools make it easy for business analysts and end users to quickly assemble new solutions. However, this requires that the underlying components are stable, secure and scalable. It also requires that organizations are consistent. These requirements will continue to drive the market for static analysis tools and service registries and repositories.

Recommendations:

• Developtestingpracticesandexpertiseinsecurity,scalabilityandautomation.

• Drivepracticesthatdrivequalityfromstarttofinishonaproject.This includes shoring up weak requirements practices.

• Establishqualitycareerpathandstandarddefinitionstosetexpectations and drive consistency.

Source: Gartner

4

Software Risk Management

The process by which IT business software is built and the resulting software product itself are to some extent intertwined. It’s tempting then to think that when we have reliable, repeatable processes for building the software product, the quality of the resulting product will be equally good.

Despite that temptation, we have all known first hand that an application delivered on time, on budget, and even on scope cannot achieve its business goals if it is slow, behaves unpredictably, or compromises privacy. Moreover, a poorly built application is expensive and slow to respond to business, further eroding present and future business value. Nonetheless, most discussions of managing software risk continue to equate the quality of the process with the quality of the resulting product.

To truly manage the business risk of applications, we must move beyond the quality of the process to the quality of the product itself. The main aim of this article is to distinguish three kinds of software product quality: functional, non-functional, and structural quality and explain why structural quality is essential for managing the root drivers of IT costs and business risks in your mission-critical applications. Structural quality metrics enables us to understand, predict, and control the key drivers of software costs and business risks.

1. Why Software Quality is Critical

Software is the backbone of the modern enterprise. Software animates critical elements of an enterprise’s value chain. These statements are obvious enough to be clichés. But current conditions – both technological and business – make modern value chains increasingly difficult to animate without incurring large amounts of business risk.

However, as Thomas Murphy points out in the excerpt above, software quality is often erroneously equated with mitigating risk in “practices and scheduling in software projects.” There is much more to software risk than that. The main, if not only, reason for building and maintaining applications is for the business value they generate. With this in mind, let’s distinguish three kinds of business risk from software applications.

1. Delivery Derailment Risk – risks that add IT cost or stop business revenue due to delayed launch or cancellation.

2. Business Case Risk – risks that affect the quality of a delivered application; even though the application works, it doesn’t work as well as it should. The number of successful transactions per unit time cannot be completed to fulfill the benefits articulated in the business case.

3. Business Opportunity Risk – risks that make the application hard to maintain and change in the face of pressing business demand. The resulting loss of agility damages future business revenue.

Managing delivery derailment risk alone is insufficient for generating business value. Reliable project management processes and the right functionality are nothing if the application works unpredictably, is slow, or breaks down often. In addition to on-time, on-budget and on-scope delivery, business value is generated by the functionality working like it should. When the application is not performing like it should you cannot achieve the benefits articulated in the business case.

Unlikethequalityoftheprocess by which software is built, enhanced, and maintained, functional, non-functional, and structural quality have to do with the software product itself – the asset that generates business value. Managing software quality by

managing delivery risk alone only addresses a part of the problem. It’s like addressing the symptoms of a disease rather than taking aim at curing its cause. To get to the root causes, we have to define, analyze, and measure software product quality.

2. Three Kinds of Software Product Quality and the Importance of Structural Quality

Let’s begin by distinguishing three basic types of software product quality.

1. Functional Quality – a measure of what the software does versus what it’s supposed to do.

2. Non-Functional Quality – a measure of how well it does it versus how well it’s supposed to do it.

3. Structural Quality – a measure of how well it will continue to perform as it is meant to in the future.

When it comes to the quality of the software product, functional quality alone is not enough. If all that matters is having the right functionality, then every car that lines up on the NASCAR starting grid would win the race! But of course, winning the race takes more than satisfying the functional specification – it takes superior performance in the real world.

Similarly, non-functional quality is not enough. Non-functional quality focuses on the visible behavior of the software – the availability and latency of critical transactions. While this is important (in addition to the software’s usability), these performance indicators are skin deep. To equate them with product quality would be to equate, for example, the destruction left in the wake of an uncontrolled skid with the quality of the suspension system that was the root cause of this destruction.

5

5

Availability and latency are classic examples of “visible” or “above-the-waterline” metrics. They are rear-view mirror metrics with little or no predictive power. They tell you how the system is doing (symptoms) but not why things are going well (or badly).

On the other hand, structural quality measures how well the application is designed and how well it is implemented (the quality of the coding practices and the degree of compliance with the best practices of software engineering that promote security, reliability, and maintainability. Structural quality metrics track the root causes of application latency and availability. They are forward-looking metrics that enable us to control how an application performs, how readily it can be enhanced in response to urgent business requests, and how much it will cost to maintain.

Let’s consider what structural quality would be for a house. In this analogy, structural quality would not be about the number of rooms or the way in which the rooms are furnished. Rather, structural quality would be about the engineering design (e.g. where the load-bearing walls are placed, the strength and pliability of materials used) and how well the materials come together (e.g. the soundness of joints, the organization of the electrical and plumbing lines). A house with high structural quality is typically easy to maintain and extend.

But what do structural quality metrics look like and what does it take to measure them? Let’s consider those questions next.

3. How to Measure Structural Quality

Structural Quality is Contextual

The fundamental challenge of software product quality is that it is contextual. The quality of a single component depends on its local and global environment. The quality of a single component cannot be evaluated independently from its environment.1

Analyzing the quality of modern applications in the context of the numerous interconnections with other code, databases, middleware, and APIs is monstrously complex. It can only be accomplished with an automated system that analyzes the inner structure of all components and evaluates their interactions in the context of the entire business application.

Moreover, the component and/or its environment changes as a result of technology upgrades, user needs, and business needs. This means that any system for measuring product quality must have both breadth and depth.

• Breadth:Comprehensivecoverageoftheentire system from end to end. In modern systems, this means it has to cover a multiplicity of languages, technologies, and frameworksallthewayfromtheGUIfrontend to the middleware to the database.

• Depth:Adetailedarchitectural/logicalview of the entire system from end to end. The quality measurement system must be able to create detailed architectural maps of the entire system -- views of all the components and how they are inter-connected. It must be able to capture the logical aspects of the system, not simply the physical representation of it. It must be able to use this detailed logical view to evaluate the product quality of the system in the context of the entire system.

Conclusion

Application quality is often equated with the results of testing or with being able to manage delivery derailment risks. This is dangerous because it completely misses the key reason for building and operating business applications: the creation of business value.

To truly manage the cost and business risks of your mission critical applications you must move beyond process metrics to product quality metrics and in particular, measure the structural quality of applications.

Structural quality metrics are forward looking and actionable – they go beyond functional and non-functional quality to the root causes of application costs and business risks. They give you the visibility and control you need to manage your mission-critical applications.

The multi-tier, multi-language, and multi-platform nature of modern applications make automation essential for measuring structural quality. No human or team has sufficient end-to-end visibility of the entire system. Moreover, because structural quality is contextual, it requires sophisticated algorithms to analyze and measure it.

In addition to the breadth of technology coverage and the sophistication of contextual quality analysis, automated systems for analyzing and measuring software quality must also be able to provide detailed information on the root cause of quality problems, and provide practical guidance on how these root causes can be fixed once and for all.

When these quality metrics are measured over time, they provide valuable information on quality trends – actionable information for prioritizing focus areas and allocating resources for improvement. And these very same quality metrics (the change in their values) serve to measure the effectiveness of these improvement efforts.

Source: CAST Inc.

1Olivier Bonsignour and Bill Curtis, “Why Application Quality Is Different From and More Important Than Code Quality” (http://www.slideshare.net/jsub/application-quality-is-not-code-quality)

6

The CAST Application Intelligence Platform is the only enterprise-grade software quality assessment and performance measurement solution available in the market today. The CAST solution inspects the source code, identifies and tracks quality issues, and provides the data to monitor development performance.

CAST can read, analyze and semantically understand most kinds of source code, including scripting and interface languages, 3GLs, 4GLs, web and mainframe technologies, across all layers of an application(UI,logicanddata).Byanalyzingalltiersofacomplexapplication, CAST measures quality and adherence to architectural and coding standards, while providing visual specification models. Managers get real time access to this information via a web interface by which they can proactively monitor, measure and improve application health and development team performance.

CAST’s unique technology is the result of more than $80 million in R&D investment. Top engineering talent, dedicated to building the best technology for assessing the structural quality of mission-critical applications, has made CAST the leader in Automated Application Intelligence. CAST’s mission is to use software measurement to transform application development into a management discipline.

Founded in 1990, CAST has helped more than 650 organizations worldwide speed IT delivery to the business, mitigate risks in production, improve customer experience, and reduce the total cost of application ownership. CAST is listed on NYSE-Euronext (Euronext: CAS) and serves Global 2000 organizations worldwide withaglobalnetworkoflocationsintheUSandEurope.

About CAST

www.castsoftware.com CAST Headquarters North America: +1 212-871-8330 Europe: +33 1 46 90 21 00

Learn more about CAST

www.castsoftware.comblog.castsoftware.com

www.facebook.com/castonquality www.slideshare.net/castsoftware www.twitter.com/OnQuality