UALITY AND THE ECHNOLOGY LIFE CYCLE - … · Defect tracking Functional testing ... commodity...

P R E S E N T A T I O N

International Conference On

Software Management &Applications of Software Measurement

February 15-19, 1999 • San Jose, CA

PresentationPaperBio

Return to Main Menu T6

Thursday, February 18, 199910:30AM

QUALITY AND THETECHNOLOGY LIFE

CYCLE

Kirk HendricksonFace Time Communications

1

Addressing Quality alongthe Product Life Cycle

Presented by Kirk Hendrickson

Co-developed with Elisabeth Hendrickson

2

Categories of QualityAssurance ActivitieslDefect detectionlDefect preventionlProcess definitionlMeasurement

3

Simplified Development Cycle

Build It

Is it done yet?

Specify Details

How are we going tobuild it?

Define Requirements

What are we buildingand why? Who arethe customers?

Release

Ready for prime time?

Test (Internal & Beta)

Did we built it right?

4

Defining the Product LifeCycle: The CustomersCustomer types as defined in Crossingthe Chasm:– Innovators

– Early Adopters (Visionaries)

– Early Majority (Pragmatists)– Late Majority (Conservatives)

5

Defining the Product LifeCycle: The Stages

Innovators

EarlyAdopters

Laggards

Pop

ulat

ion

Chasm

EarlyMajority Late

Majority

Time

6

Quality AttributesQuality AttributesQuality Attributes

Adherence to Schedule

Modeling Software QualityNeeds Along the Life Cycle

Demand For New Features


The customers' need for theproduct to contain newfeatures.

Characteristics associatedwith the overall perceivedquality of the product.


Demand For New Features

Timeliness of deliverybased on either an internalor external timeline.

7

Some Representative QualityAttributesl Reliability

l Customizability

l Scalability

l Interoperability

l Usability

8

Assess Buying BehaviorDuring Each Customer Stage

Each segment buys for different reasons• Innovators: seek whiz-bang technology

• Early Adopters: seek solutions not technology

• Chasm Crossers: focused “whole products”solving their specific business problems

• Early Majority: need “whole products” solving theirgeneral business problems

• Late Majority: resistant to new technologies

9

Innovatorsare...� Technophiles� Forgiving of

glitches, non-intuitive UI, andmost other softwaresins

� Interested inbleeding-edgetechnology

QA/Test activitiesinclude...� Source control� Defect tracking� Functional testing� Basic installation

testing� Possibly beta or

customer testing

10

Early Adoptersare...� Trying to gain a

competitiveadvantage

� Trying to solve aspecific problem

� Extremelyschedule-driven

� Willing to take risks� Project-oriented� Interested in

custom solutions totheir particularproblem

QA/Test activitiesinclude...� Project management� Configuration

management� Functional testing� Scenario-based testing� Installation testing� Configuration testing� Usability testing (begin)� Test automation

(begin)

11

Chasm Crossersare...� Trying to solve a

specific nicheproblem

� Risk averse� Quality conscious� Interested in a

“whole product”solution

QA/Test activitiesinclude...� Defined process� Usability testing� Scenario-based testing� Functional testing� Configuration testing� UI testing for

conformance tostandards

� Acceptance testing� Test automation

12

Early Majorityare...� Risk averse� Quality conscious� Interested in a

“whole product”solution

QA/Test activitiesinclude...� Usability testing� Scenario-based testing� Functional testing� Platform & configuration

testing� UI testing for

conformance tostandards

� Acceptance testing� Test automation (invest

heavily)

13

Late Majorityare...

� Price sensitive� Interested only in

mature productsthat have reachedcommodity status

� Slow to acceptchange (noradically changingUI’s please)

QA/Test activitiesinclude...

� Focused defectprevention efforts

� Regression tests toverify that changes orfixes do not affect otherareas of the product.This is bestaccomplished throughautomation.

14

Quality Shouldn’t Fall Into theChasm

Pressure to reduce time to market

Pressure to release more functionality athigher levels of quality

15

Aligning Quality Assuranceand Corporate Strategy

Corpor

ate

S

trateg

yQuality Assurance Efforts

16

Aligning Quality Assuranceand Corporate Strategy

Corpor

ate S

trateg

y

CompetitiveEnvironment

Quality

Assu

rance

Effo

rts

Customers’Needs

17

Using the Lifecycle Model

lProfile a typical customer based on lifecycle segmentlUse the profile (and real data where

possible) to determine the purchasedecision criteria and factors in post-sale satisfactionlFocus the quality assurance efforts

accordingly

18

Conclusion: The Key Question

What problem is your customer tryingto solve and how does your producthelp them solve it?

– All quality activities are a means to an end

– To define that end, understand theproblem

1998 Copyright Elisabeth Hendrickson, Kirk Hendrickson All Rights Reserved

Quality along the Life CycleKirk Hendrickson, FaceTime Communications

Elisabeth Hendrickson, Quality Tree Software, Inc.

November 30, 1998

ABSTRACT

Geoffrey Moore's books Crossing the Chasm and Inside the Tornado have had a tremendous impacton how high technology companies think about the market. Both of these books focus on theuse of the “technology adoption life cycle model” as a basis for strategic decision making.Moore introduced the life cycle model, shown in figure 1, as a framework for helping hightechnology executives understand the market’s product requirements at each stage in the lifecycle. Many high tech marketers and managers have turned these books into their new mantrafor dong business.

Innovators

EarlyAdopters

Laggards

Pop

ulat

ion

Chasm

EarlyMajority Late

Majority

Figure 1: Technology Adoption Life Cycle Model

Software Quality Assurance (QA) and Test managers can use this framework to understand howthe customers’ perception of quality changes at each stage in the life cycle. Changing qualityexpectations necessitate different testing strategies and different release criteria. The QAmanager can therefore use Moore’s framework to plan the test effort and QA activities that willhelp the company transition from the current phase to the next phase.

Furthermore, by applying Moore’s framework to quality assurance activities, we see anexplanation for a phenomenon that many QA managers have noticed: startup companies lackformal processes.

Some software development process aficionados espouse formal processes for all companies atall phases. These folks succeed at making testers and QA professionals feel that their company issomehow morally inferior for not following formal software development processes. They holdup a variety of standards to show how immature the organization is. However, it is more likelythat the technology, rather than the company, is immature. The company needs to ensure that thematurity of its development process keeps pace with the maturity of the technology and themarket. Attempting to apply an extremely mature process to an immature technology is just asdangerous as applying an immature process to a mature technology.

Quality along the Life Cycle December 29, 1998

Page 2 of 13

MODELING SOFTWARE QUALITY NEEDS ALONG THE L IFE CYCLE

Crossing the Chasm and Inside the Tornado make it clear that the market's expectations of qualitychange as a product moves through the technology adoption life cycle. The key to understandingthe quality requirements at each stage is to understand the characteristics of the customers ateach stage. The customer characteristics determine the quality needs.

We examine the quality demands of a customer using three dimensions: quality attributes,adherence to schedule, and demand for new features:

§ Quality Attributes are the list of software quality characteristics, such as “reliability” and“usability,” that are associated with the ability of the software to meet the demands of thecustomers.

§ Adherence to Schedule represents the customers' need for the vendor organization to deliver thesoftware according to an internal or external timeline.

§ Demand for New Features is the customers' need for the product to contain new capabilities. Ina 1.0 product, the product consists entirely of new features. Following the initial 1.0 release,new releases bring new features. Customers have different needs for new features atdifferent stages in the life cycle.

We chose to address these dimensions of quality in this paper because these are the factors thatthe development group can hope to influence. In contrast, the development group has littleinfluence on other dimensions of marketability and perceived quality such as pricing, distributionstrategy, or field support staffing.

QUALITY ATTRIBUTES

For the purposes of this paper, we selected a list of five representative quality attributes. Thesequality attributes are:

§ Customizable: The software provides the capability to adapt the general solution tospecific business rules. Customers can implement it with their standard look andfeel, including the vocabulary specific to the their internal business processes.

§ Reliable: The software does not corrupt data or act upon the data in an unusual andunforeseen fashion. There is a high mean time between failures; few “crashes.”

§ Usable: Easy to install, intuitive user interface, easy to learn, efficient work process,translatable to local languages, performs at expected levels.

§ Scalable: The system can scale to handle a significantly increased load or greaterdemands.

§ Interoperable: The software works with existing systems on existing platformswithout conflict.

This list of quality attributes is not intended to be a canonical list. We use the list to illustrate thechanges in quality concerns at each life cycle stage.


Page 3 of 13

ASSESSING BUYING BEHAVIOR OF EACH CUSTOMER STAGE

According to Moore, the sales process follows the technology adoption stages. The first step isto get your technology to the innovators, then to the early adopters, and so on. Each stage bringsnew customer incentives:

§ Innovators seek whiz-bang technology.

§ Early Adopters look for custom-tailored technological solutions that solves most of aspecific business problems.

§ Chasm Crosser customers need a specific “whole product” solution that addresses a nichebusiness problem completely. Chasm Crossers are the leading edge of the Early Majority.

§ Early Majority customers need a general “whole product” solution that addresses theirbusiness problems completely.

§ Late Majority customers adopt a new technology only after everyone else has debugged it.

Since the customers in each stage are motivated by very different factors, it is important tounderstand the corresponding quality demands. This understanding will give us insight into themost effective way to use QA resources.

The following sections discuss the customer profiles for each stage in more detail.

Innovators Buy Products That Move Technology Forward

Innovators are technophiles. If product is out there, they will find it. The challenge of selling totechnophiles is to get them to see the product as technologically compelling. Innovators arewilling to suffer abysmal documentation, unusable user interfaces, and lack of stability in order tobe a part of a new technology. They will cobble together their own systems calling upon theproduct company’s limited but expert technical support for the product.

Early Adopters Buy Products That Give Them a Strategic Breakthrough

The early adopters, also called visionaries, communicate with the technophiles to learn about thelatest technology and attempt to apply it to an existing problem. The early adopters are willing totake a product that still has some flaws if they are able to be the first and if it is customized fortheir needs. The early adopter market is much larger and affluent than the innovator market.Your company's first big sale will be to an early adopter. To get sales in this phase, it isimperative that you are close to your customer, understanding their customization needs, and thatyou provide a product that allows the visionary to make a strategic leap forward.

Chasm Crossers buy industry specific complete solutions

Chasm Crossers are members of the Early Majority customer segment who consider a giventechnology before the rest of the Early Majority customers because they have a specific problemthat they are trying to solve.

Chasm Crossers are concerned about the same quality attributes as Early Majority customers.The difference between a Chasm Crosser and an Early Majority customer is that the ChasmCrosser is looking at a very niche-specific solution to a critical problem. Moore describes this


Page 4 of 13

group as those looking for solutions to “broken mission-critical processes.” The specific solutionneeds to be a complete solution, not just most of the solution, before the chasm customers willbuy.

Early Majority Buyers move in a Pack

Like the chasm crossers, the early majority market, also called pragmatists, needs to find a wholeproduct solution. Entering the early majority market is the most difficult task in productmarketing. A company going through the transition from an early adopter market to an earlymajority market must make dramatic changes to succeed in their new market space.

Moore further divides the early majority market into two parts: the “Bowling Alley” and the“Tornado.” The first part of the early majority market is called the “Bowling Alley” because eachcustomer you win over is like a bowling pin that is likely to bring in other customers. This isbecause early majority buyers are risk averse, so they tend to buy the same product that theircounterparts at similar organizations purchased. Once a product enters the Tornado, it hasachieved widespread recognition and word-of-mouth recommendations.

Late Majority - Main Street customers

After a flurry of activity during the Tornado phase, the product enters “Main Street.” This is thepoint at which even conservatives adopt the product. The conservatives are looking for the basicproduct, at the cheapest price, with features that are most suited to them. They have no use forwhiz-bang features or a slick user interface. They want to buy a commodity. They are willing topay for service to support adoption of the technology. Companies whose products have hit MainStreet must pay careful attention to operating efficiencies in order to keep their costs in line withthe lower growth rate market.

ACHIEVING THE QUALITY EXPECTATIONS IN EACH STAGE

As we’ve seen, each stage brings with it new customer incentives. We believe that quality plays alarge role in customer buying decisions at each stage, but that the definition of quality changes asthe market changes, as the following table demonstrates:

Quality Concerns Innovator Early Adopter Chasm Early Majority Late Majority

Demand for NewFeatures

Innovative Specific toproblem

Specific toIndustry

Incrementalchanges

Little

Schedule Adherence Early tomarket

Must meetdemanding andaggressivetimeframe

Must meet orexceedcompetitor'sdelivery

Must meet orexceedcompetitor'sdelivery

Slow to adoptnew releases

Quality Attributes NoneCustomizable By Customer By Niche GenerallyReliable t t t t

Scalable t t t

Interoperable t t t

Usable t t t

Table 1: Quality concerns vary by life cycle stage


Page 5 of 13

By understanding what motivates customers to buy during each stage of the market, the qualityassurance manager can make decisions to ensure that internal product quality goals are alignedwith external quality expectations.

Table 2 provides an overview of the activities recommended for each stage. In table 2, theactivities for the phases are cumulative. For example, “developer unit test” appears first in theInnovator phase; developers should still be unit testing their work in the Late Majority phase.Activities in the testing column may become more automated with time.

Innovators Will Take What They Can Get

As Moore points out in Crossing the Chasm:

They [Innovators] are the ones who will spend hours trying to getproducts to work that, in all conscience, never should have been shippedin the first place. They will forgive ghastly documentation,horrendously slow performance, ludicrous omissions in functionality,and bizarrely obtuse methods of invoking some needed function - all inthe name of moving technology forward.

“Quality” to an innovator is compelling technology, not ease-of-use. As the QA manager at a hotstartup in the innovator phase, your primary charter is to ensure that the product does not harmthe innovator’s system. Innovators may tolerate crashes, but they will be less than amused if theproduct reformats their hard drive.


Page 6 of 13

CustomerTypes

QualityConcerns

Testing Activities Quality Activities

Innovators "Hot new thing"technologically

§ Developer unit testing§ Functional testing§ Beta/customer test§ Basic install testing

§ Source control§ Defect tracking

EarlyAdopters

Custom solutionsto target specificproblems

§ Functional testing§ Scenario-based testing§ Usability testing begins§ Test automation begins§ Install testing begins in earnest§ Testing of specific market

configurations and platforms

§ Project management§ Configuration

management§ Coding standard adopted§ Defect-based metrics§ Release criteria

consideredChasm Complete

solution to aniche problem

• Usability testing• Scenario-based testing• Functional testing• Configuration testing• UI testing for conformance to

standards• Acceptance testing• Test automation

§ A defined process in place§ Requirements

documented andmanaged

§ Specifications written fromRequirements

EarlyMajority

ReliabilityInstallationUsability

§ Usability testing (especiallynew features)

§ Functional tests§ Scenario based testing§ UI testing for conformance§ Acceptance/install testing§ Configuration and platform

testing§ Invest heavily in automation

and outsourcing

§ Internal productarchitecture documented

§ Code reviews andinspections

§ Code-based metrics§ Formal internal training

program§ Release criteria

established

LateMajority

Low costownership

§ Regression tests (automated)§ Outsourcing

§ Product maintenance only

Table 2: Testing and Quality Activities through the Life Cycle

Testing Requirements in the Innovator PhaseTest groups in companies in this phase often barely exist or are staffed with very inexperiencedindividuals. Testers are often encouraged to “pound” on the application rather than rigorouslytest it. This is reasonable because “pounding” is exactly the sort of testing that the productneeds at this phase.

Basic functional testing and installation testing are required, but full test coverage may actually bedetrimental to the success of the product since it could delay release. Automated testing is lesslikely to pay for itself in this phase, although some form of automated testing may be required toassure even basic functionality (load testing, for example).

Quality Activities in the Innovator PhaseThe innovator phase is a good time to implement basic development practices like source controland unit testing. If the organization has the bandwidth to take on additional quality activities,such as those described for the Early Adopter phase, below, so much the better. However, at thisphase in the technology’s life cycle, the most important goal is to release a compelling product,not to develop a comprehensive and mature process.


Page 7 of 13

Planning for Quality in the Next Phase: Innovators to Early AdoptersOrganizations in the Innovator phase must begin to plan for the Early Adopter phase in whichcustomers require greater reliability, better service, and tight schedule adherence. It is wise todevelop a small but knowledgeable team of QA professionals who are flexible enough to dealwith the oncoming changes and knowledgeable enough to introduce the new techniques whenneeded.

Early Adopters Require Strategic Leaps in a Hurry

Early adopters, also called visionaries, have different quality needs and expectations thaninnovators. Unlike the innovator, the prospect of being part of the hot new technology is notenough for an early adopter. A typical early adopter is looking for new technologies to providethem with a competitive advantage. They are willing to take risks but only if those risks willprovide them with a breakthrough improvement over the past.

Early adopters are under considerable time pressure to produce; they pass the pressure on to thesupplier. They “...tend to exert deadline pressure—the carrot of a big payment or the stick of apenalty clause—to drive the project faster.”

Large flaws in usability and scalability may persist, installation can be a bear, documentation canbe questionable, but the product must solve the customer’s problems. In order to assess thequality of the software, the Quality Assurance manager must put himself in the customer’s shoesto understand the problem that the customer is trying to solve.

The QA manager should understand that each release in this phase might target differentvisionary customers who require different feature sets. This is not the time for the QA managerto begin agitating at upper levels of management for reduced releases or general solutions inorder to improve efficiency. The company’s success depends on its ability to deliver completesolutions, not general solutions. The need for internal efficiency comes later.

Testing Requirements in the Early Adopter PhaseTesting at this stage should focus on:

§ Functional testing

§ Scenario-based testing

§ Installation testing

§ Specific configuration and platform testing

Functional testing and installation testing are critical to ensure that the product can provide thecustomer with the basic functionality the customer requires. Scenario-based testing responds tothe customer specific nature of the deliverables since each customer wants something slightlydifferent. Specific configuration and platform testing is necessary to ensure that the productworks in the customer’s environment.

This phase is particularly challenging because each customer expects deferential treatment.Customers may be on a first name basis with the developers since they require developmentsupport to implement the technology successfully. One way for the test group to achieve thislevel of focus is to dedicate individuals to specific customer projects, so that they can becomeintimate with the customer demands for the product that they are testing. This organizationmakes the test group decentralized. Having a decentralized test group has a number of


Page 8 of 13

advantages during this phase. The disadvantage of a decentralized group is that testers do nothave a support structure of other testers and centralized processes.

Quality Activities in the Early Adopter PhaseIn this phase, the development process becomes important both for meeting current marketrequirements and for preparing for the future. Although the process by which software for anInnovator market is created may be entirely ad hoc, the Early Adopters have higher qualityexpectations.

Project Management. The project is more likely to ship on time if project management is inplace. Project management includes schedule management, resource allocation, and planning.

Configuration Management. Now is also the time to move beyond source control to realconfiguration management. The difference is that a source control system simply tracks changeswhile configuration management is a discipline that allows the team to recreate any previousversion of the product quickly and easily. When managing a variety of customized solutions forthe customer, this capability is vital.

Coding Standards. Establishing coding standards that specify the “acceptable style” for thesource code during this stage helps ensure that as the product matures, the source code style willbe consistent. A consistent style makes the code easier to maintain: new developers need to learnthe company’s style rather than the style of each individual developer who preceded them.

Metrics. During the Innovator phase, metrics would have been largely meaningless. Now thereis likely to be enough data for it to be able to show statistical trends or at least provide interestinginsight into the status of the project. The first step in establishing a metrics program ismeasuring progress with defect-based measures.

Release Criteria. Finally, this is also a good time to consider setting release criteria that willassist executives in deciding whether or not the product is ready to ship.

These changes may be difficult for an organization with completely ad hoc processes to swallow.The QA manager must begin to make these changes during this phase, however, or thedevelopment organization will find itself completely unprepared for the next stage.

Planning for Quality in the Next Phase: Early Adopters to Early MajorityThere is a very large gap between the quality activities required in the Early Adopter phase andthe Early Majority phase. Therefore, the best preparation for the Early Majority phase is tofirmly establish the notion of process in the corporate culture. Successfully making the transitionfrom an ad hoc development process to a repeatable process is difficult.

Usability is not an overriding concern to the early adopter market, but it is an advantage inmaking the leap to the early majority market. Consequently, now is a good time to begin usabilitytesting.

The test group should consider starting to automate the testing of the main features of thesoftware application. Core features are those that are likely to remain the most stable movingforward. A well thought out automation strategy can take into account upcoming designchanges, and automating the core tests now prepares the test group well for the next phase.


Page 9 of 13

Chasm Crossers Need a Whole Solution

The Chasm that Geoffrey Moore describes occurs between the Early Adopters and the EarlyMajority phases. It is during this phase that the Quality Assurance manager can have the largestimpact on a company’s success or failure. However, whether in the chasm or in the earlymajority phase in proper, the customers’ are still early majority customers and thus have the sameconcerns.

The company must meet these new demands if it is to continue to succeed. The Early Majorityis considered the most prosperous customer segment because it is the largest group that is stillwilling to pay a premium for technology that provides a competitive advantage. The only way toget to the Early Majority is to appeal to early Chasm Crossers.

The QA manager now must ensure that each release is more stable than the last, that theorganization understands and meets customer expectations of quality, and that the organization iswell positioned for efficiency in the future.

Testing Requirements for Crossing the ChasmCritical testing activities at this phase include testing for scalability, interoperability, and usability.If the product fails in any of these areas, then it is likely to fail as a whole product.

Quality Activities for Crossing the ChasmDefined Process. In addition to managing projects, this is the time to introduce processmanagement so that there is a process for maintaining the process. Before this phase, having aprocess to maintain the process would have required too much overhead.

Requirements and Specification Management. The software is much more likely to meetcustomer expectations if those expectations are written down. Therefore, the project team needsdocumented requirements. Managing those requirements in the sense of tracking changes andensuring the requirements changes are disseminated and reflected in the software anddocumentation further ensures that customer expectations will be met. In addition, the testersand the technical writers can do their job more quickly and efficiently if they have writtenspecifications from which to work.

Early Majority Wants a Stable, Reliable, Easy to Use Product

Early majority customers are pragmatists. They are interested in quality and performance ratherthan new features. This represents a fundamental shift in buying behavior from the earliermarkets. Up until this phase, the company has been able to sell products on the strength of theirinnovation rather than stability, reliability, or performance. Now that quality is of paramountimportance, companies that have not planned ahead are poorly positioned to meet the newmarket demands.

The companies positioning and testing during the early majority phase also varies based on wherethe technology is in its life cycle.

Testing Requirements in the Early Majority PhaseThe test group needs to broaden their approach to include usability testing, UI conformancetesting, and acceptance testing in addition to the other tests already being performed.

This is also a good point to begin investing heavily in automated testing since the feature set andthe user interface are relatively stable. Automation can provide the breadth and depth ofcoverage that is hard to achieve in manual testing, while achieving acceptable ROI as the


Page 10 of 13

automation is reusable for many more releases. Automating in this phase also positions theorganization well for the next phase, the Late Majority, in which quality is still important butcustomers are generally unwilling to pay for it.

At this stage, it may make sense to outsource the testing of those aspects of the product notcritical to the competitive advantage of the product. For example, if the product derivescompetitive advantage from the breadth of applications with which it integrates, keep the testingof application integration in-house, but outsource other testing.

The test group also must have the foresight to see that the company will be reducing itsinvestment in quality assurance and product development when the product moves from thepragmatist to the conservative market place. If the test group has not made steps towardsdramatically increasing its productivity in this phase, the quality will suffer in the conservativestage as the investment in quality assurance declines.

Quality Activities in the Early Majority PhaseThe goal of the quality activities in this phase is to ensure that the development process isoptimized for efficiency and reliable results rather than creativity. From this phase forward,quality, rather than innovation, motivates customers. By now, the organization needs to havesuccessfully completed the transition from ad hoc processes to a repeatable development process.If processes are not in place, the organization will suffer when each major bug that ships resultsin numerous calls into technical support.

Moving forward, the organization will achieve further efficiency by implementing code reviewsand inspections to catch bugs even before they go to the test group. Coding standards tailored tothe organization and internal product architecture documentation will help developers writebetter code. Having code-based metrics in place will give managers visibility into the complexityof the code and the code coverage achieved by testing. These activities will go a long way towarddefect prevention and early defect detection.

Internal Product Architecture Documented. By this phase, the architecture of the productwill be mature an essentially unchanging. If it wasn’t documented before, it needs to bedocumented now.

Code Reviews and Inspections. Although it is a good practice to begin reviewing andinspecting code from the very beginning, most organizations do not. Organizations that choosenot to hold reviews or inspections in earlier phases need to begin doing so now. Reviews andinspections enable a team to catch defects much earlier than hands-on testing. They can also helpcross-train developers: everyone participating in the review will become very familiar with thecode.

Code-based Metrics. Two code-based metrics are particularly applicable here: complexity andcode coverage. Measuring the complexity of the code enables developers to pinpoint areas ofthe code that need to be simplified. Very complex code is more likely to break duringmaintenance mode. Code coverage enables the test organization to understand how completetheir tests are. Ensuring that the regression test suite offers relatively complete coverage now willmake maintenance, when few if any testers are assigned to the product full time, much easier.

Internal Training Program. In the next phase, the software will go into maintenance mode.The individuals who contributed the most to creating the system are unlikely to stick around tomaintain it indefinitely. It’s critical to the future success of the organization that it has processesin place to train new staff during this phase when most of the original authors of the softwareare much more likely to be available.


Page 11 of 13

Release Criteria Established. If release criteria are not already established, it is critical toestablish the criteria now. Early Majority customers expect the product to be mature and stable;release criteria help ensure that each release is at least as good as the last.

Planning for Quality in the Next Phase: Early Majority to Late MajorityThe market is unlikely to demand anything but maintenance releases in the next phase.Consequently, anything that can be done now to improve the efficiency of the development andtest process will pay off significantly in the next phase.

Late Majority Want Software Continuity

Unlike any other market, the late majority is not looking for a competitive advantage throughsoftware. They acquire new software strictly to keep up to date. They are extremely pricesensitive and risk averse. As Moore points out in Crossing the Chasm:

Conservatives like to buy pre-assembled packages, with everythingbundled, at a heavily discounted price. The last thing they want to hearis that the software they just bought doesn't support the printer theyhave installed. They want high-tech products to be like refrigerators -you open the door, the light comes on automatically, your food stayscold, and you don't have to think about it.

The late majority segment is not interested in new features. They do not care about schedulesince they are reluctant to adopt the latest release anyway. They do care greatly about quality:reliability, stability, performance, scalability, interoperability, usability, and ease of installation.

Unfortunately, although the conservatives' demand for quality is high, their willingness to pay isvery low. It is not cost effective to spend more than nominal amounts on development andquality assurance. By this phase, the testing efforts have to rely heavily on the testinginfrastructure established in the early majority phase. Because code changes at this point shouldbe relatively minor and compartmentalized, testing can focus on regression tests and useautomation wherever feasible.

The QA manager should consider outsourcing product testing in this phase. Products in thisphase no longer afford the company a competitive advantage; the organization should havedeveloped test documents that describe the test strategy in detail; and new releases are likely to beinfrequent. Consequently, testing can be performed more economically on a project basis ratherthan maintaining an entire staff to test the product. Finally, by outsourcing, the QA managerreduces the risk of over investing in an infrastructure to support a declining product segment.

The only important quality activity in this stage is defect prevention. The more cost effective thesoftware is to maintain, the better.

USING THE QUALITY ASSURANCE L IFE CYCLE MODEL

We believe that the most important conclusion in this paper is that the proper process for acompany is one that best supports achieving the quality goals set by the company’s currentcustomers. A process that is appropriate for an aerospace firm is likely to be too burdensome fora small internet start up working on hot new software. On the other hand, more mature softwaretechnologies, like relational databases, are no longer in the Innovator market and have had toadjust their internal processes accordingly.


Page 12 of 13

We hope that by applying Moore’s life cycle model to quality assurance, we have helped you tothink about quality assurance in your organization a little differently. Where are your company'sproducts on the life cycle model? Which attributes of quality are most critical for your currentcustomers and for the customers your company plans to target next year? How can the companydo a better job of supporting the product development needs during its current life cycle stage?How is your QA organization arranged? What type of structure might be better? The answersto these questions will help you, as the quality assurance manager, better organize and test for thecurrent stage of your product. We hope that the above discussion will also give you some toolsby which to communicate the quality needs of the organization to executives who are applyingthe technology life cycle model to the overall company strategy.


Page 13 of 13

B IBLIOGRAPHY

All direct quotations in this paper come from either Crossing the Chasm or Inside theTornado by Geoffrey Moore.

Moore, Geoffrey, Inside the Tornado, Harper Business, September 1995.

Moore, Geoffrey, Crossing the Chasm, Harper Business, September 1995.

Kaner, Cem, Testing Computer Software ITCP-US Computer Science Series, PublishersResources, February 1993.

McConnell, Steve M., Rapid Development: Taming Wild Software Schedules, MicrosoftPress, January 1996.

Maguire, Steve, DeBugging the Development Process, Microsoft Press, July 1994.

Freedman, Daniel P., Weinberg, Gerald M., Handbook of Walkthroughs, Inspections, &Technical Reviews, Dorset House, January 1990.

Kirk HendricksonKirk Hendrickson is a Director of Professional Services for FaceTime Communications, asmall startup just entering the early adopter phase of its market. Prior to joining FaceTime,Mr. Hendrickson was a telecomm and media strategic management consultant withPricewaterhouseCoopers, the Director of Operations for PC DOCS, Inc., and an engineerwith the Boeing Company. He received his MBA from the Amos Tuck School of Businessat Dartmouth. Kirk has published a number of management case studies focusing onvarious issues of management control systems, such as innovation management, goalcongruence through compensation and rewards, and intellectual capital management.Currently, Mr. Hendrickson also serves as a Director of Quality Tree Software, Inc. and is amember of its Board of Directors.

UALITY AND THE ECHNOLOGY LIFE CYCLE - … · Defect tracking Functional testing ... commodity...

Documents

Transcript of UALITY AND THE ECHNOLOGY LIFE CYCLE - … · Defect tracking Functional testing ... commodity...