MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

40
LESSONS LEARNED ADOPTING AGILE FOR FIXED-PRICE CONTRACTS By Glenn R. Fuller A Master of Science Research Practicum Submitted to the Faculty of The Graduate School of Management, at the University of Dallas in Partial Fulfillment of the Requirements for the Master of Science Degree in Information Technology November 03, 2009

Transcript of MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Page 1: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

LESSONS LEARNED ADOPTING AGILE FOR FIXED-PRICE CONTRACTS

By

Glenn R. Fuller

A Master of Science Research Practicum Submitted to the Faculty of

The Graduate School of Management, at the University of Dallas

in Partial Fulfillment of the Requirements for the Master of Science Degree

in Information Technology

November 03, 2009

Page 2: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

ii

DEDICATION

To my wife, my lifelong partner.

Page 3: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

iii

ACKNOWLEDGEMENTS

Page 4: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

iv

TABLE OF CONTENTS

Page

DEDICATION ……………………………………………………………………. ii ACKNOWLEDGEMENTS ……………………………………………………….iii

CHAPTER 1: INTRODUCTION..............................................................................1

Agile Software Development Method ...........................................................1 From Predictive to Agile................................................................................2 The Case For Agile ........................................................................................6

CHAPTER 2: LITERATURE REVIEW...................................................................8

Contractual Obligations .................................................................................8 Fixed-price Software Development Process..................................................9 Agile Methods................................................................................................11 Scrum .............................................................................................................13 Agile and Fixed-price Software Development ..............................................14

CHAPTER 3: ANALYSIS ........................................................................................17

The Project .....................................................................................................17 The Old Ways ................................................................................................17 The New Ways...............................................................................................19 New Process...................................................................................................20 The Tools .......................................................................................................23 The Results.....................................................................................................24 Lessons Learned.............................................................................................25

CHAPTER 4: CONCLUSION ..................................................................................28

Recommendations..........................................................................................29 Limitations .....................................................................................................31 Areas of Future Work ....................................................................................32

REFERENCES ..........................................................................................................34

Page 5: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

1

CHAPTER 1: INTRODUCTION

The holy grail of software development is to deliver the customer’s software on

schedule, within the agreed-upon price or cost, and meeting the original scope; this is

known as the Iron Triangle (Ambler, 2003). Software has evolved over the last 30 years,

from punch cards to reusable code, all in an effort to improve the software development

process. In the last 10 years, Agile software development processes were created and

now claim to reduce cost and cycle time (Turk & Vijayasarathy, 2008).

Agile Software Development Method

Agile was created by a group of software developers that felt the existing software

development methods were inadequate and failed to meet the needs of the customer. An

Agile Manifesto is credited as the starting point for the Agile movement. It states:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. (Beck, et al, 2001). This new approach eliminates the Big Up Front Design (BUFD) and Big Up Front

Requirements (BUFR) in favor of spending more time developing working software that

can be used by the customer to further refine their requirements. It requires flexibility on

the part of the customer and the development team as the requirements are in flux during

Page 6: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

2

the entire development cycle. With frequent deliverables, the customer has the

opportunity to evaluate the product and provide feedback and changes as it is being

developed.

Agile was developed originally within IT projects where the project teams have

flexibility with respect to requirements and deliverables, but due to its success,

commercial software developers have begun to adopt it also. This is a case study of a

team that utilized an Agile software development methodology to develop commercial

software under a fixed-price contract.

From Predictive to Agile

Many attempts have been made to reduce the level of uncertainty and risk in

projects, and this has resulted in the formalized processes called predictive project

management. The predictive approach is premised on using historical estimates for the

effort required to perform similar tasks, or using a bottom up estimate with the

assumption that smaller tasks can be more accurately estimated than larger ones. The

project schedule, theoretically, now becomes more certain, and time and costs can be

accurately estimated. This approach fits well with fixed-price software development, as

customers and developers believe they have reduced their risks by developing and having

the customer approve large amounts of documentation before proceeding to the next

phase.

The predictive model emphasizes the BUFD and BUFR approach, which is a

necessity for a fixed-price software development contract. It dictates that all

requirements are detailed and documented before any work is performed. Only when the

Page 7: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

3

detailed requirements documentation has been approved by the customer does the design

phase begin. The design is now created from the requirements and a design document is

generated and approved. Next, the design documentation is used to define and document

the code modules, which are then given to individual developers to create. Any change

to the product requirements at this point creates significant rework in all the

documentation prior to any change to the coded modules.

The most common predictive model, called the Waterfall lifecycle, is associated

with a process defined by Winston Royce of IBM (1970). This process has distinct

phases with strictly-defined hand-offs between each one. These hand-offs are formal

documents that have approval by the owner(s) of the previous phase.

The waterfall phases are: 1. System Requirements 2. Software Requirements 3. Preliminary Design 4. Analysis 5. Program Design 6. Coding 7. Testing 8. Operation Royce (1970) also included feedback loops from the current phase to the previous

phase. This allowed for changes to be made if an unforeseen problem occurred in the

current phase, to revisit the previous phase and make the necessary changes. Again, all

changes are documented and approved before handing off to the next phase.

The issues of the Waterfall lifecycle software development methodology are well

documented by Craig Larman (2003) when he succinctly states the root cause of the

Page 8: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

4

failures of the Big Up Front Design (BUFD) process for software development as;

A “Waterfall” lifecycle, big up-front specifications, estimates, and speculative plans applicable to predictable manufacturing have been misapplied to software projects, a domain of inventive, high-change, high-novelty work. (Larman, 2003).

The issues of the Waterfall and BUFD methodologies has led to the exploration of

different approaches to software development in the hopes of more successful results.

Starting in 2001, the Agile software development methodology (Beck, et al, 2001)

was created as a flexible development methodology that allows for changes in scope,

cost, and requirements during the development process, based on changing needs within

the customer’s business process during the software development cycle. Delivery cycles

are reduced to weeks instead of years as features are created and demonstrated to the

customer. This allows the software to evolve with the business requirements rather than

remain stagnant waiting for the software delivery. Since the process is designed to allow

changes, the large up-front documentation effort is reduced to only what is needed within

each delivery cycle.

The benefits of Agile have been well documented over the last eight years as

Agile software development methods have been adopted within IT organizations to

improve software productivity, reduce defects, and increase internal customer satisfaction

(Turk & Vijayasarathy, 2008). Due to the documented benefits of Agile for internal IT

projects, the Agile development methodology has been expanded to projects outside the

IT arena. The attempt to apply Agile methods to commercial software development

projects has been a recent phenomenon, and it is success in the commercial software

Page 9: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

5

environment has not been fully explored (Begal & Nagappan, 2006).

Though Agile has been attempted in the commercial software development

environment, the successful application of Agile methods to a fixed-price contract with a

predetermined scope and schedule has not been proven. In fact, many Agile proponents

believe that Agile methods are not compatible with fixed-price contracts.

Agile methods acknowledge the market and project uncertainties and try to adapt by building software in small and completely done increments, so that the business could change the project direction up to the point of efficiently killing the projects at pretty much any moment. Naturally this approach is not very compatible with the idea of fixing everything upfront. (Marchenko, 2008).

In the Agile Manifesto, the key aspects of the methodology are the flexibility to

“Respond to change over following a plan”, and “Customer collaboration over contract

negotiation” (Beck, et al, 2001). These aspects of Agile are in seeming conflict with the

process of negotiating and signing a contract with a commercial customer for delivery of

a specific set of software requirements for a fixed-price and within a contractual schedule

that typically has penalties for missing the dates or requirements in the contract

(Cockburn, 2005).

Are there benefits to implementing Agile Software Development Methods in a

fixed-price commercial software company, developing software to predefined and

contractually obligated schedule, scope, and cost? Many Agile practitioners believe that

Agile is not a good fit for a fixed-price software development contract due to it is

necessity for up-front estimates (Cockburn, 2005), however, there a some who believe it

may offer an improvement in the software development process over traditional

Page 10: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

6

approaches (Miller, 2007).

I don't really think the Agile answer to a fixed bid is any different from any other process. I do think that Agile practices and project management can give you far more control and feedback on the "Iron Triangle" of resources, time, and features. Agile/RUP/CMMI/waterfall whatever, the iron triangle constraints still apply. If you try to lock all three constraints you're in for either pain, unhappiness, or protective sandbagging in your estimates. I would still choose to use Agile delivery for fixed bid projects because I think that is the most efficient way to execute and allows for the ability to fail "softly" with some fraction of the features instead of total abject failure to deliver any features on time like a waterfall project. (Miller, 2007).

The Case For Agile

This case study is about a company that provides software under fixed-price

contracts to state and local government agencies. The requirements are provided by the

agency in the form of a Request For Proposal (RFP). The bidders submit a proposal that

includes scope, schedule, and price to the customer. The agency reviews the bids and

selects one vendor to enter into a fixed-price contract based on the winning proposal.

Once the contract is awarded, the software development process, based on the signed

contract is begun.

This company has been developing and delivering software for over twenty years

and has used the Waterfall development methodology to create the software. Over the

years, several issues continued to haunt the development process including missed

requirements, late delivery, numerous software bugs, unsatisfactory software

performance, and blown budgets resulting in a less than profitable business.

The organization was dysfunctional as marketing created the requirements with

Page 11: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

7

the customer, handing the documents off to engineering and then never saw the results

until the product came out of development. At that point, any changes were at a high cost

to both budget and schedule. Software was tossed over the wall from development to

system test and then tossed back from system test to development. Heated arguments

were common between developers and testers, as the definition of bugs became a

constant battle. Everyone was pointing fingers at everyone else and no one wanted to be

accountable for the final results.

Something had to change, as customers began to go to competitors. New

management was brought in to fix the problems. Both the organization and the

development process needed to be fixed. The impact of the Agile methodology, that has

proven successful in IT software development environments (Turk & Vijayasarathy,

2008), was too alluring to pass up. Agile was chosen as a methodology that not only

changes the way software is developed but puts ownership directly on the team creating

the software. This became the silver bullet the business was hoping would solve their

problems. The only question was, could Agile bring the same benefits from the

Information Technology (IT) world to succeed in a fixed-price contractual software

environment?

Page 12: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

8

CHAPTER 2: LITERATURE REVIEW

Although there is a wealth of information on the Internet and in bookstores

regarding the use of Agile software development methodologies on IT projects, few

studies have been performed on the use of Agile for commercial software development,

and even less is published on the application of Agile to fixed-price commercial software

contracts. Much of the published information applying an Agile methodology to fixed-

price contracts is speculation and not based on actual experience.

Contractual Obligations

There are inherent differences between software developed by internal IT

organizations and software intended to meet a predetermined contractual agreement

(Cockburn, 2005). The internal organization is not bound by a contractual obligation that

imposes penalties for not delivering on schedule or for missing the predefined

requirements (Marchenko, 2008). A contractual obligation can impose significant

penalties for delays, and the customer must approve the deliverable, which can mean

rework and added cost due to the interpretation of requirements.

An internal IT organization may miss deliverables or have significant delays but,

in most cases, it does not have a material impact on the organization’s customers or

business reputation. The biggest difference is the ability to work with the internal

customer to change requirements and/or provide incremental deliveries that may not meet

all requirements but to provide additional working functionality. Under a contractual

obligation, the customer expects to get delivery of the entire set of requirements and can

Page 13: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

9

impose penalties up to and including cancelling the contract if the commitment is not

fully met.

Fixed-price Software Development Process

Commercial software development has traditionally adopted a waterfall

development methodology that follows a serial sequence of events to deliver a

commercial software product. This methodology was adopted from the other engineering

disciplines, such as construction or hardware design (Northrop, 2004). The fixed-price

commercial software business is predicated on the need to have well-defined

requirements up front, along with a fixed-price for the effort and a schedule for delivery.

The Waterfall method and it is derivatives support the need to develop pricing, scope, and

schedule. The belief has been that by properly estimating the iron triangle, the project

will be a success.

However, there are significant issues with the Waterfall methodology, as stated by

Jim Highsmith and Dr. Sam Bayer (1994):

While waterfall methods brought some stability and order to the chaos of earlier development efforts, their shortcomings are becoming increasingly apparent and many software development organizations are lost in the "swamp" of waterfall development (Bayer & Highsmith, 1994). In a review of the Waterfall process, an experienced software development

manager believes,

The problem with the waterfall method is its inflexible partitioning of the project into these distinct stages. Delivered systems are sometimes unusable, as they do not meet the customer’s real requirements (Sommerville, 1995, 85). A study of over 250 software development projects, that did not include any Agile

Page 14: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

10

projects, demonstrates the failure rate of software projects:

Industry software delivery statistics are quite dismal. Fifty percent of commercial software products are delivered over schedule, 33 percent are cancelled, and 75 percent are operational failures. Government software delivery statistics are similar (Jensen, 2003).

The evidence for ongoing issues with software development projects is

overwhelming. In a large project study, The Standish Group showed that 45% of the

requirements were never used (Johnson 2002). So, even when the agreed upon

requirements have been identified up front, the customer may not really need or use the

features in the delivered product.

Figure 1 Software Project Success Rate (Hartman, 2006.)

In a summary of surveys done by The Standish Group (Hartman, 2006), the

number of successful projects has gradually improved but the success rate as of 2004 was

Page 15: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

11

still less than 30%. With all the issues and failures of software development projects,

including high visibility failures (Charette, 2005), it becomes obvious why companies are

looking for better methods to create software.

Agile Methods

In the years following the signing of the Agile Manifesto in 2001, several studies

have been done to compare Agile software development methods with the previous

Waterfall methods. In a study performed at Penn State University, two teams were

created, one using the Waterfall method and the other using the Agile methods Extreme

Programming (XP) and Rational Unified Process (RUP). According to the author:

Also, based on this assessment of the traditional Waterfall approach versus the agile methods RUP / XP discussed in this study, both approaches are used for particular reasons, the iterative methodologies are usually better for new concepts, while the Waterfall may be better for modifications to existing systems or building large scale systems (Waters, 2008). In a survey of early adopters of Agile (Turk & Vijayasarathy, 2008), using a 7-

point scale, it was found that the top benefit (Figure 2) was that the project better met the

customers’ needs, followed closely by improved quality and increased flexibility in

development. Faster delivery time and lower cost were also cited as significant benefits.

This survey’s respondents were primarily IT organizations, not commercial software

companies. Where flexibility is considered a benefit to an internal IT organization, it

may not be perceived as such by a company that is delivering software to a customer

using pre-defined requirements and fixed estimates.

Page 16: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

12

Figure 2 Extent of Benefits (Turk, Vijayasarathy, 2008)

In a case study of the Douglas County, Colorado sheriff’s office, the IT

organization could not get an IT project approved due to cost and schedule using the

traditional Waterfall approach. By changing their methodology to Agile, the IT team

delivered the project in half the time (Fredrick, 2007). The Agile Journal is full of Agile

case studies, which are predominately IT projects, in which the IT organization has

changed from a traditional Waterfall method to an Agile method. The results of

implementing Agile methods for the majority of projects have been very successful. In

his review of Craig Larman's "Agile and Iterative Development", Patrick Wilson-Welsh

noted (Wilson-Walsh, 2003):

Many large and small organizations worldwide are formally adopting agile and iterative practices. Informally, this experimentation has been going on for decades. Larman points out that iterative process design goes back more than 30 years to Tom Gilb's Evo method. Even the man whose original article gave waterfall its momentum, Winston Royce, apparently regretted later having written it. The late Royce, according to his son, was actually a misunderstood advocate of iterative and incremental development. The research seems to indicate that a shrinking percentage of projects are using waterfall "by the book" anymore.

Page 17: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

13

If you are an Agile skeptic, this book may rattle your conviction. If you are fence-sitter, it may convince you. And if you already have Agile fire in the belly, then certainly this book will stoke that fire. After reading it, I am left wondering how intelligent, experienced software development management can justify the continued use of a process that has wasted so much money and caused so much pain. And I am left with no doubt that agile methods constitute an unstoppable trend in software project management. There is, indeed, a revolution going on (Wilson-Walsh, 2003).

Mary Poppendieck believes that Agile is not just for small projects but that the methods

can also be adapted to large projects. She also believes that Agile can be adapted to work

in commercial software environments:

The bottom line is that the problems that used to be addressed by traditional software processes have changed, and those processes are no longer up to the task of addressing large project development. Meanwhile, the agile practices being honed in small projects are just the ones needed in a large project environment which deals with legacy systems and commercial components. So don’t be surprised to see agile practices move up-market, as disruptive technologies always do, and take over the large projects as well (Poppendieck, 2006).

Scrum

Scrum was developed as an Agile methodology for improving the productivity,

quality, and cycle time of IT software development projects. This methodology has been

very successful in IT organizations (Walton, 2004) and, therefore, it is thought to be a

candidate for implementing in a commercial software organization where software

companies expect to see the same benefits.

Scrum is a software development process that implements the Agile methodology

(Schwaber, 2006). It utilizes small teams of 6-10 people that work together on self-

assigned requirements called user stories at the beginning of each Scrum cycle. The user

stories for each Scrum cycle are managed by the product owner and must be completed

Page 18: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

14

within the pre-determined time frame of three to four weeks with demonstrable working

code. Scrum uses a requirements process that is flexible and allows for changes at the

end of each three- to four-week deliverable. This allows the customer or product owner

to make changes based on the working code demonstrations.

The Scrum teams become self managing, holding daily meetings, and holding

team members accountable for their progress each day. The Scrum Master facilitates the

meetings which are limited to asking three basic questions.

1. What did you accomplish yesterday?

2. What are you going to accomplish today?

3. What is blocking you?

Team members check in their working code as they complete each user story. Testing is

performed in parallel with development as test scripts are written prior to any code being

developed. The test scripts ensure the code will perform the functions as defined by the

user stories.

The freedom for the developers to manage their time and commitments imparts a

sense of ownership and responsibility. Developers cannot blame management for giving

them unrealistic expectations. They also cannot hide their lack of progress since they are

reporting to their peers. This aspect of Scrum is a powerful tool to improve productivity

if managed properly.

Agile and Fixed-price Software Development

A survey of Microsoft employees was performed in 2006 to determine the extent

of Agile adoption within the Microsoft organization (Begal & Nagappan, 2006). They

Page 19: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

15

found that the adoption of Agile practices within Microsoft had a 33% penetration. The

majority of those using Agile were using the Scrum form of Agile. This research was

done in the form of a survey of the developers that included both standardized questions

with a five point scale as well as two open-ended questions regarding the benefits and

issues of adopting Agile. Microsoft is a commercial software company that develops

products for sale to the public but does not necessarily fit the category of performing

fixed-price contractual software development, so though this study is very close to our

target, it does not quite hit the mark.

Artem Marchenko (2008), an Agile expert, has commented on the application of

Agile methods to fixed-price commercial contracts:

While it is possible to utilize fixed-price, fixed scope contracts with the agile methods, fixed contracts are still discouraged in preference of time & materials or Pay per Use (Soundair, 2008). The reason is that in most if not all cases software development is not a mass production, but a new product development (Marchenko, 2008).

This general concern regarding using Agile for fixed-price commercial contracts is

illustrated in the three-fold aspects of any project, scope, schedule, and cost. Martin

Fowler (2003), one of the original signatories’ of the Agile Manifesto commented:

Many people believe that you can't do a fixed-price contract in an agile project. Since the whole point of an agile process is that you cannot predict the future, this isn't an unreasonable supposition. However this doesn't mean you can't come up with a fixed-price agile contract, what it means is that you can't come up with a fixed-price and fixed scope contract (Fowler, 2003).

Page 20: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

16

The gains of the Agile method with respect to efficiency and cycle time reduction

were too great to ignore. The challenge for the team was to improve upon the software

development process under the conditions of a fixed-price contract. This case study will

examine the benefits and problems that occurred within the development team, and

lessons learned in the attempt to adopt Scrum to deliver commercial software on a fixed-

price contract.

Page 21: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

CHAPTER 3: ANALYSIS The Project

Due to the high level of software defects and ongoing customer issues, I was

asked to take on the challenge of fixing our software delivery problems. The

organization consisted of multiple departments and each department was responsible for

only a small portion of the effort on each project. For the project, I was put in charge of

all the departments that were involved in creating and delivering the software

applications to our customer. This included engineering, system test, support, and project

management.

I was then assigned a fixed-price software development project for a government

customer. The customer required significant modifications to our existing product such

that the architecture could no longer sustain more changes without extensive rework.

Also, the existing product was written in an obsolete programming language with a flat-

file database. The decision was made to use the new contract to update the product with

a modern relational database and use a modern programming language.

The customer had been informed of the updates to the product along with the

inclusion of their requirements. Many of the requirements were to be derived from the

existing product, based on the functionality being used by the customer. Rough estimates

were made based on using Agile as the development methodology and the teams previous

experience with the existing application. The team, optimistically, factored in

productivity improvements with the new languages and Agile. A one year schedule for

delivery, and pricing, was agreed upon, then the project team got started.

The Old Ways

17

Page 22: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

The original organization was divided into developers (engineering), system test,

trainers, product management, and support, with a manager over each department. No

one department owned the deliverable, so much time was spent by the departments

discussing who was responsible for missing customer commitments.

This lack of accountability was highlighted when the various managers were

asked about who was responsible for completing a project. All managers or their

representatives had to sign off on any changes and agree to the impact on schedule and

cost within their department. Adding up all the impacts meant that any request took

weeks to months even for the simplest change.

Developers were given their assignments and told when they had to be completed.

Collaboration among developers was rare since each person had their list of tasks to

complete and did not have time to help each other. The tasks were assigned by the

manager who had little technical understanding of the product. Only one or two senior

architects had the big picture and spent most of their time answering questions as

developers ran into problems or didn’t understand their assigned tasks. Since estimates

were done by the most senior developers, they were always optimistic, and even those

estimates were trimmed by management in order to fit the customer’s budget.

System testing was performed manually with written test scripts based on the

customer requirements document. System test wrote the test scripts as the software was

released from engineering. Software bugs continued to plague the product even though

the system test effort was more than twice the development schedule. Releases to system

test were passed back and forth to and from engineering several times with long lists of

bug fixes. Fixes were made only to discover new problems.

18

Page 23: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

When the product was finally released to the customer, testing, performed by the

customer, discovered requirements that were missed or not what the customer expected,

and of course, numerous software bugs. As a result the customer became involved in the

break fix cycle and rework of the product, which only added to the development cycle

time and cost.

The New Ways

With the new project came the authority to remake the dysfunctional organization.

The goal was to give ownership to the team responsible for creating the product.

Departments were eliminated and the people in them combined into a single organization.

Roles and individual responsibilities did not change, but each individual was part of the

team that had to create the product. The goal was no longer just counting defects, or

creating requirements, or coding a set of modules but, it was getting the working product

delivered and meeting the customer requirements. Thirty people now reported to a single

manager with individual team leads for each of the smaller teams.

Team members were collocated with each other in larger common areas. Cube

walls were taken down and desks were arranged to facilitate quick and easy access to

teammates. These changes were actually requested by the team members in order to

improve communication and reduce time spent looking for each other to get questions

answered or to discuss problems.

Estimates were made and approved by the team which consisted of everyone

necessary to create and deliver the final product. Team members were held accountable

to each other, their team lead, and the manager whom they shared. The team had their

fate in their own hands. Finally, everyone on the team owned the responsibility to make

the project successful. This responsibility fostered collaboration and communication

19

Page 24: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

among team members, as now the need to help others was not viewed as a hindrance to

getting one’s own work done but as a necessity to ensure the success of the project.

New Process

Software development outside the IT world has greater visibility and more rigid

requirements that are bound by a legal contract. This means the processes for IT

development need to be reviewed, based on the contractual environment, and may not be

a good fit for commercial software.

Unfortunately, and with a few exceptions, internal IT development is generally the focus for both research and practice in agile methodologies. Commercial software companies are on their own in figuring out the best way to take advantage of recent agile practices. Commercial software can definitely benefit from agile development, but it will take a lot more effort and understanding to balance the constraints of commercial software with the tenants of agile development (Maxwell, 2007).

The software development team reviewed different forms of Agile, such as

Extreme Programming (XP) (Wells, 1999) and Scrum (Schwaber, 1996). The project

team was searching for a methodology to fix the organizational issues and to improve the

productivity and morale of the team. Of specific interest to the team was the availability

of any published research regarding the application of Agile software development

methods within a commercial software organization, and specifically within companies

that are doing fixed-price software development.

As the team reviewed the choices, Scrum was selected as the new development

methodology due to its inclusive team roles, daily meetings, self managing teams, and

product ownership processes. The development team adopted the Agile methodology

called Scrum in the desire to gain the benefits of reduced cycle time and cost reduction

that are claimed by the adopters of the Scrum methodology. The following description is

20

Page 25: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

from Scrum For Team System, developed by Ken Schwaber and Colin Bird (2009):

At the beginning of a Sprint, we start with the Product Owner ensuring that we have a prioritised and up-to-date Product Backlog. This allows the team to select an achievable set of the highest priority backlog and work out how to build it in the Sprint Planning meeting. The team then proceeds to work through the Sprint Backlog tasks on a daily basis, synchronising their activity in a daily Scrum Meeting. At the end of the Sprint the team have built a Product Increment which they demonstrate to the Product Owner, business users and other interested stakeholders in the Sprint Review. Feedback from the Sprint Review will be added to the Product Backlog and prioritised by the Product Owner. Before starting the next Sprint the team discuss their performance and make improvements in the way they work in the Sprint Retrospective.

The team will continue Sprinting until they have developed a sufficiently useful system that can be put into production in a Release Sprint (Schwaber & Bird, 2009). © Copyright EMC Consulting 2009

The Scrum process changed the organization from a directed, top down style of

management to a self managed team in which the team leaders, or Scrum Masters, act more

like coaches than managers. For example, the project manager, i.e., the Scrum Master, no

longer needs to organize the team, but the team organizes itself and makes decisions on what

to do. Kane Mar & Ken Schwaber (2002) illustrate,

Most management is used to directing the project, telling the team what to do and then ensuring they do it. Scrum relies on self-organization, with the team deciding what to do while management runs interference and removes roadblocks (Mar & Schwaber, 2002, 3).

21

Page 26: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

The following picture is a representation of the cyclical nature of the Scrum process

(Schwaber & Bird, 2009).

Figure 3 The Scrum Process © Copyright EMC Consulting 2009 (Schwaber & Bird, 2009)

The team members chose to divide into three Scrum teams for the project. That

meant there had to be four meetings each day, a Scrum meeting for each team then a Scrum

of Scrums meeting between the Scrum Masters. This approach kept the teams in synch with

each other while keeping them small enough to encourage interaction and collaboration.

The product owner, formerly the product marketing manager, took the previously

estimated and approved requirements and used them to create the product backlog. The

customer was involved in reviewing the requirements to ensure compliance with the contract.

The teams reviewed the product backlog at the beginning of each four-week Sprint.

Each team member chose the features they would work on for the Sprint, with the goal of

22

Page 27: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

having completed working code to demonstrate at the end. During the Sprint, the team

members were required to self report on their daily progress and, when a working feature was

checked into the build, the team member would show completion and move on to the next

feature. This process of taking features off the backlog during the Sprint is called the Sprint

Burndown.

At the scheduled completion of the Sprint, a team meeting was held to have each

team member demonstrate the code they had developed and built into the product. The

meeting gave the team members a sense of accomplishment and the team morale improved

significantly over the previous Waterfall method.

At certain intervals, when a set of features were completed, a customer demonstration

was arranged to show the customer the progress being made and to ensure the requirements

had been implemented correctly.

The Tools

The original code was developed using several coding environments, many of

which are now obsolete. Some developers only knew one or two coding languages, so

could not contribute in areas where another language was used. In the new order, coding

was done using two languages, C++ and C#, Microsoft Visual Studio and using

Microsoft Team Server and Conchango’s Scrum for Team System Template to manage

and automate the build process. All developers were trained in the new languages and

use the Visual Studio environment. When they checked in their completed code, it was

included in the daily automated build.

HP Mercury test tools were purchased to automate the test process. Test scripts

were built by developers in collaboration with testers from the feature requirements. The

product owners reviewed the test scripts to ensure they complied with the customer

23

Page 28: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

requirements. All this was done prior to any code being developed. The test scripts

described how the end result should work so there was little ambiguity. This enabled the

developers to test their code and compare it to the expected results during development as

opposed to waiting several weeks for a system test. This immediate feedback reduced

cycle time, increased productivity, and improved the quality of the code.

A full regression test was performed at the end of each Sprint. This was only

possible due to the automated test tools. It highlighted areas where new code broke old

code, and developers had to fix any bugs prior to the end of the Sprint. Any code that did

not pass the final regression testing was considered incomplete and the team could not

take it off the backlog. It was rescheduled for the next Sprint.

The Results

The project team adopted Scrum because of the desire to find a better way to

develop software over the previous Waterfall method. Some team members did not buy

into Scrum and were skeptical but, as time went on, most accepted it. The main reason

given was that it gave each individual the ability to have some say in what they did each

day. As a result, the morale of the organization improved. For example, one developer

who had been cycling over the weekend broke both his wrists in a fall and ended up bed

ridden in the hospital. He asked his wife to bring his laptop to the hospital so he could

continue to work. Peer pressure was a tremendous asset in getting marginal contributors

to improve their productivity. The team owned the results and most wanted to succeed.

But, the end goal is to deliver on time, on budget, and meet the requirements.

This did not occur. The team, in the rush to adopt Scrum, did not follow through in

forecasting the progress towards the schedule. In fact, only a rough schedule existed

since it was not clear how many backlog items would be completed for the Sprints. As

24

Page 29: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

time went on, the teams found increasingly detailed requirements that needed to be added

to the backlog. The product backlog Burn Down did not support the schedule, but the

developers optimistically assumed they could still make the delivery date. Everyone

wanted to succeed and evidence that this was not going to happen was ignored.

Management was given the assurance that the delivery date would be met up until

the last Sprint. Only then did the team acknowledge that they could not deliver on time.

The customer was notified and was not pleased with the delay but gave the team more

time to complete their work. After several more months of work, it was determined to

negotiate an end to the contract as the team was not close to completing the project.

Lessons Learned

The project was a failure in the business sense, and one could suggest that the

project would have been better off staying with the previous Waterfall methodology.

That was not the case here. The amount of working code developed by the team in less

than a year exceeded the efforts of another project team that had invested several years in

a similar product with hundreds of developers. The productivity exceeded anyone’s

expectations. So what went wrong? The original estimates were inaccurate because they

did not have enough detail regarding each requirement. A feature that was estimated to

be a few days took weeks due to the additional requirements discovered when it was

detailed in the user stories.

Once it was discovered that the requirements were not in sufficient detail, effort

was made to go through the entire backlog and add missing requirements. Upon

completion of this task, the backlog almost doubled. This would have been the time to

escalate the discrepancy but the project team believed they could improve their

productivity and take on the additional requirements in the remaining four-week Sprints.

25

Page 30: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Also, the team failed to notify the stakeholders about the potential impact of the

additional requirements but continued to optimistically believe the project would be

completed on time.

The Product and Sprint backlogs should have been managed as a measure of the

Sprint teams’ productivity and used to assess the progress toward the goal. One problem

with this approach is that in a typical Scrum project, the requirements are viewed as

flexible and can be pushed out, delayed, or renegotiated with the customer so as to keep

the team on schedule and deliver a working product. In a fixed-price contract, that

flexibility is not available, as the customer has given all their requirements and the

business has agreed to deliver them within a fixed schedule. When delays occur, the

customer can charge penalties and/or cancel the project. This eliminates the flexibility

that Scrum expects, as the end goal is to deliver something that is viewed as good

enough, not necessarily everything the customer wants.

The adoption of Scrum also meant the end of project management which was

viewed as a product of the Waterfall methodology. That was a mistake. No matter what

methodology is adopted, there is a need for someone who is monitoring the progress of

the teams against the schedule, identifying issues that need to be resolved, and reporting

the progress to the stakeholders. The lack of project management allowed the team to

ignore the obvious, that the remaining product backlog could never be completed on

time.

The lack of proper requirements up front, and the resulting impact on the project,

shows the need to continue the BUFR process on fixed-price contracts. As discussed in

Chapter 1, proper estimates cannot be made without detailed requirements. Bad

26

Page 31: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

estimates cause the business to suffer financial loss and the customer suffers the loss of a

product they need to run their business. Everyone loses. It would be great if all

customers partnered with the business to create software that is good enough and allow

the flexibility to create that on a pay-as-you-go basis but government entities do not have

the political will to take the risks associated with cost-plus or Time-and-Materials (T&M)

contracts. In Scrum, the up-front requirements are only what is needed to get the project

started and provide some overall guidance; they are detailed at the beginning of each

Sprint. This process does not work in a fixed-price contract environment.

Adopting Scrum, overall, was a benefit to the organization as it instilled

discipline, daily Scrum meetings, peer pressure, individual roles rather than

organizations, and individual accountability, that are not seen in a typical software

development environment. The real benefit to the business, was allowing the developers

to manage themselves rather than have someone tell them what to do. This had the

greatest impact on the organization. Combining the different groups, testers, product

managers, and developers into a single team removed the walls, both figuratively and

literally, between team members.

27

Page 32: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

CHAPTER 4: CONCLUSION

This study reviewed the transition of a project team from utilizing a Waterfall

methodology to Scrum and the results from applying this method to a software

development contract with a fixed scope, fixed schedule, and fixed-price. The thirty

member project team, divided into three Scrum teams, implemented new tools and

processes in an attempt to improve the predictability and productivity of the software

development process. The productivity results were realized, but not predictability. The

project failed to complete within the scheduled time frame and the customer cancelled the

contract.

Agile and Scrum failed to make the project a success. So does that mean Agile

will not work for fixed-price contracts? Was it the methodology that failed? Would

Waterfall have saved the day? In the context of this case study, the implementation of

Agile did not fully replace the processes that are required for a fixed-price contract. The

biggest failure was the lack of project management to oversee the progress of the team

and identify the lack of timely progress. The project team chose to follow classical Agile

methods, which do not utilize project management, but depend on the team to keep track

of their progress. Scrum also does not fully develop detailed requirements until the

beginning of each Sprint. Thus, the big picture is obscured and classical project

management approaches do not work. The task to create each requirement is not defined,

so the team cannot be held to a pre-determined estimate that can be measured. Progress

is measured on the implementation of the features for each Sprint. The remaining

features are not developed, so the amount of time remaining to complete the project

remains somewhat of a mystery.

28

Page 33: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Agile does not recognize the concept of a fixed schedule with a fixed scope. It

relies on the flexibility of schedule and requirements to deliver product that works in

short iterations. This approach eliminates the BUFR and BUFD processes and replaces

them with a cursory requirements process that gets more detailed at the beginning of each

Sprint. Without detailed requirements up front, the estimates, required for a fixed-price

contract, are not correct, and there is little chance of the project being completed within

the pre-determined schedule. Scrum does not require, in fact, discourages, the level of

detail that would be necessary to develop a complete project schedule. But, without this

detail, it is impossible to ascertain the status of the project other than to compare the

remaining requirements list against what has been completed. What is not known, is the

effort remaining to complete the requirements. The lack of detail eliminates any level of

predictability that is a necessity for fixed-price contracts.

Recommendations

In the Waterfall days, the project manager was managing a matrix organization,

had a fixed schedule estimated from a well defined set of requirements, and tracked

progress against the schedule to get the product completed. Under Scrum, the project

management was dropped by the project team since the team now owned the schedule,

and this contributed to the failure of the project. The project team did not have a pre-

determined schedule, but only a list of requirements that the team members selected from

at the beginning of each Sprint. The inability to track progress against the fixed delivery

schedule caused the stakeholders to doubt the team’s ability to complete the project, so

when the delivery date was missed, the team’s credibility was damaged and the contract

cancelled. It is recommended that project management be continued in any Agile

implementation. There is a benefit to having an objective observer monitoring and

29

Page 34: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

tracking the team’s progress and providing feedback to the stakeholders, giving them

confidence in the team’s ability to deliver on time.

Big Up Front Requirements and, to a smaller degree, design is a necessity for

performing any fixed-price contract. The degree of effort up front determines the

accuracy of the estimates and pricing. Typical Agile projects minimize this effort since

the method encourages customer feedback and redefining requirements during the

process. This approach does not work under the conditions of a fixed-price contract and,

therefore, BUFR is required. A scope and schedule are required, and the process chosen

needs to support this effort. The only way to tell if a project is on schedule is to measure

progress against original estimates.

Agile provided an improvement in developer productivity along with the ability to

review completed code in very short increments. This quick feedback to both developers

and customers provides ongoing assurance that work is being completed and also shows

how requirements are being implemented. Any discrepancies in the requirements can be

resolved and the changes implemented within another four-week Sprint as opposed to

waiting until the final product is finished.

The collaboration, peer pressure, and daily Scrum meetings add a high level of

visibility to the developers and holds them accountable to meeting their commitments.

The handoffs between development and test are eliminated, as are the issues with regard

to the testing process since both testers and developers are on the same team and have the

same goals. Developers are engaged in creating the test scripts with the testers before

developing code so there is little ambiguity as to how the final result should perform.

30

Page 35: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Limitations

This study was limited to one project and one business unit within a larger

company. The team had practiced Waterfall development up until the adoption of Scrum

and thus had no exposure to Agile until this project. A broader exposure to other Agile

methods may have caused the team to choose a different methodology than Scrum.

Time was another limitation, as the study was completed in 12 weeks. Had more

time been available, further analysis of the team and lessons learned could have been

performed.

The Agile development method chosen by the team was Scrum, which is only one

Agile method. There are several Agile methods, such as Extreme Programming, Lean

Software Development, Feature Driven Development (FDD), and Adaptive Software

Development (ASD), that were not evaluated and may be better alternatives for fixed-

price contracts.

The scope of the study is Agile development within the context of a fixed-price

contract and this is a limitation of this study. Other contractual alternatives to fixed-price

contracts were not examined.

The scope of the case study limited it to a single project which was the first

implementation following the adoption of Scrum. If the team had more experience with

Scrum on more projects, the outcome may have been quite different. Agile itself is agile

and thus capable of adapting to new methods of implementation. A team with more

experience may have foreseen the pitfalls and created methods that would be a better fit

for a fixed-price contract.

31

Page 36: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Areas of Future Work

The business unit continues to utilize Scrum to some degree on other projects. A

follow-up study should be performed to determine what aspects of Scrum continue to

provide a benefit to the software development process.

Fixed-price contracts limited this study and had a negative effect on the project.

An area of future study would be to evaluate other contractual methods that would be

more consistent with Agile development methods. Some alternatives would be Time and

Materials (T&M), Cost Plus, or other concepts that allow changes to the requirements,

without contractual obligations, during the development process.

This study only evaluated one Agile development methodology and there are

other Agile methodologies that should be studied that may be a better fit for a fixed-price

contract. Scrum may not be the best methodology for fixed-price contracts, the use of

other Agile or semi-agile methods should be examined to determine if there is a better

methodology for these projects.

Other software development methods, besides Agile, should also be studied.

Software development continues to play an ever increasing role in the development of

products and the creation of new business processes. Finding a methodology that will

facilitate greater software development productivity and predictability would provide

huge benefits to the software industry.

A recurring theme in the fixed-price contract environment is the issue of trust. In

other projects, such as construction, the customer can see the results and the progress

toward the goal. Agile provides this same degree of visibility for software projects by

giving the customer demonstrations of working code at the end of each four-week Sprint.

This builds trust between the customer and the project team as they see the product being

32

Page 37: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

developed. So, how do you get the typical fixed-price customer to trust that the company

will provide the best solution in the shortest amount of time without a fixed-price

contract? This is an area of further study that could revolutionize the software industry as

customers become more sophisticated and partner with the development business to

create products they need.

Commercial software companies need to develop better contracting vehicles that

create a trust relationship with the customer. Over time, as companies prove their ability

to deliver better products at less cost using Agile software development, customers will

move toward a more flexible method of developing requirements. This will benefit all

parties by allowing developers to efficiently manage their own processes while delivering

high quality products to their customers.

33

Page 38: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

REFERENCES Ambler, S. (2003). Something’s Gotta Give. Retrieved from http://www.ddj.com/architect/184414962?cid=Ambysoft Beck, K; et al, (2001). Manifesto for Agile Software Development. Retrieved from http://www.agilemanifesto.org/ Begal, A., & Nagappan, N. (2006). Usage and Perceptions of Agile Software Development in an Industrial Context: An Exploratory Study. Retrieved from http://research.microsoft.com/hip/papers/AgileDevAtMS-ESEM07.pdf Bayer, S. & Highsmith J. (1994). RADical Software Development® 1994. American Programmer Magazine, June. 20-26. Retrieved from http://www.jimhighsmith.com/articles/RAD_article.htm Cauvin, R. (2005). BUFR. Retrieved from http://cauvin.blogspot.com/2005/09/bufr.html Charette, R. (2005). Why Software Fails. IEE Spectrum, (September). 1-2. Retrieved from http://www.spectrum.ieee.org/computing/software/why-software-fails/1 Cockburn, A. (2005). Fixed-Price contracts in an agile organization. Retrieved from http://codebetter.com/blogs/darrell.norton/archive/2005/04/22/62326.aspx

Fredrick, C. (2007). How Douglas County, CO Cut A Project Timeline In Half. Retrieved from http://www.agilejournal.com/content/view/276/111/

Fowler, M. (2003). Fixed-price. (July). Retrieved from http://www.martinfowler.com/bliki/FixedPrice.html

Hanly, S. (2006). Zen and the art of software development. Retrieved from http://www.bcs.org/server.php?show=ConWebDoc.6631

Hartman, D. (2006). Interview: Jim Johnson of the Standish Group. Retrieved from http://www.infoq.com/articles/Interview-Johnson-Standish-CHAOS

Jensen, R. (2003). Lessons Learned From Another Failed Software Contract, Crosstalk Journal, (Sep.) Retrieved from http://www.stsc.hill.af.mil/crosstalk/2003/09/0309jensen.html

Johnson, J. (2002). Keynote speech at XP 2002, Sardinia, Italy. Retrieved from http://ciclamino.dibe.unige.it/xp2002/ Larman, C. (2003). Agile and Iterative Development: A Manager's Guide, Addison-Wesley Professional, 1st Edition. 27.

34

Page 39: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

Mar, K. & Schwaber, K. (2002). Scrum With XP. Retrieved from http://faculty.salisbury.edu/~xswang/Research/Papers/SERelated/scrum/ScrumXP.pdf. 3. Marchenko, A. (2008). Why no fixed contracts in Agile. Retrieved from http://agilesoftwaredevelopment.com/blog/artem/why-no-fixed-contracts-agile Maxwell. (2007) Agile Development For Commercial Software. Retrieved from http://www.exotribe.com/maxx/2007/03/02/agile-development-for-commercial-software/ Miller, J. (2007). The Shade Tree Developer. Retrieved from http://codebetter.com/blogs/jeremy.miller/archive/2007/08/31/trying-to-answer-hard-questions-about-agile-development.aspx Northrop, R. (2004, March). The Fall of Waterfall. Retrieved from http://intelligent-enterprise.informationweek.com/info_centers/imperatives/6/showArticle.jhtml?articleID=17701624 Poppendieck, M. (2006). Is Agile Software Development Sustainable? Retrieved from http://www.poppendieck.com/sustainable.htm Rashid, D. (2006). Agile and Design for Lean Six Sigma. Retrieved from http://www.airacad.com/default.aspx?Page=PaperAgileDFLSSSoft Royce, W. (1970). Managing the Development of Large Software Systems, Proceedings of IEEE WESCON, August. 1-9. Schwaber, K. (1996). SCRUM Development Process. Retrieved from http://jeffsutherland.com/oopsla/schwapub.pdf Schwaber, K. (2004). Agile Project Management With Scrum. Microsoft Press, 1st Edition. 1-32. Schwaber, K. (2006). Scrum For Team System. Retrieved from http://www.scrumforteamsystem.com/ProcessGuidance/ProcessGuidance.html Schwaber, K. & Bird, C. (2009). Scrum For Team System. Retrieved from http://www.scrumforteamsystem.com/processguidance/v2/Process/Process.aspx Sommerville, Ian. (1995). Software Engineering. (5th Ed), Addison-Wesley. 85. Soundar, V. (2008). Choice of Agile Product Development Model. Retrieved from http://agilesoftwaredevelopment.com/node/441

35

Page 40: MS_Practicum_Research_Paper_Glenn_Fuller_IP_110309

36

Turk, D., & Vijayasarathy, L. (2008). AGILE SOFTWARE DEVELOPMENT: A SURVEY OF EARLY ADOPTERS. Journal of Information Technology Management, ISSN #10421319 A Publication of the Association of Management. 1-9. Waters, S. (2008). Traditional Software Development versus Agile Development. Retrieved from http://www.personal.psu.edu/saw283/ Walton, B. (2004). Iterative vs. waterfall software development: Why don't companies get it?. Retrieved from http://computerworld.com/printthis/2004/0,4814,90325,00.html Wells, D. (1999). Extreme Programming: A gentle introduction. Retrieved from http://www.extremeprogramming.org/ Wilson-Welsh, P. (2003). The Waterfall Approach: a Critique. Retrieved from http://adaptionsoft.com/adios_waterfall.html Zwicker M. (2007). War Stories - Fighter Jets and Agile Development at Lockheed Martin. Retrieved from http://www.agilejournal.com/content/view/313/111/