Maturing Requirements Engineering Process Maturity Models
Pete Sawyer
Computing Department, Lancaster University, Lancaster, UK, LA1 4YR
tel: +44 1524 593780
fax: +44 1524 593608
KEYWORDS
Socio-Technical Systems, Requirements Analysis, Requirements Definition, Requirements Specification, Process Improvement
Maturing Requirements Engineering Process Maturity Models
ABSTRACT
The interest in Software Process Improvement (SPI) in the early 1990s stimulated
tentative work on parallel models for Requirements Engineering (RE) process
improvement in the late 1990s. This chapter examines the role of SPI and the
implications of the exclusion of explicit support for RE in the most widely-used SPI
models. The chapter describes the principal characteristics of three RE-specific
improvement models that are in the public domain: the Requirements Engineering
Good Practice Guide (REGPG), The Requirements Engineering Process Maturity
Model (REPM) and the University of Hertfordshire model. The chapter examines the
utility of these models and concludes by considering the lessons learned from
industrial pilot studies.
INTRODUCTION
The risks posed to software development projects by weak requirements engineering
(RE) practice have become widely recognised during the last decade. This has
spawned a great deal of investment in RE methods, tools and training by practitioner
organisations, and in RE research by the wider software and systems engineering
communities.
The focusing of attention on RE during the early 1990s coincided with the
deployment of software process improvement (SPI) that was stimulated by
Humphrey's pioneering work in the 1980s (Humphrey, 1989). However, a European
survey of organisations engaged in SPI programmes during this period (Ibanez, 1996)
confirmed that the SPI models then available offered no panacea for RE problems.
Indeed, the organisations consulted identified requirements specification and the
management of customer requirements as the principal problem areas in software
development that they faced. Even enthusiastic adopters of SPI programmes found
that while SPI brought them significant benefits, their problems with handling
requirements remained stubbornly hard to solve.
The Software Engineering Institute's Capability Maturity Model for Software (SW-
CMM) (Paulk, 1993), which was becoming widely deployed at this time, touched on
RE practices but provided little specific guidance. To redress this, the Requirements
Engineering Adaptation and IMprovement for Safety and dependability (REAIMS)
project conducted the first systematic application of the principles of SPI specifically
for RE. This resulted in the publication of the Requirements Engineering Good
Practice Guide (REGPG) (Sommerville, 1997; Sawyer, 1999) in 1997.
This chapter reviews the state-of-the-art of process improvement for RE. It starts by
reviewing the background to process improvement in the software and systems
engineering industry. It then considers the nature of RE processes and the pressures
and trends that have merged in recent years. It argues that for socio-technical systems,
RE practice needs to be particularly strong. It then reviews three RE process
improvement methods and examines the extent to which they have been validated.
The chapter concludes by summarising the options open to organisations seeking to
systematically improve their RE processes.
SPI MODELS AND STANDARDS
Humphrey's pioneering work on Software Process Improvement (SPI) in the 1980s
(Humphrey, 1989) led to the development of the Capability Maturity Model for
Software (SW-CMM), developed at the Software Engineering Institute under the
sponsorship of the US Department of Defence. Humphrey’s work reflected a
realization that the piecemeal adoption of better methods and tools would not deliver
the improvements in software quality increasingly demanded by customers. Rather,
the whole development life-cycle needed to be addressed in order to identify weak
areas and focus improvement efforts. From the customer’s perspective, SPI allows
software contractors to be assessed against a common model and provides a stimulus
for contractors to increase product quality and meet cost and delivery targets. From
the contractor’s perspective, SPI represents a strategic organizational tool for
containing costs and increasing market-share.
The SW-CMM does this by defining a generic model of the software development
process structured around 5 maturity levels. Process maturity represents the degree to
which a process is defined, managed, measured, controlled and effective (Paulk,
1993). The more mature a process, the more it is possible to accurately forecast and
meet targets for cost, time of delivery and product quality. As a process becomes
more mature, the range within which results fall is narrowed. The emphasis moves
from understanding the software process, through exerting control over the process, to
achieving on-going improvement.
The SW-CMM maturity levels range from level 1 (Initial) which is an ad hoc, risky,
process to level 5 (Optimizing) where a process is robust, adaptive and subject to
systematic tuning using experiential data (Fig. 1). Each maturity level has a focus that
is defined by a number of key process areas (KPAs). The KPAs effectively set
capability goals that should be met if the supporting key practices are adopted and
standardized. The set of key practices prescribed for each KPA define how the KPA
can be satisfied and, since SPI is concerned with organisational maturity,
institutionalised.
Leve l 1Initial
Leve l 2Repeatable
Level 4Managed
Level 5Optimizing
Leve l 3Defined
Figure 1. The 5-level SW-CMM process maturity model
SW-CMM level 2 (Repeatable) is interesting because its focus is on project
management and is concerned with gaining control of the process. It is also the one
level to explicitly address requirements engineering. It does this by defining
Requirements Management as a level 2 KPA (the others are Software project
planning, Software project tracking and oversight, Software subcontract management,
Software quality assurance, and Software configuration management).
Since achieving level 2 is the initial target of almost all SW-CMM-led improvement
programmes, the SW-CMM has had the effect of greatly raising awareness of
requirements management. It has stimulated the market for support tools and has
made it more widely (though still far from universally) practiced. In this respect,
therefore, it has been enormously beneficial. However, implementing requirements
management is hard (Fowler, 1998), in part because it exposes weaknesses elsewhere
in the requirements process. For example, one of the practices mandated for the
requirements management KPA is the allocation of requirements to software
subsystems. Arriving at the stage where a set of requirements are ready for allocation
(by successfully eliciting, validating, negotiating, and prioritizing them) is itself a
complex and error-prone process for which the SW-CMM provides no explicit help.
The omission of detailed advice for the systematic improvement of requirements
processes is true of all SW-CMMs (including the SW-CMMI (SEI, 2002)), of the
draft ISO/IEC 15504 (Konrad, 1995; Garcia, 1997) international standard for software
process assessment and of the ISO9001-3 quality standard (Paulk, 1994). However,
help for requirements processes is not entirely lacking. There are software and system
engineering standards such as ESA PSS-05 (Mazza, 1994) which provide valuable
and explicit guidance on requirements practice. However, these do not address
systematic process improvement. They do not provide a method for assessing the
weak points in an RE process or map out a route for the incremental adoption of their
recommended practices. Few organizations can afford to make revolutionary changes
to their requirements processes so they need help in planning and controlling
evolutionary improvement so as to minimize the risk that inevitably accompanies
change. This is the great strength of SPI.
THE RE PROCESS
Perhaps the most orthodox model of the RE process is that represented by the
following three IEEE standards (it is interesting to note that RE process is defined
implicitly in terms of documents that are products of the process):
• IEEE std 1362-1998 Guide for Information Technology - System Definition -
Concept of Operations (ConOps) Document (IEEE, 1998a)
• IEEE std 1233-1998 Guide for Developing System Requirements Specifications
(IEEE, 1998b)
• IEEE std 830-1998 Recommended Practice for Software Requirements
Specifications (IEEE, 1998c)
This process begins with scoping a problem and, in very broad terms, the solution (the
concept of operations or ConOps for short). This is followed by a process in which
requirements are elicited from customers, validated by developers and other technical
specialists, and constrained by factors associated with the system's environment. The
product of this is a System Requirements Specification document that defines the
requirements for the overall system. Following this, the system requirements are
allocated to components that will be configured in a system architecture that satisfies
the system requirements. As part of this, subsets of system requirements are allocated
to software components. For each software component, a further round of analysis is
performed in which a set of software requirements are derived from the allocated
system requirements. These are documented in a Software Requirements
Specification (SRS). This forms the definitive set of requirements that the component
must satisfy and is sufficiently detailed to allow development to commence.
This process has evolved to deal with large custom systems, comprising both
hardware and software, that are developed for single customers. These are typically
developed using a supply chain of sub-contractors responsible for delivering system
components and who are managed in a hierarchical relationship with, at its root, a
main contractor. Such projects still represent a substantial proportion of the economic
activity in the software industry, particularly since heterogeneous systems have
become increasingly software-intensive. Despite this, their relative significance has
declined during recent decades. This is not so much due to a decline in this sector of
the industry as increases elsewhere. These include the booming market for retail
software products and the development of the Internet as a medium for business and
entertainment.
As the industry has changed shape, the value of orthodox RE has been increasingly
called into question. Clearly, an RE process optimised for large heterogeneous
systems is unlikely to be easily applied to, for example, small short-duration projects.
Agile development methodologies, of which eXtreme Programming (XP) (Beck,
1999) is perhaps the best known, can be seen as a reaction against orthodox software
engineering and, by implication, orthodox RE.
Agile methodologies deviate from orthodox RE by eschewing detailed analysis,
modelling and documentation where a project might move straight from a concept of
operations-like activity to development. The promise of agile methodologies to be
lightweight and reactive is a seductive one. Although they are not entirely
incompatible with SPI (Paulk, 2001), they take what is essentially a programming
perspective on system development but for many domains, this is inappropriate. For
example, almost all large businesses are dependent on their computer systems, yet
programmers are seldom equipped to find business solutions. Many systems are
embedded within organisational contexts to support and manage business processes
that are enacted by people and software. These sociotechnical systems are typically
too critical to risk getting wrong, too sensitive to how the actors interact (which may
be dependent on, for example, tacit understanding between the actors) to be easily
understood, and too complex to make refactoring a viable option.
Sociotechnical systems need careful appraisal of change options. They need careful
analysis of the problem context and elicitation of user requirements. They need an
overall system specification that documents the validated customer and user
requirements and defines the new system in terms of its function, its quality attributes
and its context within its environment. They need responsibility for the satisfaction of
the system requirements to be carefully allocated to appropriate software and human
'components' (actors). And they need these components to be specified so that
development work, which increasingly takes the form of selecting and integrating off-
the-shelf components, can proceed. In these terms, most of the activities and products
of orthodox RE processes still have a vital role to play in ensuring that systems are
developed that meet their users' real needs.
However, it would be naive to think that simply applying IEEE stds 1362, 1233 and
830 would guarantee success. There are many practices, techniques, methods and
tools that may be deployed to aid RE. Although they embody much accumulated
wisdom, RE standards can only set out the general principles or give detailed
guidance for particular activities or documents. They offer no aid for selecting
appropriate methods (for example) or for designing an RE process optimised for a
particular organisation.
Recognition of this problem was the motivation for the RE process improvement
work described in the next section.
PROCESS IMPROVEMENT FOR RE
Selection of the RE process improvement models for description below has been
based on the following criteria: they should include advice on RE practice, they
should present this advice within the framework of a maturity model, and they should
provide a method for assessing existing processes. This set of criteria omits other
valuable and pragmatic work (see below) that can be used for RE process
improvement. However, the criteria have the effect of isolating work designed
specifically to help organisations integrate RE process improvement within a general
SPI programme.
There are currently three RE process improvement models that meet the criteria; the
Requirements Engineering Good Practice Guide (REGPG), the University of
Hertfordshire model and the Requirements Engineering Process Maturity Model
(REPM). Others sources of improvement advice exist. In particular Young (2001) and
Wiegers (1999) are excellent textbooks based around sets of recommended RE
practices and include practical advice on how organisations can use them to improve
their RE processes. These are not reviewed further here simply because they do not
include a process maturity model or assessment method. However, they contain much
wisdom and practical advice for organisations who do not need or wish to undertake a
formal, calibrated improvement process.
The review of improvement models is mainly concerned with the improvement
framework and assessment rather than the validity of the practices that the models
recommend. All three models have derived the set of RE practices that they
recommend from a variety of sources including practices widely used in industry,
practices recommended or mandated in standards and practices learned from direct
experience. However, as Wiegers (1999) warns:
“… not all … have been endorsed as industry best practices …”
Nevertheless, the developers of each model have been careful not to recommend
practices that have not been validated in some form and shown to be practical and
practicable for practitioners.
The Requirements Engineering Good Practice Guide
The REGPG (Sommerville, 1997) was the first public-domain process improvement
and assessment model for RE. Like the SW-CMM, the REGPG uses an improvement
framework with several process maturity levels (Fig 2). The REGPG maturity levels
are design to mirror the SW-CMM levels. However, at the time of REGPG's design,
the adoption of RE practices across the industry was patchy (El Eman, 1995;
Forsgren, 1995) and there was insufficient empirical evidence for the existence of
requirements processes that could be characterised beyond Defined (level 3). The
characteristics of highly mature RE processes were essentially unknown so the
REGPG mirrors only the SW-CMM's bottom three levels (initial, repeatable and
defined).
Level 1Initial
Level 2Repeatable
Level 4Managed
Le vel 5Optimizing
Level 3Defined
Figure 2. The 3-level REGPG process maturity model
In the REGPG model:
• Level 1 - Initial level organizations have an ad hoc requirements process. They
find it hard to estimate and control costs as, for example, many requirements have
to be reworked. The processes are not supported by planning and review
procedures or documentation standards. They are dependent on the skills and
experience of the individuals who enact the process.
• Level 2 - Repeatable level organizations have defined standards for requirements
documents which are more likely to be of a consistently high quality and to be
produced on schedule. They also have policies and procedures for requirements
management.
• Level 3 - Defined level organizations have a defined process model based on good
practices and defined methods. They have an active process improvement
programme in place and can make objective assessments of the value of new
methods and techniques.
The key to improvement is in adopting appropriate good practices, in the right order,
at the right pace and with the required degree of strategic commitment. The REGPG
distils guidance on good practice adoption into 66 guidelines (key practices in SW-
CMM terms). The practices are classified according to whether they are Basic,
Intermediate or Advanced to reflect, for example, dependencies among the practices.
The practices are organised according to process activities. These are the
requirements document, requirements elicitation, analysis and negotiation, describing
requirements, system modelling, requirements validation, requirements management
and RE for critical systems. Although analogous to KPAs, REGPG process activities
serve only to classify good practices and form no part of the maturity framework.
Unlike the SW-CMM (but like the more recent SW-CMMI and ISO/IEC 15504), the
REGPG uses a continuous architecture. This means that the REGPG does not
prescribe the process activities to be targeted at each improvement stage. Instead
organisations can adopt practices across a range of process activities according to
their priorities.
Before improvement can be achieved, causality has to be established between the
evident problems and weaknesses in the organisation's RE process. Discovering
weaknesses and identifying priorities for targeted improvement is the role of process
assessment. Process assessment is based on a system of checklists designed to reveal
what practices are in use and the extent to which they are used. A checklist of the 66
REGPG guidelines is used as the starting point for the assessment. The activities used
in the assessment are:
1 Prune checklist Assign a score of 0 (see below) to each practice that is
already known not to be used. This is simply a
pragmatic step for streamlining the assessment.
2 Select the right
people to interview
Knowledge of an organisation's RE practices may
reside with many people, particularly where there are
different units working on different products/projects.
This activity is tasked with selecting a small group of
people in order to gain a representative snapshot of RE
practice within the organisation.
3 Score process
against checklist
The purpose of this stage is to assign scores against
each practice for which a score can be confidently
assigned and to flag those practices where the extent of
adoption is uncertain.
4 Resolve uncertainty This stage focuses on seeking additional information
(perhaps by interviewing other people, looking for
documentary evidence, etc.) that allows a score to be
assigned to the uncertain practices identified at stage 3.
5 Compute maturity This stage derives an overall maturity score based on
the results of stages 1, 3 and 4.
During the assessment process, each good practice is assessed as being:
• Standardised (score 3). The practice has a documented standard in the
organisation and is integrated into a quality management process.
• Normal use (score 2). The practice is widely followed in the organisation but is
not mandatory.
• Discretionary (score 1). Some project managers may have introduced the practice
but it is not universally used.
• Never (score 0). The practice is never or rarely applied.
The maturity level is calculated by summing the numerical scores for each practice
used:
• Level 1 (initial) processes score less than 55 in the basic guidelines.
• Level 2 (repeatable) processes score over 55 in the basic guidelines but less than
40 in the intermediate and advanced guidelines.
• Level 3 (defined) processes score over 85 in the basic guidelines and more than 40
in the intermediate and advanced guidelines.
The end result is an indicative maturity level for the organisation's RE process, and a
map of whether, and to what extent, each of the 66 RE practices described in the
REGPG is used in the RE process. Subsequent improvement is based on identifying
un- or under-used practices that:
• are likely to yield benefits that outweigh the cost of their introduction;
• can be feasibly introduced given dependencies on underpinning practices.
The practices' classifications (basic, intermediate, advanced) give some help in
selecting an appropriate subset for introduction. The classifications are supplemented
with additional qualitative information in the good practice guidelines about benefits,
how to implement them, and associated costs and problems. At the end of an REGPG
process assessment, therefore, an organisation should have the basis for an RE process
improvement agenda in terms of areas of weakness and options for improvement.
The two more recent models described below both share broadly similar aims and
comprise a similar maturity framework.
The Requirements Engineering Process Improvement Model
The Requirements Engineering Process Maturity Model (REPM) (Gorschek, 2003) is
targeted at SMEs which, its authors observe, lack the resources to apply models like
the REGPG. Instead, REPM claims to take an approach to the RE process that is
sufficient to:
“… give an indication of whether a problem exists, and … where the problem areas
reside.” (Gorschek, 2003)
REPM uses a six-level maturity model. It actually defines five but since it is staged
model and level one has eleven actions (analogous to SW-CMM key practices or
REGPG practices) associated with it, there is an implicit level zero that broadly
corresponds to the SW-CMM's initial level. Each action is associated with one of the
levels one to five and, additionally, is classified according to three process activities
(called, in REPM, main process areas or MPAs): Elicitation, Analysis and
Negotiation, and Management. Each maturity level has a set of actions defined for it
under each of the three MPAs. Despite their superficial similarity, MPAs are therefore
unlike SW-CMM KPAs which are only associated with a single maturity level.
However, MPAs are also different to process activities in the REGPG where it is not
obligatory to implement practices for each process activity in order to achieve a given
maturity level.
MPAs may be further classified into sub-process areas (SPAs). For example, the
Elicitation MPA comprises Stakeholder identification, Stakeholder consulting,
Domain knowledge and Scenario elicitation. The authors argue that this allows the
model to evolve easily since a composite action may be promoted to an SPA allowing
new actions to be derived from it.
REPM is designed for project rather than organisational assessment. Like the
REGPG, it uses checklist-driven interviews of key personnel in order to make an
assessment. Because an REPM assessment is scoped at the project level, selection of
personnel to interview defaults to the person responsible for RE in the project under
assessment. This obviates the need to select a range of people across an organisation
(c.f. step 1 in the REGPG assessment model) and minimises the interaction between
assessor and practitioners.
The project evaluation checklist is, like that of the REGPG, a list of all 60 actions.
REPM recognises that failing to implement an action does not necessarily fatally
weaken the RE process, provided there is a sound rationale for not doing so.
Producing a user manual draft (a level 2 action) may be meaningless for the
development of an embedded system, for example. This allows checklist actions to be
marked satisfied-explained. Actions in this category carry the same weight as
completed actions.
As with the REGPG, the authors of REPM have selected a set of RE practices (actions
in REPM terms) and made a judgement about how fundamental they are to an RE
process. Because REPM uses a staged model, each action is locked in to a particular
maturity level. To achieve a maturity level n, a process needs to have completed or
satisfied-explained all of the actions associated with levels one through n.
The University of Hertfordshire Model
The RE process improvement model (Beecham, 2003) developed at the University of
Hertfordshire is a direct adaptation of the SW-CMM framework for RE processes.
This is intended to help dovetail RE process improvement with a wider SW-CMM-
based software process improvement programme.
The Hertfordshire model uses the 5 SW-CMM maturity levels (Fig 1) to classify RE
processes. Since it is based upon the SW-CMM it uses a staged architecture in which
a set of 68 practices (called processes) are mandated for each maturity level. Like the
REGPG and REPM, these are classified according to RE process activities, called
phases. These are management, elicitation, analysis and negotiation, documentation
and validation. Like REPM MPAs, each phase in the Hertfordshire model defines a
set of processes at each maturity level.
As with the other RE process improvement models reviewed here, the Hertfordshire
model draws its set of processes from analysis of RE practice in industry (including
(Hall, 2002)). However, to help integrate the model with the SW-CMM, some
processes are drawn directly from the SW-CMM. For example, the Hertfordshire level
2 process P1 below is essentially the same as the SW-CMM level 2 requirements
management KPA commitment to perform key practice.
P1. Follow a written organisational policy for managing the system requirements
allocated to the software project.
Process assessment is broadly similar to that of the REGPG even to the extent that, as
currently defined in the available documentation, it reflects a continuous rather than a
staged improvement model. In order to assess the extent to which a process is
satisfied by an organisation, the Hertfordshire model allocates a score to each process
against three assessment criteria. These are:
• Approach. A measure of the organisational commitment and capability.
• Deployment. A measure of the extent to which a process is implemented across
the organisation.
• Results. A measure of the success of a process' implementation.
• Each process is assessed against each of these criteria and given a rating of:
− Outstanding (10)
− Qualified (8)
− Marginally qualified (6)
− Fair (4)
− Weak (2)
− Poor (score 0)
The average of the score for approach, deployment and results is recorded for each
process. Hence, if a process was marginally qualified in terms of approach, fair in
terms of deployment and weak in terms of results it would rate an average score of
fair (4). The scores are then summed for each phase (e.g. to assess the strength of
requirements management in an organisation) and the sum of all five phases yields an
overall score.
The Hertfordshire model is still under development. At the time of writing, the model
has not been calibrated to map overall scores onto maturity levels and the relationship
of this to the staged model implied by the association of processes with maturity
levels has not been worked through. It has undergone an initial validation phase that
has focused on the overall framework and the processes and process descriptions used
for maturity level 2. The set of processes and process descriptions for levels 3 to 5
currently exist in draft form.
Model Maturity levels CMM Key Process Area analogues?
CMM Key Practice analogues?
Staged or Continuous?
REGPG 3 Process activities but no direct
Good practice guidelines
Continuous
analogy REPM 6 (1-5 with an
implicit level 0) Main process areas but no direct analogy
Actions Staged
Hertfordshire model
5 Phases but no direct analogy
Processes Staged but assessment is currently continuous
Table 1. Summary of RE Improvement Models
EXPERIENCE OF RE PROCESS IMPROVEMENT
Process improvement is a costly activity that requires substantial investment.
Organisations therefore require confidence in the improvement models that they use.
This is not simply confidence that the set of practices that a model embodies are
proven in industry. Confidence is required that the maturity framework and
assessment method will not caricature the organisation's processes. Organisations
need to know that the process weaknesses revealed by an assessment are real
weaknesses that inhibit process performance. They also need to know that the
remedial action proposed by process analysts will address real organisational
priorities in a cost-effective way. Validation is important for establishing confidence
in improvement models and each of the models reviewed in the previous section have
undergone some form of validation.
REGPG
Over 10000 copies of the REGPG have been sold and, as the longest-established and
most widely-disseminated model, it should be unsurprising that the REGPG has
undergone the most extensive validation. It has been the subject of two projects
specifically intended to validate and improve the model: Qure (Kauppinen, 2002) and
Impression (Sommerville, 2003).
The Qure project conducted by the Helsinki University of Technology included an RE
process improvement theme. The REGPG was evaluated as part of this in which the
model was piloted in four companies working in a range of application domains. The
scope of the analysis was to discover whether the REGPG process analysis method
was accurate and usable. The question of whether a subsequent improvement cycle
based upon the results of the analysis resulted in real improvement was outside the
project's scope.
Qure found that, in the case studies used, the REGPG yielded useful results and was
appropriate for organisations beginning an RE process improvement programme. A
key weakness discovered was that the selection of good practices to address revealed
process weaknesses required more guidance and that, as a result, companies tended to
be over-ambitious in the improvement programmes that they undertook.
This general point was echoed in the results of the Impression project which
concluded that the assessment method was too passive. The scope of the REGPG
validation in Impression was wider than that of Qure. It involved nine companies
from a variety of domains and ran for a full improvement cycle from initial
assessment through good practice introduction to follow-up assessment. Like Qure,
assessment and improvement piloting was performed by a third party organisation.
However, REGPG authors acted as trainers and consultants for the staff performing
the process assessments.
The passivity of the assessment method emerged when, perhaps unsurprisingly, it
became apparent that the companies attached greater priority to actual improvement
than mere diagnostics and maturity classification. The assessment method was
modified to include an explicit step for selecting good practices to address the
revealed weaknesses. This was backed up with the development of a decision support
tool that processed the analysis data and listed those best practices likely to provide
solutions most cost-effectively.
Other significant results included:
• Implicit dependencies between practices (e.g. that intermediate practices cannot be
introduced until appropriate basic practices have been deployed) were sometimes
wrong. This implies greater than anticipated freedom of choice in the selection of
practices for introduction and is something that would be hard to accommodate in
a staged architecture.
• The maturity model was not entirely generic and different norms of practice that
derive from different priorities across application domains were not accurately
reflected. The REGPG evolved from the experiences of the REAIMS project's
industrial partners who worked in dependable systems domains. The REGPG
inevitably retains some of this flavour which fits slightly awkwardly with, for
example, socio-technical systems.
At the end of the Impression assessment-improvement cycle a follow-up assessment
was performed. This established that, in terms of the REGPG maturity model, all but
one of the nine companies had improved their RE processes. Four had moved from
Initial to Repeatable. The remaining four had all improved their score but remained at
the Initial level. The strategies that companies used to improve ranged from
concentrating on further embedding existing practices within company standard
procedures to introducing large numbers of new practices (one company introduced
14).
Although Impression covered a whole improvement cycle and appears to demonstrate
the REGPG's utility for improving RE processes, there was insufficient time to follow
this through and establish whether increased process maturity in REGPG terms was
reflected in concrete business benefits. The signs are good but unproven.
REPM
(Gorschek, 2003) describes the evaluation of REPM using a pilot study involving four
companies. These were SMEs of the size for which REPM was designed. The scope
of this pilot was more restricted than that of Qure or Impression and had a dual aim of
using REPM to discover norms of RE practice (which we ignore here) and of
validating REPM.
Despite the limited scope of the REPM validation, the results indicate that its
lightweight assessment method yields useful results. REPM as a vehicle for
improvement has not been demonstrated but within the limited aims of REPM, the
model appears to show promise as an assessment tool for SMEs. REPM was explicitly
designed to be lightweight and the authors report no serious problems with applying
it. However, validation of the model's applicability does not appear to have been an
explicit goal of the evaluation.
University of Hertfordshire Model
The methodology for validation of the Hertfordshire model differs from that used for
the REGPG and REPM. The model's development was in part influenced by a study
of RE practice in twelve companies (Hall, 2002). Validation of the model itself,
however, has to date been performed by assessment by a panel of experts rather than
deployment in practitioner companies. At the time of writing the results of this
exercise were being processed and will feed into maturation of the model.
CONCLUSIONS
Given the level of interest in SPI in recent years, RE process improvement has
received surprisingly little attention from the SPI and RE research communities or
from industry. This may be because RE process improvement takes place, but takes
place adequately without the formal framework of a CMM-like maturity model.
Nevertheless, RE process improvement is an important issue as surveys such as that
reported by Ibanez (1996), clearly demonstrate.
Organisations interested in RE process improvement and who, perhaps because of
investment in wider SPI programmes, wish to use a maturity level framework now
have a choice of at least the three improvement models reviewed here.
Of these, the REGPG has the benefit of being widely known. It has been validated
across a range of domains and it's strengths and weaknesses have been identified. The
REPM and the Hertfordshire models currently occupy more specialised niches.
REPM has been purposely designed as a lightweight method suitable for SMEs. It is
perhaps the model most in tune with the current mood for agile and lightweight
methodologies although it has not yet completed a full validation cycle.
The Hertfordshire model is a work in progress that is carefully aimed at the many
companies who have already invested in SW-CMM based improvement programmes.
If subsequent development can complete the model and integrate it within the SW-
CMM framework (and track CMM developments), then it has the potential to offer
substantial benefits to companies.
REFERENCES
Beck, K. (1999). Extreme Programming Explained: Embrace Change. Boston:
Addison-Wesley Longman.
Beecham, S., Hall, T. & Rainer, A. (2003). Defining a Requirements Process
Improvement Model. Technical Report TR379. University of Hertfordshire.
El Eman, K. & Madhavji, N. (1995). A Field Study of Requirements Engineering
Practices in Information Systems Development. Proc. 2nd IEEE International
Symposium on Requirements Engineering (RE95), York, UK. 68-80.
Forsgren, P. & Rahkonen, T. (1995). Specification of Customer and User
Requirements in Industrial Control System Procurement Projects. Proc. 2nd IEEE
International Symposium on Requirements Engineering (RE95), York, UK. 81-88.
Fowler, P., Patrick, M., Carleton, A. & Merrin, B. (1988). Transition Packages: An
Experiment in the Introduction of Requirements Management. Proc. Third
International Conference on Requirements Engineering (ICRE’98), Colorado Springs,
Colorado. 138-147.
Garcia, S. (1997). Evolving Improvement Paradigms: Capability Maturity Models &
ISO/IEC 15504 (PDTR). Software Process: Improvement and Practice, 3(1), 47-58.
Gorscheck, T., Svahnberg, M. & Kaarina, T. (2003). Introduction and Application of
a Lightweight Requirements Engineering Process Evaluation Method. Proc.
Requirements Engineering Foundations for Software Quality '03 (REFSQ'03),
Klagenfurt/Velden, Austria. 83-92.
Hall, T., Beecham, S. & Rainer, A. (2002). Requirements problems in twelve software
companies: an empirical analysis, IEE Proceedings-Software, 149 (5), 153-160.
Humphrey, W. (1989). Managing the Software Process. Boston: Addison-Wesley
Longman.
Ibanez, M. & Rempp, H. (1996). European Survey Analysis. ESSI Project technical
Report. European Software Institute.
IEEE Std 1233-1998. (1998). IEEE Guide for Developing System Requirements
Specifications. New York: IEEE.
IEEE Std 1362-1998. (1998). IEEE Guide for Information Technology - System
Definition - Concept of Operations (ConOps) Document. New York: IEEE.
IEEE Std 830-1998. (1998). IEEE Recommended Practice for Software
Requirements. New York: IEEE.
Kauppinen, M., Aaltio, T., & Kujala, S. (2002). Lessons Learned from Applying the
Requirements Engineering Good Practice Guide for Process Improvement. Proc.
Seventh European Conference on Software Quality (QC2002), Helsinki. 73-81.
Konrad, M. & Paulk, M. (1995). An Overview of SPICE's Model for Process
Management. Proc. Fifth International Conference on Software Quality, Austin,
Texas. 291-301.
Mazza, C., Fairclough, J., Melton, B., De Pablo, D., Scheffer, A., Stevens, R., Jones,
M. & Alvisi, G. (1994). Software Engineering Guides. London: Prentice Hall.
Paulk, M. (1994). A Comparison of ISO 9001 and the Capability Maturity Model for
Software, CMU/SEI-94-TR-12, Software Engineering Institute.
Paulk, M. (2001) Extreme Programming from a CMM Perspective. IEEE Software,
18 (6), 19-26.
Paulk, M., Curtis, W., Chrissis, M. & Weber, C. (1993). Capability Maturity Model
for Software, Version 1.1. CMU/SEI-93-TR-24. Software Engineering Institute.
Sawyer, P., Sommerville, I. & Viller, S. (1999). Capturing the Benefits of
Requirements Engineering. IEEE Software, 16 (2), 78-85.
SEI. (2002). CMMI for Software Engineering, Version 1.1, Continuous
Representation (CMMI-SW, V1.1, Continuous). CMU/SEI-2002-TR-028. Software
Engineering Institute.
Sommerville, I. & Ransom, J. (2003). An Industrial Experiment in Requirements
Engineering Process Assessment and Improvement. Technical report, Computing
Department, Lancaster University.
Sommerville, I. & Sawyer, P. (1997). Requirements Engineering - A Good Practice
Guide. Chichester: John Wiley.
Wiegers, K. (1999). Software Requirements, Redmond: Microsoft Press.
Young, R. (2001). Effective Requirements Practices, Boston: Addison-Wesley
Longman.
Biography
Pete Sawyer holds a BSc and PhD from Lancaster University where he is a senior
lecturer in the Computing Department. His research interests are in the general area of
Software and Systems Engineering. In particular, he is interested in requirements
engineering, software process improvement and applications of natural language
processing to the systems engineering process. He is a member of the British
Computer Society (BCS), an affiliate member of the Institution of Electrical and
Electronics Engineers (IEEE) and a professional member of the Association of
Computing Machinery (ACM). He is also a member of the executive committee of the
BCS Requirements Engineering Specialist Group (RESG). He has over fifty peer-
reviewed publications and is the co-author (with Ian Sommerville) of Requirements
Engineering - A Good Practice Guide (John Wiley, 1997).
Top Related