Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques
-
Upload
susan-schanta -
Category
Documents
-
view
184 -
download
0
Transcript of Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques
© 2015 Cognizant
© 2015 Cognizant
Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques
October, 2015
Susan Schanta, Director
© 2015 Cognizant 2
Agenda
2
Topic Page
Section 1: Improving Elicitation Sessions through the use of Elicitation Checklists 4
Common Challenges 5
Introduction to Industry Standards 7
Industry Requirement Definitions 8
S.M.A.R.T. Requirements 9
Create Common Language for WHAT vs. HOW 10
Defining Elicitation 11
Pre-Elicitation Planning & Meeting 12
Meeting Etiquette 14
Elicitation Checklists 15
Building Elicitation Checklists 18
Section 2: Removing Ambiguous Language from Requirements 19
Ambiguity Reviews 20
Requirements Review Phase 21
Ambiguity Types 22
© 2015 Cognizant 3
Agenda
3
Topic Page
Section 2: Removing Ambiguous Language from Requirements (Continued)
Ambiguity Examples 23
Ambiguity Words 24
Section 3: Measure Requirements Effectiveness through Metrics 25
Impact of Poor Requirements in the Industry 26
Measuring Requirements Within & Across Projects 27
Ambiguity Review Metrics 28
References 31
© 2015 Cognizant 4
Improving Elicitation Sessions through the use of Elicitation Checklists
© 2015 Cognizant 5
Common BA Challenges Strategic Approach
BA Ownership – Role & Responsibilities
Business Analysts do not own the Elicitation and Requirements Documentation Process
BA Team lacks involvement in full software lifecycle
Assess and implement a standard approach that positions the BA as the requirements owner based on industry best practices with customization where needed to address your specific needs
Development Driven Requirements
Requirements are written in technical speak
Technically driven requirements may not address business need
Create guidelines for authoring requirements that define the user’s interaction with the system
Assess current Technology documentation to determine best approach to pair it with requirements that define business needs
Elicitation & Documentation Standards
Lack of standard requirements templates
Lack of standard methodology
Lack of End to End Impact Assessment of Proposed Functionality
Apply industry best practices as documented in BABOK@ and Karl Wiegers Software Requirements
Introduce Ambiguity Reviews to measure adoption of standards
Develop BA Style Guide based on lessons learned to drive continual improvement
Common BA Challenges
© 2015 Cognizant 6
Common BA Challenges Strategic Approach
Requirements Quality
Lack of defined Elicitation Session approach leads to inconsistent requirements
Multiple interpretations applied to written requirements
Technology drives the approach where gaps exist in requirements
Define methodology to elicit requirements Identify SMEs vs. Approvers Conduct Pre-Elicitation Meeting to establish topics, timelines and meeting
participants
Introduce Ambiguity Reviews to measure adoption of standards
Develop BA Style Guide based on lessons learned to drive continual improvement
Financial Impact
Delayed project release due to: Scope Creep Requirement Change Requests Defect Rates
Set benchmarks to create platform for financial measurement Establish benchmark of evidence to measure financial impact
Measure of Effectiveness
Increased Change Requests to Address Requirement Gaps
Increased Defect Leakage to QA/UAT/Production
Negative Impact to Upstream/Downstream Systems
Limited Requirements to TC traceability
Create scorecard to measure improvement rates Defect leakage rates from Requirements to Development to QA/QA to
UAT/UAT to Production Increase in requirements traceability Rate of Change Requests Impact to upstream/downstream system interfaces
Strategic Approach
© 2015 Cognizant 7
Introduction to Industry Standards
Use Standard Set of Industry Defined Templates − Business Requirements state the functional goal − User Requirements describe the interaction between the user and system − Functional Requirements state the system behavior required to fulfill the user requirement
Drive Requirements Gathering through a Defined Elicitation Process
− Pre-meetings with business stakeholders − Early analysis to define scope of Elicitation Sessions − Document process paths
Introduction of Ambiguity Reviews
− Requirements are clear, concise and measurable − Lessons learned are formulated and applied to continuing to improve the Requirements Phase
Requirements Traceability Matrix
− Requirements are traced to test cases to ensure implementation and exercise of application functionality prior to production launch to ensure fulfillment, stability and reliability
Standard Measurement
− Success measured through metrics • Requirements measurability through Ambiguity Reviews • Requirement Defects • Volume of Change Requests
A Culture of Excellence in the Requirements Phase
© 2015 Cognizant 8
Industry Requirement Definitions
Business Requirement A business requirement is a goal of the organization requesting the system. User Requirement A user requirement is a task that the user must be able to accomplish using the system. • User requirements • User stories • Use cases Functional Requirement A functional requirement is a conversation between the system and a user or other system requesting/providing information. A functional requirement states the change or addition of a system feature to satisfy the user requirements. Nonfunctional Requirement A nonfunctional requirement is a quality attribute that the system must have. These attributes are not system features (functional requirements), but they do influence how the functionality of the system is implemented. Nonfunctional requirements deal with some aspect of usability, performance, or security or are operational in nature.
Laying the Foundation for a Common Understanding…
© 2015 Cognizant 9
S.M.A.R.T. Requirements
Specific requirements are precise
− The requirement has the appropriate level of detail and states exactly what is required
− The definition of the requirement must result in only one interpretation no matter who is reviewing it
− The use of conjunctions (and, or, but) are avoided to prevent misinterpretation
Measurable requirements can be verified as complete
− The behavior described in the requirement provides an expected outcome
− Ambiguous adjectives/adverbs have been removed to remove a subjective interpretation to the requirement
− Test scenarios can be derived from the requirement
Attainable requirements are actionable
− The requirement is not gold-plated or too grandiose to be fulfilled
− The requirement can be achieved given current environments
Realistic requirements are appropriate in relation to other requirements
− Can the requirement be fulfilled given the current systems?
− Can the requirement be satisfied within the project budget and resource constraints?
Time-Bound (Traceable) requirements have time parameters
− The requirement provides a specific time when the requirement needs to be completed or executed
− The requirement avoids ambiguous references such as quickly, fast or immediately
Requirements are deemed measureable when they are S.M.A.R.T
© 2015 Cognizant 10
Create Common Language for “What” vs. “How”
“What the user is able to do with the system.”
Requirements are a detailed instruction of WHAT the system will do Business requirements state WHAT will be achieved by implementing stated requirements User requirements state WHAT the flow will be for scenarios, conditions and outcome required Describes the Happy Path or the ideal flow to perform a function Describes alternate flows for exception conditions and special circumstances Describes constraints that restrict the function Describes error conditions resulting in failure
Functional requirements detail WHAT feature will be built into the system Nonfunctional requirements detail WHAT type of performance is expected from the system and directly
related to customer and user satisfaction. Technical Design details HOW the system will fulfill the business, user and functional requirements Technical requirements describe HOW the requirements will be fulfilled
© 2015 Cognizant 11
Defining Elicitation
Elicitation is a collaborative process to identify requirements through elicitation sessions with business users. The Business Analyst engages with stakeholders and subject matter experts to define the AS IS and TO BE process. The process of elicitation allows the Business Analysts to collect, analyze and define requirements based on the information collected in these sessions.
© 2015 Cognizant 12
Pre-Elicitation Planning & Meeting
Gather and review available documentation
Hold Pre-Elicitation Meeting
− Work with business sponsor to write Scope Statement to clarify what is included or not included
Define Discussion Topics
− Review Elicitation Checklist
− Identify SMEs by topic
− Estimate time necessary to elicit requirements for each topic
− Identify key stakeholders needed at the elicitation meeting and ascertain who provides approval
Select Elicitation Session format
Establish Guidelines for Meeting Etiquette
Identify action items needed for follow-up
Create agendas for Elicitation Sessions
Send invitations to meeting attendees
Setting Expectations for Elicitation
Subject: E&E Congressional Group Setup – PLEASE DO NOT FORWARD
***************************************************************
Invited: Joe Smith, John Doe, Sally Mill, Jane Green, Sam Jones, Mike Smith, Donna White
Objective: Define special processing rules for Congressional Group Setup
Schedule:
11:00 to 11:30 Incoming 820 Enrollment File differences
11:30 to 12:00 New Business Rules for processing Congressional members
12:00 to 12:30 Arrears processing for Congressional members
Roles/Responsibilities
Business Analyst/Meeting Facilitator Peter Graves
Requirements Recorder Martin Landau Primary Business User Greg Morris
Timekeeper Martin Landau
If you believe this invitation should have included someone who is not invited, please contact the meeting organizer.
© 2015 Cognizant 13
Requirements Pre-Elicitation PlanningB
usi
ne
ss S
po
nso
rB
usi
ne
ss A
nal
yst
Requirements Phase
Start Pre-Elicitation Planning
Gather Information & Schedule
Meeting
Meeting with Business Sponsor to
Confirm Scope & Plan Elicitation
Elicit Scope of Enhancements
& Modifications
Identify SMEs by Topics &
Sub-Topics & Time Needed
Assign Stakeholder Approvals
Capture Scope, SMEs and Approval Stakeholders
Present Elicitation
Checklists to Business Sponsor
Review Elicitation
Checklists w/Business Sponsor
Set Expectations
for SME preparedness
Schedule Elicitation Sessions (Restrict Access)
Establish Agenda for Elicitation Sessions
Determine if Follow-up
Action Items Needed
End Pre-Elicitation Planning
© 2015 Cognizant 14
Meeting Etiquette Streamlining Elicitation Session Participation
Facilitating the Elicitation Session:
Two BA’s will be present – 1 for elicitation/1 to capture requirements and meeting notes
Opening the Meeting:
Establish the protocol for each meeting at the beginning of the session by reviewing the:
Agenda topics
Time allowance for each topic
Process to park items outside of scope for the discussion
Conference Call Etiquette:
When key stakeholders are participating in a conference call format, establish protocol for phone etiquette:
All phones must be put on mute when the participant is not speaking
The Business Analyst reserves the right to put everyone on mute if people begin talking over each other without respecting the person who was speaking in order to regain control of the discussion
− The Business Analyst will bring the team back to the discussion at hand
The Business Analyst will return the floor to the person who was speaking when the interruption occurred
© 2015 Cognizant 15
Elicitation Checklists Establish Expectation for SME Preparedness
Present the Elicitation Checklists targeted for use in sessions • Review checklist to set expectation for the content to be covered in
Elicitation Sessions • Questions will be grouped based on subject matter by topic
© 2015 Cognizant 16
Elicitation Questions for User/Functional Requirements
Does the requirement state what triggers the action performed in the requirement?
Does the happy path user requirement detail the workflow and expected outcome?
Does the requirement identify the user performing the action?
Does the requirement state the permissions required to perform the operation?
Are the requirements boundaries clear? For instance, if a payment file is received monthly, what day of the month is it received?
Are the business rules stated that determine whether a condition is true?
Does the requirement address error conditions with the expected outcome? Are error messages given?
Does the requirement include exceptions such as responses to abnormal situations including error handling and recovery?
Does the requirement address alternate flows with an expected outcome?
Does the requirement address exceptions with an expected outcome?
Are user requirements created for alternate flows with the applicable workflow and expected outcome?
Are user requirements provided for error conditions with the applicable workflow and expected outcome?
Are valid inputs and outputs stated where required?
Is the sequence of operations or processing steps provided?
Is the effect of parameters changing considered?
Is the relationship of outputs to inputs stated, including
Input/ output sequences
Formulas for input to output conversion
‘As Is’ and ‘To be’ Process Flow diagrams
Business Rule & Workflow Focus – What is the Expected Outcome?
© 2015 Cognizant 17
Elicitation for Nonfunctional Requirements
• What are the initial estimated users of the application?
• What is the expected growth rate (in terms of users) of the application annually?
• What is the number of users accessing the site per day, peak hours, off-peak hours, Monday morning, Friday evening (or any specific day depending on the nature of site)?
• What is an acceptable response time for web enabled application that includes back end processing?
• What is an acceptable response time for processing handled by the internal enterprise router, firewall and web server?
• What is an acceptable response time when calling an external system?
• What are peak periods and non-peak periods daily, weekly, monthly and annually?
• What are the transaction volumes and number of users that define peak processing time?
• What are the transaction volumes and number of users that define non-peak processing time?
• Will the users perform complex queries during peak periods?
• Will the users perform complex calculations during peak periods?
Avoid use of words such as fast, efficiently, immediately…
© 2015 Cognizant 18
Building Elicitation Checklists
Internal Sources for Elicitation Checklists
Clarification Trackers
Change Requests
Defects
Internal Interviews with Project Stakeholders
Internal reviews with Business Analyst Team
Internal reviews with Development
Internal reviews with Quality Assurance
Internal reviews with Performance Engineers
Web Searches
http://www.bridging-the-gap.com/what-questions-do-i-ask-during-requirements-elicitation/
http://www.batimes.com/articles/questions-for-eliciting-user-requirements.html
http://www.bbc.co.uk/guidelines/dq/pdf/is/is_requirements_questionnaire.pdf
http://reqtest.com/requirements-blog/how-to-use-interviews-to-gather-requirements/
http://www.seilevel.com/wp-content/uploads/RequirementsGatheringTools.pdf
Using Past Clarifications to Improve Future Requirements
© 2015 Cognizant 19
Removing Ambiguous Language from Requirements
© 2015 Cognizant 20
Ambiguity (am·bi·gu·i·ty) Why do an Ambiguity Review?
20
Ambiguity can be defined as… 1. Use of words that allow alternative interpretations 2. An unclear, indefinite, or equivocal word, expression, meaning, etc. 3. The possibility of interpreting an expression in two or more distinct ways An Ambiguity Review is a formal review process that focuses on the identification of ambiguities in language, structure and logic of a requirement. The process is a collaboration between the reviewer and the Business Analyst who authored the requirements document. Review a document from the consumer’s viewpoint Assumes the reader has no knowledge of the subject Why do an Ambiguity Review? Limit Scope Creep Increase Requirements Traceability to Test Cases Reduce Defect Leakage to Production Improves Estimation Accuracy Reduce Cost of Maintenance How much is your organization spending to maintain systems today?
Reduces redo work Increases velocity of Test Case creation Increase BA productivity and work product
Fortune 50 Insurance Company #1 Reduced ambiguities from 11 to 3 per
Requirement Reduced Defect Leakage from 35% to 8% Introduced BA Style Guide to drive a
consistent approach Fortune 50 Insurance Company #2 63% of all defects were traced to
Requirement Defects
© 2015 Cognizant 21
Requirements Review Phase
© 2015 Cognizant 22
Ambiguity Type Description
Ambiguous Term Terms (Phrase or Word) used in requirements which can be interpreted by the reader in multiple ways e.g. frequently, occasionally, efficiently
Conflicting Requirement Requirements which contradict each other – either in the same document or across multiple documents. e.g. The field name is Effective Date but the data type is defined as an integer
Glossary Word or acronym used in requirements that is new or not commonly used but has not been defined in the Glossary/Definitions section.
Grammar, Spelling & Wording
Grammar, spelling corrections and proposed wording improvements to increase clarity of the requirement
Incomplete Requirement Incomplete requirement or statement describing conditions when information is not fully detailed preventing design or test validation e.g. The system shall handle 15-25% increase in the second year
Missing Requirement Missing requirements that were not documented or may not have been elicited from the business user. e.g. Missing requirements – alternative flows, business rules, exceptions and error conditions (Questions – What, When, Where)
Unclear Requirement Requirements or statements requiring further clarification to allow the reader user to fully understand the requirement (Questions – How, Why, What do you mean by…)
(Wiegers, Bender) (Bender/Wiegers)
© 2015 Cognizant 23
Ambiguity Examples
Ambiguity of Reference & Ambiguous Statements A condition when a requirement uses words such as pronouns, adjectives, adverbs and verbs that can be interpreted differently based on the reader’s view. Boundary Ambiguity A condition when the author uses terms - among or up to. The scope of the requirement is ambiguous because the stated requirement can be interpreted in multiple ways.
The payment file shall be sent monthly. Ambiguity: What is the name of the payment file? Ambiguity: What day of the month is the payment file sent? Ambiguity: Is the payment file sent on the business day before or after if the date falls on a weekend or holiday? Ambiguity: What happens if there is no data for the payment file? Ambiguity: What happens if the payment file generation fails?
(Wiegers, Bender)
1. If an employee makes less than $20,000 per year, the employer pays 100% of the healthcare premium. 2. If an employee makes more than $20,000 per year, the employer pays 50% of the healthcare premium for the employee. Ambiguity: What if an employee makes exactly $20,000?
(Bender/Wiegers)
© 2015 Cognizant 24
Ambiguity of Reference Ambiguous Adjectives Ambiguous Adverbs Ambiguous Variables Ambiguous Verbs
above ordinary infrequently the database derive
below rare intuitively the field determine
it same just about the file edit
such seamless more often than not the frame enable
the previous several more or less the information improve
them similar mostly the message indicate
these some nearly the module manipulate
they standard normally the page match
this the complete not quite the rule maximize
those the entire often the screen may
Ambiguous Adjectives transparent on the odd occasion the status might
all typical ordinarily the system minimize
any usual rarely the table modify
appropriate valid roughly the value optimize
custom Ambiguous Adverbs seamlessly the window perform
efficient accordingly seldom Ambiguous Verbs process
every almost similarly adjust produce
few approximately sometime alter provide
frequent by and large somewhat amend support
improved commonly transparently calculate update
infrequent customarily typically change validate
intuitive efficiently usually compare verify
invalid frequently Ambiguous Variables compute
many generally the application convert
most hardly ever the component create
normal in general the data customize
(Bender/Wiegers)
© 2015 Cognizant 25
Measure Requirements Effectiveness through Metrics
© 2015 Cognizant 26
Impact of Poor Requirements in the Industry
Countless whitepapers have been written documenting the impact of poor requirements on a company’s enterprise success.
• As high as 60% in time and cost premiums are paid on projects with poor requirements
• Fewer than one third of companies are well-equipped to produce quality requirements
• Poor requirements consume approximately 41.5% of the average IT budget due to maintenance and redo work
• Only 20% of projects are delivered on time with 28% within budget
Impact of Poor Requirements
Impact of Requirements to Product Quality* Defects found during requirements $250 Defects found during design $500 Defects found during coding and testing $1,250 Defects found after release $5,000 * May 5, 2009, Capers Jones, Capers Jones & Associates LLC, “A Short History of the Cost per Defect Metric v1.1”
Analysts report that as many as 71% of software projects that fail
do so because of poor requirements management, making it the
single biggest reason for project failure…”
CIO Magazine
Great requirements are all about great process – not about the
document itself. If the process for getting the requirements is
poor, it is likely the document will be poor. If the process is
excellent, it is likely the document will be excellent.
Keith Ellis, IAG Consulting
There is an abundance of stories of failed projects. Research has
shown the reasons for IT project failure in the USA, indicated that
project success rates were only 34%, with the rest of projects
being either “challenged” in some way or failing outright.
Sergey Korban, BA Times
© 2015 Cognizant 27
Measuring Requirements Within & Across Projects
• Maintenance as % of IT Budget
• Ambiguities per Requirement
• Ambiguity Heat Map
• Schedule variance due poor or missing requirements
• Project Budget: Estimated vs. Actual
• Volatility due to Scope Changes
• Volatility due to Missing Requirements
• Volatility due to Changes in Requirements
• Volatility due to Technical & Design Changes • Volatility due to Clarifications Required
• Rate of Introduction of Requirements
• Rate of Change Requests
• Rate of Defects with Root Cause as Requirements
• Rate of Post-Launch Defects
• Frequency of Change of Requirements
• Process Design
• Process Compliance
• Tool Compliance
Requirements Metrics
© 2015 Cognizant 28
Ambiguity Review Metrics
Top 10 Ambiguities by Type Total
Ambiguous Term 55
Grammar Spelling & Wording 79
Incomplete Requirements 59
Unclear Requirements 57
Total Ambiguities 250
Patterns within the Requirements Document
© 2015 Cognizant 29
Program Level Ambiguity Review Metrics Patterns across the Requirements Document
© 2015 Cognizant 30
Program Level Ambiguity Review Metrics Patterns across the Requirements Document
© 2015 Cognizant 31
References
Writing High Quality Requirements By Karl E. Wiegers, 2006 The Ambiguity Review Process By Richard Bender Assessing the Impact of Poor Requirements on Companies By Keith Ellis Executive Guide to Evaluating Business Requirements Quality By Keith Ellis Getting Consensus on Business Requirements – Tips and Traps By Keith Ellis The Quest for Good Requirements By Dr. Martin Schedlbauer, 2011 How to Prevent the Negative Impacts of Poor Requirements By Sergey Korban, April 30, 2013 The Business Value of Better Requirements By Karl E. Wiegers, 2006
© 2015 Cognizant 32
References
S.M.A.R.T. Requirements By Mike Mannion, Barry Keepence; Software Engineering Research Group, Napier University, April 1995 Tools for Requirements Gathering Sessions By Seilevel Measuring Requirements May Reinforce Bad Behavior By Jeffrey, March 26, 2012 What Questions Do I Ask During Requirements Elicitation? By Laura Brandenburg Questions for Eliciting User Requirements By Karl Wiegers, December 6, 2011 Information Security Requirements Gathering Questionnaire By Andy Leigh, December 15, 2004 How to use interviews to gather requirements By Ulf Eriksson, October 29 October 2012
Thank you for your time Susan Schanta Director Cognizant Technology Solutions 500 Frank W. Burr Blvd. Teaneck, NJ 07666 (201) 478-0571 [email protected] www.cognizant.com