Ambler IBM Agile Full Story

37
® IBM Software Group © 2006-2007 IBM Corporation Agile Software Development: The Full Story Scott W. Ambler Practice Leader Agile Development [email protected]

description

Agile Story

Transcript of Ambler IBM Agile Full Story

Evolving AgileScott Ambler - Background
Fellow – International Association of Software Architects
www.ibm.com/rational/bios/ambler.html
www.ibm.com/developerworks/blogs/page/ambler
Agenda
Warning!
Warning!
Some may not fit well into your existing environment
Some will challenge your existing notions about software development
Some will confirm your unvoiced suspicions
Don’t make any “career-ending moves”
Be skeptical but open minded
IBM Software Group | Rational software
Agenda
Warning!
Has Your Organization Adopted One or More Agile Techniques?
85% have run multiple agile projects
24% of “No” respondents hope to do Agile this year
Source: Dr Dobb’s 2007 Agile Adoption Survey www.ambysoft.com/surveys/
IBM Software Group | Rational software
% of Successful Agile Projects
(296 co-located, 251 not co-location, 130 offshoring): Agile Adoption Survey
IBM Software Group | Rational software
Largest Team Size Attempted vs. Successful
IBM Software Group | Rational software
Why Agile/Lean? It’s More Successful
Quality: 87.3% believe that delivering high quality is more important than delivering on time and on budget
Scope: 87.3% believe that meeting actual needs of stakeholders is more important than building the system to specification
Money: 79.6% believe that providing the best ROI is more important than delivering under budget
Staff: 75.8% believe that having a healthy workplace is more important than delivering on time and on budget
Schedule: 61.3% believe that delivering when the system is ready to be shipped is more important than delivering on schedule
Source: Dr Dobb’s 2007 Project Success Survey
IBM Software Group | Rational software
Agenda
Warning!
Agile User Experience
Agile Development Practices
True earned value, not documentation-based “earned value”
Daily Stakeholder Interaction
Continuous code integration is nice
Continuous system integration is nicer
IBM Software Group | Rational software
Test First Design (TFD)
www.agiledata.org/essays/tdd.html
With TFD you write a single test and then just enough production code to fulfill that test
Test-Driven Development (TDD) = Refactoring + TFD
TDD is a just-in-time (JIT) specification activity
TDD is a continuous confirmatory validation activity
TDD via Customer/Acceptance Tests

Other Agile Quality Practices
Refactoring
Small change to your code which improves the quality of the design without changing the semantics
Code refactoring
UI refactoring
Database refactoring
Database Refactoring
A database refactoring is a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. Examples: Move Column, Rename Table, and Replace Blob With Table.
A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers.
Important: Database refactorings are a subset of schema transformations, but they do not add functionality.
www.agiledata.org
Working in Priority Order: Agile Change Management
www.agilemodeling.com/essays/agileRequirements.htm
Agile Planning
Create and maintain a high-level Gantt chart indicating the iterations, milestones, and major dependencies
Plan each iteration in detail at the beginning of the iteration
Done by the team, not just the manager
The people best suited to plan the work are the people who are going to do the work
Consider planning poker, www.planningpoker.com
#5 was an iteration task list
#18 was a high-level Gantt chart
#19 (of 19) was a detailed Gantt chart
IBM Software Group | Rational software
Agile Model Driven Development (AMDD)
www.agilemodeling.com/essays/amdd.htm
Do just enough initial envisioning to understand the scope and technical direction
Model storm on a just-in-time basis to gather the details when you need them
While Agile was once considered viable only for small, co-located teams, the improvements in product quality, team efficiency, and on-time delivery have caused larger teams to take a closer look at adopting Agile principles in their environments. In fact, in a recent study conducted by the Agile Journal, it was determined that 88% of companies, many with over 10,000 employees, are using or evaluating Agile practices on their projects. Agile is truly posed to become the dominant software development paradigm.
This trend is also echoed in other industry studies, this one done by our own Scott Ambler, suggesting a 65% adoption rate of Agile techniques.
Agile techniques, such as test-driven development (TDD), continuous integration, iterative development, agile modeling, daily stand-up meetings, and so on are quickly being adopted within large IT organizations
IBM Software Group | Rational software
Agile User Experience (UEX)
Observations:
User interface (UI) and usability issues are critical to the success of most systems
The UI is the system to the end user
Few developers have solid UEX skills, although many think they do
Advice:
Everyone should have some UEX training
Have someone with UEX expertise within your organization, and ensure that they pair regularly
Part of initial envisioning should address UEX issues
UEX issues will need to be addressed throughout development
Recognize that few of us are building the iPod, but when we tread into new territory we may need to do more up-front work than usual
IBM Software Group | Rational software
Agile Documentation Practices
Are treated as a requirement
Have a specific customer and facilitate the work efforts of that customer
Are concise
Describe “good things to know”
Are sufficiently accurate, consistent, and detailed – But aren’t perfect
Value
Effort
Ideal
Realistic
Agenda
Warning!
Scaling TDD’s via Comprehensive Testing
Scaling On-Site Customer/Product Owner
Portfolio Management
Enterprise Architecture
Challenges with Agile in the Mainstream
Agile Development
Minimal
Significant
In the early days of Agile, the applications where Agile development was applied were smaller in scope and relatively straightforward business applications. Today, the picture has changed significantly. As companies want to apply agile development to a broader set of projects. Agile hence needs to adapt to deal with the many business, organization, and technical complexities today’s software development organizations are facing.
Complexity can take on many different forms:
What if the team is much larger? 50 people? 100 people? 1000 people? It can be difficult (if not impossible) to get everyone in the same room on a regular basis to keep information flowing properly.
What happens when the team is distributed, either in different locations or perhaps some of your development force is offshore or outsourced to a third party? Suddenly effective handoffs become more challenging and disconnects are more likely to occur.
If you’re dealing with a more mature or intricate application, or one that must be delivered in different configurations or on multiple platforms. This may require the organization to scale its development operations to accommodate platform-specific requirements or multiple hardware resources.
What if regulatory issues, such as Sarbanes Oxley, are applicable? These issues bring requirements of their own that may be imposed from outside the company which are in addition to the customer-driven product requirements.
The similar issue is true of internal governance mandates. If there are specific reporting requirements to support internal constituencies, then this must be considered as part of the overall development effort to avoid last minute firedrills to collect this information.
And finally, if you’re moving from a more traditional process to a more Agile approach, you’ve got to take into consideration the current climate and define the logical steps for an effective transition.
The good news is that IBM Rational has been helping customers deal with these types of complexity issues for over a decade, and can apply their learnings to help you transition to this complex Agile environment.
IBM Software Group | Rational software
Scaling TDD: Agile Model Driven Development (AMDD) www.agilemodeling.com/essays/amdd.htm
IBM Software Group | Rational software
Scaling TDD: Comprehensive Agile Testing
January 2007 Dr. Dobb’s Magazine (www.ddj.com/dept/debug/196603549)
IBM Software Group | Rational software
Scaling XP’s On-Site Customer and Scrum’s Product Owner
On-site customer is nice, so put them to work
Stakeholders can be active participants in modeling
Product owner is really a communication conduit between the team and stakeholders
Must have agile business analysis skills
PO gets the team access to the relevant stakeholders just in time
Negotiate, negotiate, negotiate
IBM Software Group | Rational software
Database Testing
The Generic Agile Lifecycle
Scaling via Rational Unified Process (RUP)
RUP socialized many of the concepts taken for granted by the Agile community
RUP is really a process framework, not a process
RUP can be as Agile, or non-Agile, as you want to make it
Many organizations struggled to implement RUP effectively
RUP:
Is risk-driven
Contains advice for most of the challenges currently faced by Agile
RUP done right is Agile, RUP done wrong is just plain wrong
IBM Software Group | Rational software
Portfolio Management
Focus on collaboration and enablement, not command and control
Manage enterprise risk
Identify potential projects
Steer existing development projects and programs
Manage services contracts
Monitor projects
Enterprise Architecture
Promote reuse and common infrastructure
Develop reference architectures
www.agiledata.org/essays/ enterpriseArchitecture.html
Enterprise architecture artifacts evolve and are fleshed out over time
Communicate architecture to stakeholders
Agile Data Management
Traditional data management has clearly failed:
Data Warehouse Institute (DWI) estimates data quality to have a $611B annual impact in the US
DDJ found that 62% of organizations have production data quality problems yet the majority have no viable strategy for addressing them
DDJ found that the majority of organizations have no database testing strategy in place, and many haven’t even considered it
DDJ found that over 60% of development teams now go around their organizations’ data groups
This is now the “elephant in the room” for most organizations
A new vision:
Dovetail into enterprise architecture and administration efforts, no longer a silo effort
IBM Software Group | Rational software
Lean Development Governance
Integrated Lifecycle Environment
Valued Corporate Assets
Align HR Policies With IT Values
Align Stakeholder Policies With IT Values
Other industries has gone through a major paradigm shift over the last 50 years. In the 70s, Toyota launched the Toyota Manufacturing Systems which revolutionized how the car industry is run, with concepts such as self-organized teams, continuous improvements (kai-zen), and more agile governance structures, where e.g. any factory worker was allowed to stop production if they say a quality problem. Similarly, Walmart has lead a revolution in the retail industry where you have much more of a collaborative governance model with supplier and buyer working hand in hand to enable an adaptive model enabling inventories to be adjusted rapidly to meet demands and fluctuations caused by hurricanes or holidays. In the same way, modern military has moved from a strict command and control to a structure of steer by objective and self governed teams with accountability. A platoon is given the authority to adapt to reality and find the best possible path to accomplish established objectives. We are starting to see the same paradigm shift in the software industry, and we have captured a set of practices that captures this new approach to governance, or "right-sizing governance".
INTRO TO THIS SLIDE:
Traditional governance is often based on a command-and-control mindset, something that doesn’t work well for intellectual workers.
Traditional governance is often described as trying to herd cats. You can’t herd cats.
To govern Agile projects, and in an effective manner in general, you want to:
Take a collaborative approach with IT professionals
Involve them with decisions, respect their expertise
The key is that governance will likely happen whether you want it to or not. The key is to be PROACTIVE about governance instead of REACTIVE. That way you will have greater influence over how the organization will address these issues.
DETAILED DISCUSSION OF Practices/Patterns:
Pragmatic Governance Body – cost effective techniques, focus on enabling development teams, not controlling them
Staged Program Delivery – roll out the program in chunks / increments. Subprojects must sign up to release date. If subproject misses, it skips to the next release, but this minimizes the impact to customers.
Manage Project Pipeline by Business Value – invest in the projects that make the most sense, with less focus on “cool technology”
Iterative Development – incremental approach to development. Build a system in time-boxes, not as a single-big bang effort
Adapt the Process – one process size does not fit all, tailor the process to meet a team’s exact needs. Processes must be evaluated and evolve over time. Shouldn’t have a “because we’ve always done it that way” mentality
Risk-based Milestones – huge feature of RUP, Mitigate business, technical, … risk early in the lifecycle
Continuous Improvement – Identify and act on lessons learned throughout the project, not just at the end
Embedded Compliance – Build in compliance into your day-to-day process, do not have a separate compliance process which causes unnecessary overhead. Automation is critical
Simple and Relevant Metrics – Automate metrics collection, minimize the number of metrics collected, know why you’re collecting them
Continuous Project Monitoring – Use automated metrics to monitor projects, identify potential issues and work with teams to resolve them early
Integrated Lifecycle Environment – Automate as much of the drudge work as possible, tools and processes should fit together effectively throughout the lifecycle.
Valued Corporate Assets – People should follow guidance, reuse assets, and conform to common infrastructure because these things add value, not because they’re forced to do so. Make it easier to follow existing guidelines rather than creating their own.
Flexible Architectures – SOA, components, objects, patterns, … lead to greater levels of consistency, reuse, enhancability, and adaptability
Self-Organizing Teams – IT professionals are intelligent people who can determine their own strategies for working, for planning their work within established parameters (such as iteration boundaries) and are provided the right coaching.
Align HR Policies with IT Values – Hiring, retaining, and promoting technical staff requires different strategies than non-technical staff. Promote rewards for people to learn new skills. Limit bureaucracy, and put incentives/rewards in place for timely delivery or other key accomplishments.
Align Organization Structure With Architecture – Distributed teams motivate you to have partitioned architectures
IBM Software Group | Rational software
Agenda
Warning!
A Call To Action
Recognize that “the age of hype” is over
Talk about everything that we do, not just the cool/extreme things that we like to talk about
Bring agile concepts to other communities
Their questions will reveal many of the challenges we still face
Invite outsiders into our community
We need more “uncomfortable” keynotes
Police mailing lists a bit better
We turn off a lot of smart people who have something to contribute
IBM Software Group
© 2006-2007 IBM Corporation
Keep In Touch!
Scott W. Ambler
References and Recommended Reading
www.agilealliance.com
www.agilemodeling.com
www.agiledata.org
www.enterpriseunifiedprocess.com
www.ibm.com/rational/agile/
Ambler, S.W. (2002). Agile Modeling: Effective Practices for XP and the UP. New York: John Wiley & Sons.
Ambler, S.W. (2003). Agile Database Techniques. New York: John Wiley & Sons.
Ambler, S.W. (2004). The Object Primer 3rd Edition: AMDD with UML 2. New York: Cambridge University Press.
Ambler, S.W. and Sadalage, P.J. (2006). Refactoring Databases: Evolutionary Database Design. Reading, MA: Addison Wesley Longman, Inc.
Larman, C. (2004). Agile and Iterative Development: A Manager’s Guide. Reading, MA: Addison Wesley
McGovern, J., Ambler, S.W., Stevens, M., Linn, J., Sharan, V., & Jo, E. (2003). The Practical Guide to Enterprise Architecture. Prentice Hall PTR. 
IBM Software Group | Rational software
Agile Values
Agile Principles
www.agilealliance.com
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity--the art of maximizing the amount of work not done--is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
0102030405060
90%+
75-90%
50-74%
25-49%
>25%
Just Barely
Good Enough
Create initial
Communicate
architecture
Add a test
Run the tests
Make a little