Creating & Leading the High Performance Test Organization€¦ · Creating & Leading the High...

Creating & Leading the High Performance Test Organization

Bob Galen

President & Principal Consultant, RGCG, LLC – Leading you down the path of agility…www.rgalen.com [email protected]

v2.3 -- Summer/Fall 2008 Copyright © 2009 RGalen Consulting Group,

LLC 2

Introduction - Bob GalenSomewhere ‘north’ of 20 years experience ☺Various lifecycles – Waterfall variants, RUP, Agile, Chaos, etc.Various domains – SaaS, Medical, Financial, Computer Systems, and TelecommunicationsDeveloper first, then Project Management / Leadership, then Testing‘Pieces’ of Scrum in late 90’s; before Agile was AgileAgility @ Lucent in 2000 – 2001 using Extreme ProgrammingFormally using Scrum since 2000Most recently at ChannelAdvisor as Dir. Product Development and Agile Architect/Coach (2007-2008)Connect w/ me via LinkedIn if you wish…

Bias Disclaimer:Agile is THE BEST Methodology for Software Development…

However, NOT a Silver Bullet!


LLC 3

Outline – ½ Day Version

Part 1 @ 90 minutes1. Effective Communication2. Using Operations Planning

BREAK

Part 2 @ 90 minutes3. Hiring & Performance Management4. Handling the “Hidden” Bits


LLC 4

Introduction

Background on the class, 2 software testing classes –Effective Communication Workshop7 Habits of Effective Leadership Workshop

Emphasis on –LEADING the High-Performance Test OrganizationOperational PlanningTesting Centric Challenges – Team dynamics and value & impact differentiation

It’s OUR class – please participate and share your lessons learned, challenges, approaches, ideas, etc.


LLC 5

Communicating Test “State”

Topics:

1. Keys to effective communication2. What different about test communication?3. Dashboards, status reports & project updates4. Verbal communication effectiveness5. Don’t forget your defect reports


LLC 6

Communications - The ProblemYou’re in the middle of a testing cycle for a business critical project. You’re testing a single component of a large system - roughly 10 testers are on your team. The Vice President of Software development walks up to you in the lab and asks you – “How’s it going?”

What do you say?

He challenges you on several defects that you’ve entered –disagreeing on priority and severity

How do you respond?

This is a great opportunity. You’re either ready for it and respond well or you don’t…which do your choose?


LLC 7

The ProblemSame situation, although time has passed and the project has missed several of it’s planned Beta dates and things are “dicey”. You’re in the middle of the “last” testing cycle prior to going to Beta test. You’ve found some regressions that you “suspect” will impact the products ability to go to Beta. The Vice President of Marketing walks up to you in the lab and asks you – “How’s it going?”

What do you say? How do you say it?

Another, even more critical opportunity to make an impression…


LLC 8

The Point

We’re in communicating situations all of the time

As Test, QA and Process engineers -We’re representing the product, it’s correctness, completeness and overall qualityWe’re representing our test team and ourselvesWe’re the living embodiment of “how is it going?” And “is it ready yet?”

I refer to these ongoing and ever present conversationsas a communications & PR effortIt’s all of our jobs and we do it anyway – so why not learn techniques for doing it often and well?


LLC 9

Introductions Breakout

Break into groups of 2Take a minute or two and introduce yourselves. Share on:

Background information (Overall experience, where you work, etc.)Biggest challenge you face at work Ideas for facing that challenge

I’ll time each exchange

Let’s debrief…


LLC 10

30 Second “Commercial”

In job search circles, they refer to your developing and delivering a “30 second commercial” for networking. It’s a -

Quick introductionConcise overview of your backgroundIncludes your professional historyDelivered to fit the situation, allowed time and specific audience

You take the time to develop your “commercials” from your resume, you should have at least a few – to many of them. They’re targeted towards different audiences and situations.


LLC 11

30 Second Testing “Commercials”

Current work status:What are you working on, what are your recent successes and your challenges. Very importantly - what’s next?Do you need any help? (escalations, ideas, alternatives, workarounds, etc.)If you have one message to send for status – what would it be? Make sure you communicate it!

Current product status:Overall view to your area of testing responsibilityWhat is the overall product stability, feature set maturity and performance?High level defect trends, schedule status and work projections

Always practice your commercials - Preparation is the key!


LLC 12

Characteristics

Keys to Effective CommunicationConcise communications – remember the “Top 1/3” ruleIf you could only say 2-3 things, what would they be?All forms matter – written, verbal, non-verbal, what’s not said, status reports and defects

Target your communicationsTheir functional role and level within the organizationTheir point of view (adopt their POV - empathize)What they want to hear and what they need to hearWhat will they do with the information you give themCan they “handle” the truth and how much of the truth


LLC 13

Breakout

30 Second Commercial, Breakout #1


LLC 14

Dashboards

Major Issues and Risks1) Resources, need 1 usability tester and 1 MWare test resource2) Resource, still running too lean and vacation season approaching3) We had planned for parallel automation development, resource

constraints prevent that. Long term effect to efficiency on regression testing

Release DatesAlpha: 2/1Beta: 3/1Production: 4/1

Team Overall StatusUI functionality, GREEN, ready for UI prototypeUI usability RED, waiting for approval for usability consultantMWare, business logic YELLOW, test case development behind by 2 days Lab & infrastructure GREEN, fully prepared for prototype testingTools GREEN, CM and defect systems ready for releaseAutomation N/A, at this stage, automation not engagedPM TESTING STATUS - GREEN

Key milestones10/1 – UI prototype11/1 – B/E complete, phase 112/1 – UI v2 available2/1 – Alpha ready

Project Name: Snoopy DASHBOARD Reported on: 12-4-2005Test Mgr: Lucy


LLC 15

Quality AssessmentWhat is it? At any point in time you may be asked to assess the current quality of the product under test ---

Formal, written assessmentWith a current, data driven view to statusWith your observations, opinions and insightsFocused towards making a “next step” decision based on “next step” criteria (Phase, Beta, Release or others)

It is the template for the above and one of the primary vehicles for testing PRMake sure you’re always ready and prepared for an assessmentBe true to yourself and your functional responsibility, if it’s not a “Go”, be prepared to recommend a “Stop”


LLC 16

Quality Assessment

Current totals, # open, # high priority/severityTrends supporting the software release plan

Defect Status

Overall # of tests – passed, failed, deferred and blocked. Current coverage.

Issues, challenges and risks

Testing Status

Trends & observations supporting test and project plans

Connection to business requirements, schedule & release criteria

Recommendations

Current release version# of test passes completed so far, # plannedTest dynamics: # of testers, test cases

Overview of Product Status


LLC 17

Quality Assessment

What the assessment is –It is a forum for test to communicate accurate, honest product status from the POV of testing and recommend next steps based on current release goalsIt is intended to present information for the whole team to use in making “next step” decisionsIt is crisp, clear, simple and targeted towards the “receiving”audience

What the assessment is not –It is not 5+ pages of informationIt is not a defect by defect analysis nor exhaustive graphs and chartsIt is not intended as a win-lose situation (test “win” or test “lose”)


LLC 18

Defects – Windows into your Work

Communication starts at the individual defect levelEvery defect is a window into your work. What will your team read and how will they perceive it?Beyond the “raw” data, consider –

Your audience – who will be reading the defects, what do they need to seeInsure that you explain things simply, thoroughly and clearlyDon’t speculate too much, keep it data driven and factualDon’t let your feelings creep inIt should “connect” the problem to requirements - customer -business impact

Every one matters!


LLC 19

Defects – Windows into your Work

5 heuristics for crafting a solid defect “message” -

1. Target the “core level” of your audience (high technology startup, vs. a telecommunications giant vs. OS provider)

2. Clear, concise and meaningful description - headline3. Include steps, time and level of difficulty to reproduce4. Clearly identify Business & Customer impact5. Short, with most of the important information in “front”

Sometimes it’s useful to review historically “good” & “bad” defects to establish an expectation baseline


LLC 20

Defects – General Guidelines

According to Kaner, Falk & Nguyen –

A good report is written, numbered, simple, understandable, reproducible, legible and non-judgmental

Simple: single defect, focused problemUnderstandable: context to the product, domain and development staff (terms, technology, experience), connected to the audienceReproducible: details, directions, ease setup, include supporting documentationLegible: clear and precise prose, time taken to do it rightNon-judgmental: just the facts, no “attacks”


LLC 21

Defects – General Guidelines

According to Kelly Whitmill –1. Condense – Say it clearly but briefly2. Accurate – Is it truly a defect?3. Neutralize – Just the facts4. Precise – Explicitly, what is the problem?5. Isolate – What has been done to isolate the problem?6. Generalize – How general is the problem?7. Re-create – essential environment, steps, conditions8. Impact – To the customer, to testing, safety?9. Debug – Debugging materials (logs, traces, dumps,

environment, etc.)10.Evidence – Other documentation proving existence


LLC 22

Using the Operations Planning Process

Topics:

1. Vision & Mission Setting2. Operations Planning

1. Annual review2. Project workflow planning3. Budgeting4. Team development


LLC 23

Setting the Stage

VisionA vision is a statement about

what your organization wants to become

It should resonate with all members of the organization and help them feel proud, excited, and part of something much bigger than themselvesA vision should stretch the organization’s capabilities and image of itself.It gives shape and direction to the organization’s future

MissionMission or Purpose is a

precise description of what an organization

doesIt should describe the business the organization is inIt is a definition of “why” the organization currently existsEach member of an organization should be able to verbally express this mission


LLC 24

Setting the Stage

Vision is a critical characteristic for a good leader. Looking ahead; anticipating & preparing for the futureIdentifying opportunitiesIdentifying areas requiring change; disruptive and game changing

The eternal dilemma is keeping your eye on the ball of today (Mission), while playing on the field of tomorrow

(Vision).--- Galen


LLC 25

Starting Projects

Team organizational structureClear roles & responsibilitiesNecessary skillsComplementary fit (experience levels, attitude)

CharteringDefining requirements & scopeDesign, architecture, tools, approachesGetting to the true purpose of the effort. Specifically define what success will look like.

Establish working patternsConstruction, progress tracking, re-workProcesses and teamwork

Enroll your team to meet the goals


LLC 26

Using Collaborative Workshops

Card based or stick note planning workshopsJAD session likeWideband Delphi, XP card planning / planning poker, SNP, others…Adjust the “level” to the project state, goals & expectations

Get the team togetherAnalyze the project landscapeBrainstorm

Tasks, issues & risks, workflow & scheduleDate alignment planning, front-to-back-to-front passes

Great for gaining team buy-in & commitment


LLC 27

Operations Planning

Operations PlanningPerformed on an annual basis, include team retrospectives, 12-18 business views (guess if you have to)Annual performance review and goal setting at the team levelShort term strategy (projects, initiatives)Team staffing (core competency, gaps and training plans)Budget

Primary purposeInternal planning & group communicationExternal exposure for support plans and gaps / needs


LLC 28

Ops – Group Overview

Setting the stage for your teamFunctional GroupPeriod Covered by the Ops PlanMission & VisionDifferentiationBudget

I like to emphasize the differentiation aspect. How will the organization make a significant and visible impact to the overall organization?


LLC 29

Test Team Differentiation

ST&P November 2005 Future Testing column -Moving Beyond Commodity Testing

1. Deep Product KnowledgeBreadth of product experience, real-world usage, priority

2. Broad-View CommentaryTechnology vs. Business vs. Quality vs. Customer trade-offs

3. Cycle Time ReductionSpeed: automation, risk-based testing, just in time & just enough

4. Outsource ThinkingBalanced recommendations & assistance on best ways to outsource for quality / speed advantage


LLC 30

Ops – Projects

Project Initiatives, for each project analyze:OverviewResource requirements (human & equipment)High level budget (equipment, tools, recruiting, training, etc.)Core competency “gap” analysisResource application / assignment curvesExternal dependenciesChallenges


LLC 31

Ops – Team Development

Team DynamicsOrganization chartLeadership chartKey roles & responsibilitiesAnticipated changesGroup growth & promotion planningExternal relationships

Group TrainingCore competenciesGroup evaluationTraining plan – group & new hire


LLC 32

Break and Breakout

Operations Planning, Breakout #2


LLC 33

Hiring & Performance Management

Topics:

1. Setting up an effective hiring process2. Group interviewing techniques3. Making sound decisions4. Creating high impact teams – complementary skills5. Coaching for performance – effective feedback,

mentoring & coaching skills


LLC 34

Setting up an effective hiring processIt starts with a commitment to building your team

Assess your gaps & needs from a current / future perspectiveDefine clear goals for your needsWrite a solid job description; review it with your team

Composing a team takes creative, now vs. future thinking

Don’t be afraid to be disruptive, to hire someone “different”

Behavior, skill, culture, attitude, experience, etc.

Phone screens are not an interview. Instead theyDetermine domain match, technical match, and team fit at a HIGH levelThe phone screener decides whether to bring someone in


LLC 35

Setting up an effective hiring process

Craft a unique interview for each positionAlways phone screen (notion of an “on-site” screen)Plan the interview (agenda, focus points, teaming)Define clear roles and responsibilities

After the interview:Discuss direct & indirect (observed) responsesDebrief as a team, try to achieve consensus, or at least no firmopposition (no thumbs down)Always go with your “gut”Should be excited about the “possibilities”!


LLC 36

Group interviewing techniques

Multiple group interviewsGroup has a specific focus – technology, skill area, team fitHave a question asking protocolObservers (answer + body language)Caution: prepare the interviewee for it, no more than 2-3 interviewers

AuditioningPrepare a “piece of work” – code, architecture & design, test Present it in front of a group – 30 minute presentation, Q&ALooking at the work AND the presentation abilities

Guerrilla Guide to Interviewing – Joel on Softwarehttp://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html


LLC 37

Group interviewing techniques

Information sharingDemo products that they’ll be testingShare company & domain information; on culture & expectationsLab tours & office tours

Not so great practicesToo many interviewers, lunch interviews, no breaksOver use of testing or scenario based interviewingPoor group combinations, little planning, interrogation, it’s all about you

Don’t forget the lost art of the thoughtful, targeted open-ended question

Active listening and the effectiveness of silence


LLC 38

Making sound decisions

Decide as a groupOpen with reminder of position description and key goalsRoundtable discussion, pro/con, differentiatorsDrive to consensus – Yes, No, Neutral/Support

Decisions go both waysEnsure the candidate has sufficient information to make a decision (60:40 information transfer)Share key positions challenges or expectations with the candidate, be passionate; see how they respondGive them sufficient time – it’s a BIG decision


LLC 39

Composing Your Teams

Complimentary skillsBroad skills – for now and into your future

Technical – in the large and smallDon’t forget soft skills (teaming, presentation, communication)

With natural, inquisitive, quick learnersConsider passion for the Profession of TestingOne size does not fit all – be disruptiveDo certifications matter or help?Always, always hire the BEST – Top 20% caliber peopleHire your replacementCultural fit is important too!


LLC 40

Good to Great – Excerpt #1

The Good-to-Great leaders began the transformation by first getting the right people on the bus (and the

wrong people off the bus). And then figuring out where to drive the bus.

Who should always come before what (vision, strategy, organization structure, and tactics)

-- Good to Great, Jim Collins


LLC 41

First Impressions

First ImpressionsAre critical - it takes more time to “recover” than you thinkIt takes time to build trust

Notion of “On-boarding” new employeesEnroll them as part of the interview processWork area preparations and introductionsSetting expectations and effective mentoringWeekly 1:1 meetings and 30 day goal setting


LLC 42

Coaching for Performance

What do I mean by Coaching?Performance management, timely and honest feedbackFinding and receiving feedbackManaging group performanceMotivating your team

Why so hard?Because it deals with PEOPLEPeople aren’t neat, tidy, one sized, or computer programsSo easily avoided or handled too lightly or heavilyFocused forward – towards improvementYou get little short-term recognition for your efforts


LLC 43

Constructive Feedback

Face the ChallengesDon’t look for divine intervention100% of the time, it doesn’t improve on it’s ownMore people recognize it than you realizeIt takes time for a direction to form, so don’t wait

Learn to give feedbackAt the point of attack and continuouslyTo all members of your teamBe honest and constructive, say what’s on your mind, and ensure it’s clearBe congruent across your team


LLC 44

Feedback – The 95% “Gap”

As a “Team Repair” consultant I notice that:Leaders often share strong opinions related to team & individualperformance issuesBut when I look for direct, clearly communicated evidence in performance appraisals I find…5% or less of what I “heard”

As a Leadership Skills instructor, I notice that:When asked, about 95% of the IT Leaders feel that providing detailed, precise, real-time, clear feedback is clearly a strength of theirs

So why the gap?


LLC 45

Receiving as Well as Giving

Listen to your team…truly listenGroup 360 degree communications

Learn to receive feedbackIt’s a gift from others, truly welcome and cherish feedbackIt isn’t good or bad, it simply isMy strategy is to gather it first, then digest it, then look for the “truth” in it, then work to adjust If you don’t receive it well, you won’t receive it often. Be receptive!It isn’t just what’s been said, it’s what hasn’t been said. Consider body language and other clues


LLC 46

Managing Group Performance

Consistent principles, but tailor to individuals

Communicate your “soft side” performance expectationsEnergy, enthusiasm, learning, teamwork, stretching – over hoursResults – over hoursAttitude and effort – over hours

Deciding who gets your time?Top down, not bottom upStart with your top 20% and high potential employees, then focuson the remainder – First Break All The Rules, Buckingham & CoffmanPareto Principal applies here as well


LLC 47

5 Dysfunctions of a Team -- Lencioni

Absence of

Trust

Fear of

Conflict

Lack of

Commitment

Avoidance of

Accountability

Inattention to

Results


LLC 48

Good to Great – Excerpt #2

The Good-to-Great leaders were rigorous, not ruthless, in people decisions. They did not rely on layoffs or restructuring as a primary strategy for improving

performance.

Three practical disciplines for being rigorous in people decisions:

1. When in doubt don’t hire – keep looking.2. When you know you need to make a people changes – act!3. Put your best people on your biggest opportunities, not your

biggest problems.

-- Good to Great, Jim Collins


LLC 49

Motivating Your Team

Employee’s naturally “connect” to their 1’st level manager

In survey’s, approximately 80% say that it’s the most important thing keeping them on the job – certainly in the Top 3!

Show themYou have a mission, you’re invested and excited about the possibilities!You understand the domain and technologyYou understand the challengesYou understand & care for themBe yourself, be open & honest


LLC 50


Apply effective empowerment and delegation principlesRealize you can’t do it all – so trust your team!Work your top 20%Delegate even when you realize the results won’t be “perfect”Allow the organization to learn and, yes I’m saying it, make mistakes

Allow decisions to made without your approval

Create alternate decision-making modelsConflict can be very good. It generates ideas, alternatives, passion, buy-in, energy, teamworkCreate healthy competition


LLC 51


Stephen M. R. Covey’s book – The Speed of

Trust

13 Behaviors that Foster & Increase Trust

1. Talk Straight2. Demonstrate Respect3. Create Transparency4. Right Wrongs

5. Show Loyalty6. Deliver Results7. Get Better8. Confront Reality9. Clarify Expectations10. Practice Accountability11. Listen First12. Keep Commitments13. Extend Trust


LLC 52


Recognize individual successTake the time and effort to recognize those who are supporting the visionSometimes it’s a simple thank youOther times, peer and public recognition can be wonderfulIt’s rarely about the moneyDon’t forget their families and the more personal the recognition the betterAlso recognize those who are “stretching” the most –remembering that effort counts


LLC 53


Communicate a compelling vision to where you’re goingStrategic longer term vision (1 year and beyond)Tactical, laser focus (steps to getting there)And repeat…

Recognize your successes and build on themRecognize those that are supporting your visionTake time to celebrate success – even if it’s an email, meeting, day off, ice cream social, etc.Share “stories” of success. What are the proud examples from the teams’ past?


LLC 54

Break and Breakout

Effective Hiring, Breakout #3


LLC 55

Handling the “Hidden” Bits

Topic:

1. Making the hard call – You’re Fired2. Other options for performance issues and problems3. How to manage group compensation4. Handling confidentiality5. Handling politics


LLC 56

Making the Hard Call

You’re firedLet the process take care of itselfHR leads this effort, particularly legal reviewShort meeting, less than 15 minutes, use a “script”Leave first, pack laterNotifying the team

It will change things though…Within your team, their perceptions, your own views

Other options?Transfers, promotions, restructuring responsibilities

Other words of advice or experience from the group?


LLC 57

Managing Group Compensation

Multiple tiers to compensationSalary + raises, bonus, promotionsYou can only do so much – HR guidelines & limits

Create a compensation strategy Reward (define first) high performance (peanut butter vs. zero)Messages to the teamGap analysis as you bring in new hires – equity strategy

Remember – people share salary levels, don’t expect much confidentiality


LLC 58

Proactively Manage Up

Where is your loyalty – Team? Management?Actually, it’s a bit of both

Managing perceptionsThey’re everywhere! Are they reality? Sometimes…First look to understand your perceptions - 360 degree feedback, mentors, leadership inquiriesUnderstanding your technology context and business domain is paramountBeing open-minded and flexible is next


LLC 59


Effectively handling schedule pressure

Don’t simply agree. If pushed for a reply, buy whatever time you can. Remember, to some degree, it’s a game of chickenRAD workshops are ideal forums for pulling together estimates, schedules and optionsAlways provide estimates in ranges and provide levels or areas of risk, then commit to more accuracy over time and ITERATEDon’t over commit based on overtime assumptionsAlways speak in terms of options and trade-offs


LLC 60

Iterative Estimation Source: Larman’sAgile & Iterative Development, Page 18


LLC 61


Be prepared to say – No, or Yes, but…Technologist have a natural tendency to over-commit. To over negotiate to over simplify.Always the “Can Do” attitude is rewarded, while “Push Back” is not

Respond with your real thoughtsIf’ it looks like too great a challenge – say why. Don’t be afraid to say what’s on your mind – be honest and emoteBe prepared to be a lone voice. Saying “No” takes a great deal of courage

Career risk and trust factors come into playUse it ONLY when necessary, don’t become a naysayerRemember - it does get their attention


LLC 62


If you are going to say No or Yes, but…

Provide some context – Explain why it’s not possible, the limitations, the risks, the possible business outcomesProvide some options – Explain what might be possible, trade-offs, if only…Provide some decision criteria – Provide details (time, resources, $$$) on critical decision criteria

Remember, they can “Handle the Truth” or at the very least need to be “Hearing the Truth”


LLC 63


Buffering 101

You’re there to help, notion of servant leadershipIn the Scrum Methodology, the Scrum Master is there to remove impediments to progress. They’re also responsible for “buffering”the team from unnecessary interruptions and activity.Expose the success and power of the organization, but don’t let the organization derail your focusAlways keep your team focused on - Core functions, priorities and the “next” two weeks


LLC 64

Provide Compelling Leadership

Handling ambiguity

Not everything will be clear or known. Learn to deal with thatA decision made immediately with 70% clarity is better than a decision made later with 100% clarity

Failure is part of the game of ambiguity…Fail small though!Guide your teams through it with confidence – mapping next steps clearly and precisely to your vision, mission, purpose andgoals


LLC 65


Dealing with change

You can’t simply react to change or ignore it. You must lead your teams through itGrasp the changeMap it to your function and roleLook for the impacts and requisite adjustmentsMake the strategy and journey visible within the teamBe cognizant of the change adoption curve and be patient

Remember, change has a counterpart called complacency. Don’t change for it’s own sake, but

do change.


LLC 66


Balance is the Key!Tactical vs. Strategic Now vs. FutureDelivery vs. MaintainabilityInitial cost vs. Long term cost

Weighed against your point of view – short term vs. long term

You at least have to appear to be interested in the long term – or maintain a “Longer Term Mindset”Shorter job tenures equate to a greater short term focus


LLC 67

Politics

Decide on the landscape in your organizationExplore & identify the requisite activity level & power playersDetermine what matters – role, function, perception, skill levelsYour acumen matters to you AND your team

Always manage to your central boundariesCharacter, ethics, comfort zone – staying true to yourselfApply the Golden Rule – treating others how you would like to be treatedBe honest and tell it like it isIt’s about the results!


LLC 68

Thank you for taking the time!

Any questions?


LLC 69

Contact Info

Robert GalenRGalen Consulting Group, L.L.C.

PO Box 865, Cary, NC 27512919-272-0719

[email protected]

Software Endgames: Eliminating Defects, Controlling Change, and the Countdown to On-Time Deliverypublished by Dorset House in Spring 2005. www.rgalen.com for order info, misc. related presentations, and papers.


LLC 70

ReferencesAltier, William, “The Thinking Manager’s Toolbox – Effective Processes for Problem Solving & Decision Making”, Oxford University Press, (1999)Brantley, Mary Ellen and Coleman, Chris, “Winning the Technology Talent War – A Manager’s Guide to Recruiting and Retaining Tech Workers in a Dot-Com World”, McGraw-Hill, (2001)Buckingham, Marcus, Coffman, Curt, “First Break all of the Rules – What the Worlds Greatest Managers do Differently”, Simon & Schuster, (1999)Collins, Jim, “Good to Great – Why Some Companies Make the Leap and Others Don’t”, Harper Business, (2001)Covey, Stephen, “The Seven Habits of Highly Effective People”, Simon & Schuster, (1989)Covey, Stephen, “The 8th Habit – From Effectiveness to Greatness”, Free Press, (2004) Covey, Stephen M. R., “The Speed of Trust – The One Thing That Changes Everything”, Free Press, (2006)DeMarco, Tom, "The Deadline - A Novel About Project Management", Dorset House Publishing, (1997) DeMarco, Tom, “Slack – Getting Past Burnout, Busywork, and the Myth of Total Efficiency”, Broadway Books, (2001)DeMarco, Tom and Lister, Timothy, "Peopleware - Productive Projects and Teams", Dorset House Publishing, (1999, 1987)


LLC 71

ReferencesFisher, Roger and Ury, William, “Getting to Yes – Negotiating Agreement Without Giving In”, Penguin, (1981) Haneberg, Lisa, “High Impact Middle Management – Solutions for Today’s Busy Managers”, Adams Media, (2005)Hohmann, Luke, "Journey of the Software Professional - A Sociology of Software Development", Prentice Hall, (1997) Kerth, Norman, “Project Retrospectives – A Handbook for Team Reviews”, Dorset House Publishing, (2001)Lencioni, Patrick,

“The Five Temptations of a CEO – A Leadership Fable”, Jossy Bass, (1998) “The Four Obsessions of an Extraordinary Executive”, Jossy Bass, (2000)“The Five Dysfunctions of a Team – A Leadership Fable”, Jossy Bass, (2002)“Silos, Politics and Turf Wars – A Leadership Fable”, “Jossy Bass, (2006)

Manns, Mary Lynn, Rising, Linda, “Fearless Change – Patterns for Introducing New Ideas”, Addison Wesley, (2005)Maxwell, John, “Failing Forward – Turning Mistakes Into Stepping Stones for Success”, Thomas Nelson Publishers, (2000)Patterson, Kerry; Grenny, Joseph; McMillan, Ron, and Switzler, Al, Crucial Conversations, McGraw-Hill, (2002)


LLC 72

ReferencesPatterson, Kerry; Grenny, Joseph; McMillan, Ron, and Switzler, Al, Crucial Confrontations, McGraw-Hill, (2005)Rothman, Johanna, “Hiring the Best Knowledge Workers, Techies & Nerds", Dorset House Publishing, (2004) Rothman, Johanna & Derby, Esther, “Behind Closed Doors – Secrets of Great Management”, The Pragmatic Programmers, (2005)Tabaka, Jean, “Collaboration Explained – Facilitation Skills for Software Project Leaders”, Addison Wesley, (2006)Weinberg, Gerald M., "Quality Software Management - Congruent Action", Dorset House Publishing, (1994)Weinberg, Gerald M., "Quality Software Management – Anticipating Change", Dorset House Publishing, (1997)Welch, Jack, “Winning”, Harper Business (2005)Whitney, Diana and Trosten-Bloom, Amanda, “The Power of Appreciative Inquiry – A Practical Guide to Positive Change”, Berrett-Koehler Publishers, (2003)


LLC 73

www.rgalen.com Web References

7 Habits of Highly Effective IT Managers -http://www.rgalen.com/t_files/SevenHabitsOfHighlyEffectiveManagersShort.ppsCommunicating “state” –http://www.rgalen.com/t_files/SoftwareTestingAndPR.pps, QA Assessment - http://www.rgalen.com/t_files/ProductAssessmentTemplate.docProject Chartering –http://www.rgalen.com/t_files/ProjectChartering.pps, http://www.rgalen.com/t_files/CharteringExampleScenarios.docCoaching –http://www.rgalen.com/t_files/Coaching.ppsCard based planning (WideBand Delphi) -http://www.rgalen.com/t_files/SoftwareWBandNoteEstimating.ppsOps Plan –http://www.rgalen.com/t_files/OperationsPlanTemplate.docSuccessful projects -http://www.rgalen.com/t_files/ProjectManagement10KeysOriginal.pps, http://www.rgalen.com/t_files/PM_10KeysPaper.doc


LLC 74

General Web Referenceswww.appreciative-inquiry.org – AI centered sitehttp://appreciativeinquiry.cwru.edu/ - AI Commonswww.apln.org – Agile Project Leadershipwww.ccl.org – Center for Creative Leadershipwww.dorsethouse.com – Dorset House publishing. Wonderful technology leadership resourcewww.greenleaf.org – Greenleaf Center for Servant-Leadershipwww.injoy.com – John Maxwell’s leadership websitewww.joelonsoftware.com/articles/GuerrillaInterviewing3.html - Interviewing: Joel on Softwarewww.retrospectives.com – Norm Kerth’s sitewww.stickyminds.com – Software collaboration site – testing resource and www.bettersoftware.comwww.stpmag.com – Software Testing & Performance magazine (free!)www.testingreflections.com – Testing blog (many contributors)

www.testingreflections.com/node/view/928, Testing a stapler examplewww.tablegroup.com – Patrick Lencioni’s websitehttp://www.nwlink.com/~donclark/leader/leadcom.html - communication info

Creating the High-Performance Test Organization – Breakout Examples

Fall 2008 Copyright © 2001-2008, RGCG, LLC 1

30 Second Commercial – Breakout (15 minutes)

• Break out into groups of 2

• One of you assume the role of the tester and the other of the project manager

• Have a 3 minute conversation regarding test status and overall product quality

• Now switch roles, the project manager remains the same and the tester becomes the VP of Marketing – checking up on status

• Again, have a 3 minute conversation regarding viability of Beta testing and next

steps.

• Note: Really get into your roles!

Breakout Situation You are testing a brand new web HR application for your company IT team. You’re not alone, there are three more members of the team involved in the effort and you’re the team leader. The project has been in test for about 3-4 weeks. By this time, the code should have been “frozen”, but development was late in delivering and there is still some code coming in - “yet another” release scheduled for tomorrow. The schedule says that Beta is to begin in exactly two weeks. However, you believe this is wishful thinking (at best) and terminal denial (at worst). There are 25 severity 1 defects open and being worked by the development team at the moment. And the trending on severity 0-2 is rising. Speaking of the development team, there are only 10 of them and they appear to be very overworked. In fact, you’ve seen the quality and productivity of their work significantly decline over the past few weeks. Sam, the lead database developer is particular prone to errors lately. In fact, you’re working on five of his components now and you’re troubled by 5 new severity 1 defects AND several nagging regressions. Something should be done about this. The VP of Development and QA just assigned a new project manager to the team to “get things in order”. She has scheduled some time with you this afternoon to get acquainted and ascertain status. How do you prepare for the meeting? And what do you plan to say? Not to say? How will you go about preparing?



Quality Assessment – Breakout (30 minutes) Break out into groups of 2-4. Using the project scenario below, you need to construct a Quality Assessment as a team. The target audience in this case is the Project Release Team, composed of all functional (Marketing, Development, Quality, Documentation, Customer Support, Manufacturing) senior managers who looking to make a release decision.

Project high level view • HR management applications

o Personnel management (recruiting, basic information) o Benefit management o Payroll o Training and development

• Web based system (3 tiers) • Multiple browsers supported • Internet and intranet based access • Secure system • 15,000 users • Distributed across 4 primary sites

Project development view • 500 individual screens, using XP stories for requirements • Primary languages: html, java, and ruby • Using several 3’rd party java components • Oracle primary, SQL server secondary database • Using ADP for payroll processing, several API interfaces used to dialogue with

ADP databases and exchange data • Relatively small, 7 engineers, and inexperienced development team • They continue to experiment with different development methods, XP being the

latest • There seems to be very little “customer” input or involvement in the effort

Project testing status • We’ve been testing the application for 3 months, with 2 testers • Through about 6 testing cycles • The code set froze right before the last testing cycle. Prior to that, it was about

50% complete • We have had an increasing view to open / new defects. We found 156 new defects

in the last release



• Priority 0-1 defects have been relatively stable for each release, running at about 20-30 defects in each

• There was a critical set of defects in the payroll interface subsystem in the last release and we couldn’t test that functionality

• In fact, we have never closed the loop on payroll testing • The java reporting applet has several severe problems and the vendor has been

non responsive in making repairs • Performance testing will only seriously commence next week. To-date, we’ve

only tested with about 200 clients attached to the application

Late Breaking News • Management has indicated that the project is taking too long. They’ve decided to

add 4 more developers to the team and 1 more tester. • You are two days into your most recent release, of a 2 week test cycle. • They want to schedule a Beta for next week and are looking for a quality

assessment as a check for readiness

• All “eyes” look to the test team… ☺ On a flip chart, capture the key points your will raise in your Quality Assessment. Pick a spokesperson to review the conversations and your assessment and we’ll debrief as a group.



Operations Plan – Breakout (30 minutes)

• Break out into groups of 2-4 • Read through the Operations Plan template • As a group, discuss:

o What parts do you think would be most useful for your organization or

group context? o Least useful? o What parts would be too difficult to implement? And why? o What would be the inhibitors to you’re trying to implement Ops Planning

on an annual basis? Discuss ways to overcome them. Be prepared to share your conversation notes and we’ll debrief as a group



Effective Hiring – Breakout (20 minutes)

Option A

• Break out into groups of 2-4 • One side of the room discusses and characterizes the 2-3 BEST interviewing and hiring

practices that they’ve ever experienced or been a part of. o For each, define what made them great – characteristics and examples

• Other side if the room discusses and characterizes the 2-3 WORST interviewing and

hiring practices that they’ve ever experienced or been a part of. o For each, define what made it terrible – characteristics and examples

• Select a spokesperson who will represent your experiences

Debrief as a group – comparing the two points-of-view, collecting a list of Do’s and Don’ts

Option B Individually, reflect on the best hiring decision you’ve ever been a part of. Either from an interviewee, hiring manager or interview team participant point-of-view. Answer the following questions –

1) Describe the interview process

2) What best practices stand out in your mind?

3) What single interview step or practice – led to the success of the hiring decision? Debrief as a group – collecting a set of “Best Practices”



Executive Negotiation – Breakout (30+ minutes) Project Assignment As of today, Friday, December 13, you’ve been selected to be part of a team for a complex project with a compressed schedule. Although the general nature of what is wanted is known, the specific functionality that will implement it will have to be determined as the project progresses. You have a meeting with Mr. Trump on Monday morning to discuss any last minute risks around his upcoming announcement. This is the last chance you’ll get to talk with him prior to the news conference, as he’s going on a 3 week vacation to Australia with his new bride.

Background TrumpTicket is the show ticket purchasing arm of Trump Enterprises – yes, that Trump. It handles the show ticketing for all of the Trump Entertainment venues and has been in operation for 6 years. The systems was one of the first web ticketing sites ever made available and has proven to be quite successful in supporting the limited ticketing requirements of Trump Entertainment. However, the system has begun to develop maintenance issues. It is increasingly more difficult to repair or enhance. The team has also has been struggling with an archaic design that seems to resist scaling and, simply put, it’s clearly showing its age. From a business point of view though, TrumpTicket is booming. To the point of recently becoming the ticket purchasing focal point for musical concert venues and Broadway productions in a series of Trump Entertainment acquisitions. The total of these acquisitions is going to drive a 10x short term and 20x longer term growth requirement over current TrumpTicket capabilities. Mr. Trump has already gone on record saying that he could easily expand his organizational and IT capabilities to accommodate the acquisition targets. In fact, that was a fundamental part of the business case and justification for the acquisitions.

Project The TrumpTicket ticket portal will need to be upgraded to handle the abrupt increase of functional ticket purchasing and load placed on it by the acquisitions. Mr. Trump himself will be announcing the new TrumpTicket at a news conference on January 15. He hopes for some functionality to be available by March 30, and for the site to be fully functional by April 30, with performance verifications completed by May 30. The anticipated release schedule is:

1. March 30 – Functional pilot available for a limited set of 1000 users. 2. April 30 – Same as March 30 release except full-functioning capability present,

perhaps with (some) performance limitations. 3. May 30 – Fully functioning site performance.

Funds for the project are ample and should not be considered an unreasonable constraint. The date and functionality are the deliverables. Facilities or packaged software to support TrumpTicket can be either bought or developed, whichever supports meeting the date. However,



partnering or any agreement needing extensive legal review can probably not be accomplished during the time window.

Functional Requirements 1. Virtual functional refactoring of all existing TrumpTicket functionality towards a new

architecture and design. 2. There will be a pilot program to shake out the similarity kinks between the old and new

interfaces of 30 days conducted with a set of 1000 extremely familiar and valuable customers.

3. There is an outstanding list of 243 defects that need to be addressed in the refactoring. 4. There is also an outstanding set of 57 enhancement requests that need to be rationalized

as well.

Nonfunctional Requirements 1. 250,000 simultaneous users with sub-second response times. (previous system supported

25,000 simultaneous users) 2. Secure for the level of financial activity envisioned (25,000 tickets per day at an average

price of $75). (previous system supported 5,000 tickets per day) 3. Scalable to 1,000,000 simultaneous users as needed. 4. 99% availability 24x7.

Development Context 1. A development environment for building products is ready for you. 2. Websphere has been chosen as the new architectural / deployment environment.

However, the broad team has little knowledge of it. 3. There are currently cubicles in the development site. The development environment is

wireless and has all power and networking capabilities already operating. 4. You are required to use a source code library, check in code every time it’s changed, built

the software at least daily, and unit and acceptance test the software every time that it is built.

5. Extreme Programming will be used as the development practice. All other engineering practices, such as coding standards, are up to the team.

6. All of the developers have excellent engineering skills, but they have only used XP sparsely so far.

7. The team consists of all development engineers with excellent design and coding skills. However, they are still responsible for all testing and user documentation. They may acquire contractors to assist with this. The engineers on the team average 10 years of progressive experience on software projects using complex technology. The team is familiar with building products using the identified development and target environments.

8. A QA environment already exists. However, there are no adequate testing tools, continuous build tools, refactoring tools, and CVS is perhaps not adequate for the job.

Group __________ Operations Plan for the period __________ Author __________


Operations Plan Template v2.1 The basic perspective of the operations plan is that of a functional manager over their functional grouping or team. The basic purpose of the operations plan is to identify the key focus points for your organization over a longer term period. It is a vehicle to first, guide your efforts to evolve your group to meet external requirements and second, to communicate the vision and direction to your team. Typically, it takes a 12 – 18 month view towards the future. There is an organizational structure (ORG) component, a project focus (WORK) component and a training and development (TRAIN) component associated with the plan. Another way of thinking about it is as a yearly performance plan or review for your entire team.

Construction Strategy Step 1 -- Vision & Mission are usually the first step. I personally don’t like to spend too much time on it at the beginning. Instead, I’ll pull together a “strawman” proposal for the team. Then I’ll gather feedback and evolve the vision and mission in real-time. Step 2 -- The next pre-work item is usually getting a good feel for upcoming project activity. This can sometimes be quite challenging, particularly if you looking for a one year plus forward view. My suggestion is to get whatever product roadmaps you can find from your customer and marketing views and extrapolate your role in supporting them. While you’ll never get a complete view, you can usually do a reasonable job of pulling together a draft. Then do you best to fill in the gaps and adjust it as the future becomes clearer. It’s also important during this stage to think of differentiators. How can you leverage your functional role and group strengths to impact the overall business objectives in a very visible and creative way! Step 3 -- I personally find the final organizational steps the most interesting. Usually I start by defining a core skill set for my team, both from the point of view of current skills and future skills. I look to differentiate the skills into two primary categories – technical vs. leadership & intangibles. Step 4 – Once I get a draft Ops Plan together I like to bounce it off of my team, first off of the leadership team and then the entire team. Not only am I looking for ideas to develop and new gaps, but I’m looking to make it a shared plan moving forward. On final point, there are usually some confidential parts of the Ops Plan that really aren’t appropriate to share with your team at-large. That implies two representations of the Ops Plan, one for the team version and the other for leadership, be created and maintained.



Section One – Overview

1.1 Functional Group Name the functional group. List its purpose and primary focus. Also the types of work the group is targeted towards, rough group size and overall levels of experience. Give the audience a thumb nail sketch of the overall group.

1.2 Period Covered by the Ops Plan Identify the period of time you are focusing the plan towards, typically one to two years. It’s probably a good idea to synchronize the ops plan with your annual review process.

1.3 Mission & Vision Take some time and put some thought into crafting a mission and vision statement for the team. Typically, vision is longer term in view and less liable to change based on changes in the environment. Mission on the other hand, is more discrete and short term in view – leading towards accomplishing a step or series of steps in support of the vision.

1.4 Differentiation This section is” linked” to your mission and vision. It identifies key areas where you plan to “stretch” your organization to better meet your support of business goals. The term differentiation implies a focus on areas that are new, unexplored opportunities and gaps that you can fill. It requires some strategic thinking and a broader view to identify these areas. It’s the way you morph your team to adapt to changing business, product, and technical opportunities.

1.5 Budget This section contains all budget information outside of the training budget. It doesn’t contain excruciating detail, instead, listing high level personnel, equipment and tools budgets for the period. It should highlight budget increases or exceptional extensions. It’s also useful to differentiate into meaningful areas, for example – lab, hardware tools, software tools, infrastructure servers, and reoccurring software license renewals.



Section Two – Project Focus

2.1 Major Project Initiatives Chronologically list all project initiatives in the covered period of time. For each project, speak to:

□ Single paragraph project overview (Name, core features, core focus, business need, etc.)

□ Approximate project resource requirements (human & equipment)

□ Does the project map to your core competencies? If not, what are the higher level “gaps”?

□ Estimates for duration for the effort – both from a business / program (magical)

perspective and from an internal (reality) perspective (of course being careful how this is represented…)

□ When estimating resource assignments, think of the appropriate curves for applying

resources o Early on only apply approximately 20% of the total requirement for early project

definition. These are typically project managers, leaders and domain experts o Then ramp the staff up to full execution o Start ramping down early and gradually. Always factor in “remainder”

resource requirements that are needed to keep the project going

□ Resource assignment graph – types, numbers, timing

□ Key external linkages – are you using any external or contract resources? Are there any 3’rd party tools or components that are critical to the effort?

□ Key challenges the project presents to your organization

Keep in mind that this is a “high level” view to the project and it should hang together against most project changes – excluding cancellation ;-)



Section Three – Team Dynamics

3.1 Organization Chart The organization chart for the team.

3.2 Leadership Structure Chart Similar to the organization chart, this highlights direct and soft leadership structure within the team. It identifies the managers, team leaders and domain experts. It reflects direct reporting relationships and dotted line relationships. It should accurately reflect the management and leadership structure of the team

3.3 Key Roles & Responsibilities Identify key roles and responsibilities within your team. For example, if your team is supporting IT infrastructure, then you might have R&R along infrastructure, Windows and Unix boundaries. You also want to speak to requirements you might have of “leadership” roles. Be clear and specific in defining your expectations. In fact, that’s what this section is about – team expectations.

3.4 Anticipated Change What are the anticipated growth or contraction targets for the team? How will growth be managed within the organization? Modified organization & leadership structure charts should be placed in this section reflecting anticipated changes. Also speak to organizational development in this area. Do you need leadership training? Are there any key areas you want to focus on for improvement? If your group size suddenly doubled, how would you handle it? Secession planning is also part of this section.

3.5 Group Growth and Promotion Planning You might or might not share this section with the team. This is your internal compass to how you envision shaping and growing your team. It should clearly encompass internal and external plans for shaping the team.

3.6 External Relationships Today many teams cross different locals and/or collaborate with external vendors and outsourcers. This creates a unique challenge for effective teamwork and introduces additional risk in creating integrated products. If you’re in this situation, speak to the methods you plan on employing to improve cross team connections and ultimate work outcomes.



Section Four – Group Training Plans

4.1 Required core competencies List the required core or minimal competencies for your team. If there are specialized skill areas, then list them in groups. Think of this as if you were evaluating the core skills of your team. What would they be? And don’t forget about soft skills. Sometimes it’s useful to gather this information directly from the team – since it’s always difficult to know and understand all of the important competencies from a managers’ perspective. This also helps to prepare the team for subsequent self evaluation.

4.2 Group Evaluation Against Core Competencies Go through an evaluation of your team against your set of core competencies. I prefer self-assessments by the team members against the core competency list. Simply accumulate the results here. You should also identify any key gaps and prioritize them from the perspective of impacting project and vision goals.

4.3 Group Training Plan Identify the core training initiatives that are planned to fill the “gaps” identified in the evaluation above. Include formal and information plans – keeping in mind that there are many ways to improve the overall training within your team. Don’t forget about setting up mentoring relationships as well. Part of this section should be “linking” the planned training results back to the “gaps” and setting some expectation as to the results. For example, will the gap be narrowed or closed entirely as a result of executing the training plan?

4.4 New Hire and Entry Level Training Plans If you are planning growth, insure that you have these training areas covered.

Creating & Leading the High Performance Test Organization€¦ · Creating & Leading the High...

Documents

Transcript of Creating & Leading the High Performance Test Organization€¦ · Creating & Leading the High...