The  New  Agile  Tes,ng  Quadrants:    Bringing  Skilled  Testers  and  Developers  Together  

 

James  Bach  james@sa,sfice.com  

Michael  Bolton    michael@developsense.com)  

 (and  with  helpful  comments  from  Interna,onal  Society  of  SoAware  

Tes,ng  members:  Anne-­‐Marie  CharreI,  James  Lyndsay,  Simon  Morley,  and  Ben  Kelly)    

Marick’s  Original  

•  Marick’s  original  confuses  concept  of  cri,cal  distance  with  that  of  process  integrity–  sees  simple  output  checks  as  more  “integral”  to  the  programming  process  than  vigorous  tes,ng.  (Let’s  treat  it  all  as  connected  together.  It’s  agile,  dammit.)  

•  Crispin/Gregory  version  implies  that  cri,que  is  not  suppor,ng  the  work  of  programming.  This  helps  perpetuate  the  ignorant  aVtude  that  testers  do  not  belong  in  Agile  unless  they  write  code.  (Tes,ng  IS  suppor,ng  the  team.  Testers  ARE  on  the  team.)  

•  Both  versions  confuse  output  checking  (which  is  completely  automatable)  with  tes,ng  (which  is  not).  (Tes,ng  is  a  live  thought  process,  just  like  programming.)  

•  Crispin/Gregory  version  makes  confusing  and  unnecessary  dis,nc,ons  about  tes,ng  with  tools  and  without  tools.  (Tools  are  not  remarkable  in  tes,ng.  Good  testers  use  them  anywhere.)  

•  Both  versions  pin  certain  techniques  and  approaches  to  certain  quadrants.  (Any  test  technique  or  approach  may  relate  to  any  quadrant–  which  represent  overarching  tasks  and  goals.)  

“Facings”  are  beside  the  point.  •  THE  BUSINESS  needs  us  to  produce  something  of  value.  •  THE  BUSINESS  needs  us  to  do  that  efficiently.  •  THE  BUSINESS  needs  to  learn  what  it  values  over  ,me  rather  

guessing  and  freezing  that  guess  forever.    

•  Hence,  the  core  heuris,c  of  agile:  con,nually  re-­‐focus  on  value  (in  order  to  produce  value)  and  ply  our  craQ  in  ways  that  reduce  the  cost  of  change  (rather  than  deny  change).  

 •  “Technology-­‐facing”  simply  means  doing  things  that  help  us  

build  with  change  in  mind–  an  ac,vity  our  business  clients  need  but  do  not  directly  care  about  (or  some,mes  even  know  about.)  

Reifica,on  Fallacy  

•  The  versions  of  the  quadrants  I’ve  seen  also  commit  the  “test  cases  are  tes,ng”  and  “examples  are  tests”  reifica,on  fallacies.    

•  Tes,ng  cannot  be  encoded.  (Just  as  “programming”  cannot  be  encoded.  You  cannot  “code  me  a  programmer.”)  

•  It  is  pointless  to  discuss  whether  “business  people”  can  “read  the  tests”  because  what  they  can  read  are  not  tests–  they  are  par,al  representa,ons  of  tes,ng  ac,vity,  or  else  they  are  checks.  

•  If  you  try  to  communicate  tes,ng  primarily  through  wri,ng  then  you  are  doing  it  wrong  (viola,ng  Agile  principles).  Instead:  prefer  conversa,on  and  demonstra,on.  

Dimensions  of  Crispin/Gregory  “Agile  Tes,ng  Quadrants”  Based  on  Marick  

 

First,  refactor  those  dimensions…    

(This version avoids alienating professional testers and more directly addresses the tension between business and technology “facings.”)

“Con,nuous  aIen,on  to  technical  excellence  and  good  design  enhances  agility.”    

“Our  highest  priority  is  to  sa,sfy  the  customer  through…valuable  soAware”  

And  remind  ourselves  of  the  core  tac,cs  of  Agile…    

And  remind  ourselves  of  the  core  tac,cs  of  Agile…    

(These are the core tactics as we see them. You may prefer a slightly different list.)

But  hold  on  a  moment.  Development  plays  out  over  ,me…  

 

All development of any new design

works like this.

But  hold  on  a  moment.  Development  plays  out  over  ,me…  

 

Waterfall  “Built  it  right  the  first  ,me.”  

Agile  “Build  with  change  in  mind.”  

So,  we  have  our  clockwise  cycle…    

…and  corners  that  represent  enabling  paIerns.  

“As”  does  not  mean  “aAer”  

Although  there  is  a  cyclic  tendency  to  these  ac,vi,es,  they  also  overlap,  combine,  and  support  each  other,  in  big  loops,  small  loops,  

sudden  turns,  and  epicycles.    

Like  swirls  from  s,rring  a  cup  of  coffee…   …not  like  being  ,ed  to  the  hands  of  a  clock  

Development  isn’t  strictly  sequen,al!    

Stories,  spikes,  itera,ons,  sprints,  releases;  whatever  name  for  some  

burst  of  development  work.  

Now,  let’s  create  the  tes,ng  quadrants…  

Each  quadrant  represents  a  set  of  Agile  tes,ng  ac,vi,es.  

(Testing suffuses Agile development, but the character of the activities is quite different in each of the quadrants.)

(Notice that there are no test techniques or tools listed in the activities. That’s because test techniques and tools do not live in any particular quadrant.)

“Distance” refers to the difference between one perspective and another. Testing benefits from diverse perspectives. Shallow testing doesn’t need critical distance, but deeper or naturalistic long-form testing tends to require or create more distance from the builder’s mindset.

Deep  tes,ng  requires  cri,cal  distance.  

Envisioning  Success  

An,cipa,ng  Failure  

Focusing  Mindset  

Defocusing  Mindset  

Central  Obstacle  Divides  Work  

Mt.  Mindset  

NOTE: We do NOT claim that this work must be done by different people, or that the people must have different roles. We DO claim that roles on an agile team (collaborating with each other) are a powerful heuristic for solving the mindset switching problem.

Developer  skill  focus  

Tester  skill  focus  

Business  analyst  skill  focus  

Skilled  tes,ng  and  skilled  development  interact  in  a  “trading  zone”    

Peter Galison introduced the notion of a trading zone in Science as a situation wherein people from different disciplines try to work together despite their very different and incompatible concepts and language.

Collins’  Trading  Zones  Model  

Complex  Tes,ng  Example  #1  

Example  #2A:  3000  iden,cal  queries  on  eBay  in  1.5  hours,  graphing  number  of  hits  returned  

What  explains  the  weirdly  different  levels?  

284900  

283950  

0 200 400 600 800 1000

44700

44800

44900

45000

Index

sim$V1 Example  #2B:  Simulator  created  

by  tester  to  explore  one  theory  of  the  strange  hit  paIern:  One  Hadoop  cluster  out  of  a  hundred  has  flaky  servers  in  it.