Measuring & Analyzing Things That Matter When You Have Too Many Things To Keep

Track Of*

* this title easily understood by 13-15 year old students

by @chadfowler

(chocolate salmon)

Thursday, February 23, 12

Thursday, February 23, 12

How does this happen?

Thursday, February 23, 12

How do we fix it?

• code reviews are hard

• code reviews are time consuming

• there are too many commits to review

• bulleted lists are awesome

Thursday, February 23, 12

information radiator“Coined around 2000 while standing in a Thoughtworks office looking at all the paper on the walls around me, “information radiator” refers to a publicly posted display that shows people walking by what is going on. Information radiators are best when they are big, very easy to see (e.g. not online, generally), and change often enough to be worth revisiting.”

http://alistair.cockburn.us/Information+radiator

Thursday, February 23, 12

http://xprogramming.com/articles/bigvisiblecharts/

“Chart what you care about, what you worry about, what you want other people to know.”

Thursday, February 23, 12

http://www.exampler.com/old-blog/2005/04/02/Thursday, February 23, 12

1/2 Marathon Training

Thursday, February 23, 12

What Should We Measure?

• Project progress

• Code complexity

• Developer “productivity”

• Business “key performance indicators”

• Bug/error rates

• Everything important

Thursday, February 23, 12

Code “Quality”

Thursday, February 23, 12

Rudolph Flesch

Thursday, February 23, 12

Score Notes

90-100 easily understood by an average 11-year-old student

60-70 easily understood by 13- to 15-year-old students

0-30 best understood by university graduates

Flesch-Kincaid Reading Ease

Thursday, February 23, 12

competing algorithms

• SMOG

• Gunning-Fog

http://www.editcentral.com/gwt1/EditCentral.html

Thursday, February 23, 12

Maurice Halstead “Software Science”

http://en.wikipedia.org/wiki/Halstead_complexity_measures

Thursday, February 23, 12

Measure units of human thought used!!!

http://yunus.hacettepe.edu.tr/~sencer/complexity.html

T ' = ( n1N2( n1log2n1 + n2log2n2) log2n) / 2n2S

“Stroud’s (1955, 1967) original proposal, which is usually called a discrete-moment model, claims that

all incoming information is processed in nonoverlapping (i.e., temporarily discrete) samples

or scans and that the temporal order of stimuli within a scan is not preserved.”

Thursday, February 23, 12

abc

http://www.softwarerenovation.com/ABCMetric.pdf

Assignment – an explicit transfer of data into a variableBranch – an explicit forward program branch out of scope

Condition – a logical/Boolean test

|ABC| = sqrt((A*A)+(B*B)+(C*C))

Thursday, February 23, 12

Cyclomatic Complexity

M = E − N + 2P

E = the number of edges of the graphN = the number of nodes of the graphP = the number of connected components

Thursday, February 23, 12

Basis Path Testing

http://users.csc.calpoly.edu/~jdalbey/206/Lectures/BasisPathTutorial/index.html

Thursday, February 23, 12

Complexity Conservation

“One of McCabe's original applications was to limit the complexity of routines during program development; he recommended that programmers should count the complexity of the modules they are developing, and split them into smaller modules whenever the cyclomatic complexity of the module exceeded 10.”

http://en.wikipedia.org/wiki/Cyclomatic_complexity#cite_note-mccabe76-2

NO MORE THAN TEN GALLONS OF CYCLOMATIC COMPLEXITY PER MODULE!!!!!

Thursday, February 23, 12

Complexity & Cohesion

“Our results support the idea that there is a relationship between complexity and cohesion, basically that a lack of

cohesion is associated with high complexity”

Exploring the Relationship between Cohesion and Complexity - Stein, Cox, Etzkorn - 2005

Thursday, February 23, 12

saikuro

http://saikuro.rubyforge.org/Thursday, February 23, 12

yup

Thursday, February 23, 12

flog

• Written by Ryan Davis

• opinionated

• ruby specific

Thursday, February 23, 12

Thursday, February 23, 12

Thursday, February 23, 12

Score Notes

1-2 easily understood by an average Visual Basic developer

3-10 easily understood by PHP developers

10-20 best understood by Perl, Smalltalk, or Lisp developers

20+ rich hickey

Flog Method Reading Ease *

* all totally contrived

Thursday, February 23, 12

flogtime

Not quite done, but a fun quick hack

https://github.com/chad/flogtime

Thursday, February 23, 12

code coverage

https://github.com/colszowka/simplecovThursday, February 23, 12

code coverage

• C0 - every line executed

• C1 - every branch executed

• C2 - every code path across lines executed

It doesn’t matter which one. Only direction matters.

Thursday, February 23, 12

metric-fu

http://metric-fu.rubyforge.org/

roodiflogflaychurnstatsrails_best_practicesreeksaikurocoverage

Thursday, February 23, 12

Ward’s Signature Survey

http://c2.com/doc/SignatureSurvey/Thursday, February 23, 12

something stupid

Thursday, February 23, 12

Thursday, February 23, 12

Bug Prediction at Google

http://google-engtools.blogspot.com/2011/12/bug-prediction-at-google.html

“Here at Google, we have thousands of engineers working on our code base every day. In fact, as previously noted, 50% of the Google code base changes every month.”

Thursday, February 23, 12

Bug Prediction at Google

https://github.com/igrigorik/bugspots

Thursday, February 23, 12

$> cd /your/git/repo

$> git bugspots -d 500

.. example output ..

Scanning /git/eventmachine repo

Found 31 bugfix commits, with 23 hotspots:

Fixes:

- Revert "Write maximum of 16KB of data to an SSL connection per tick (fixes #233)" for #273

- Do not close attached sockets (fixes #200)

- Write maximum of 16KB of data to an SSL connection per tick (fixes #233)

- Merge branch 'master' into close_schedule_fix

- Remove dependency on readbytes.rb for ruby 1.9 (fixes #167, #234)

- Fix compilation on MSVC2008 (fixes #253)

- EM::Deferrable#(callback|errback|timeout) now return self so you can chain them (closes #177)

- Make EventMachine::Connection#get_peername and #get_sockname valid for IPv6 (closes #132)

- reconnect DNS socket if closed

- Use String#bytesize in EM::Connection#send_datagram for 1.9 (closes #153)

Thursday, February 23, 12

Hotspots: 0.9723 - ext/ed.cpp 0.3311 - ext/ed.h 0.3271 - ext/em.cpp 0.3034 - lib/eventmachine.rb 0.2433 - lib/em/protocols/postgres3.rb 0.2403 - ext/project.h 0.0431 - lib/em/deferrable.rb 0.029 - ext/cmain.cpp 0.0278 - ext/rubymain.cpp 0.0277 - ext/eventmachine.h 0.0241 - lib/em/resolver.rb 0.0241 - tests/test_resolver.rb 0.0225 - lib/em/connection.rb 0.0013 - lib/em/protocols/smtpserver.rb 0.0003 - ext/extconf.rb 0.0002 - tests/test_basic.rb 0.0001 - ext/em.h 0.0001 - ext/cplusplus.cpp

Thursday, February 23, 12

turbulence

https://github.com/chad/turbulence

http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDyn=2

Thursday, February 23, 12

turbulencehighest complexityhighest turbulence

Thursday, February 23, 12

gitstats

http://gitstats.sourceforge.net/Thursday, February 23, 12

gitstats

Thursday, February 23, 12

The Church of Graphs

Thursday, February 23, 12

graphite

http://graphite.wikidot.com/

Highly scalable open source real time graphing engine.

Thursday, February 23, 12

statsd

https://github.com/etsy/statsd

Open source node-based daemon for stats aggregation. UDP front-end for fault-

tolerance*.

Pushes aggregated stats into graphite.

Thursday, February 23, 12

statsd gem

Thursday, February 23, 12

Resque failures

Thursday, February 23, 12

Error rates that matter

Thursday, February 23, 12

various key stats overlaid

Thursday, February 23, 12

looking at pictures only gets you so far

Thursday, February 23, 12

Shewhart Charts

Measure process stability

Thursday, February 23, 12

doesn’t work for data like this though

Thursday, February 23, 12

Double Exponential Smoothing (Holt-Winters)

http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdf

Thursday, February 23, 12

Thursday, February 23, 12

What is QA?

50 releases per day redefines “QA”Thursday, February 23, 12

Thanks!

Thursday, February 23, 12