Measuring

Measuring & Analyzing Things That Matter When You Have Too Many Things To Keep

Track Of*

* this title easily understood by 13-15 year old students

by @chadfowler

(chocolate salmon)

Thursday, February 23, 12

How does this happen?


How do we fix it?

• code reviews are hard

• code reviews are time consuming

• there are too many commits to review

• bulleted lists are awesome


information radiator“Coined around 2000 while standing in a Thoughtworks office looking at all the paper on the walls around me, “information radiator” refers to a publicly posted display that shows people walking by what is going on. Information radiators are best when they are big, very easy to see (e.g. not online, generally), and change often enough to be worth revisiting.”

http://alistair.cockburn.us/Information+radiator




http://xprogramming.com/articles/bigvisiblecharts/

“Chart what you care about, what you worry about, what you want other people to know.”




http://www.exampler.com/old-blog/2005/04/02/Thursday, February 23, 12

http://www.exampler.com/old-blog/2005/04/02/

http://www.exampler.com/old-blog/2005/04/02/

1/2 Marathon Training


What Should We Measure?

• Project progress

• Code complexity

• Developer “productivity”

• Business “key performance indicators”

• Bug/error rates

• Everything important


Code “Quality”


Rudolph Flesch


Score Notes

90-100 easily understood by an average 11-year-old student

60-70 easily understood by 13- to 15-year-old students

0-30 best understood by university graduates

Flesch-Kincaid Reading Ease


competing algorithms

• SMOG

• Gunning-Fog

http://www.editcentral.com/gwt1/EditCentral.html




Maurice Halstead “Software Science”

http://en.wikipedia.org/wiki/Halstead_complexity_measures




Measure units of human thought used!!!

http://yunus.hacettepe.edu.tr/~sencer/complexity.html

T ' = ( n1N2( n1log2n1 + n2log2n2) log2n) / 2n2S

“Stroud’s (1955, 1967) original proposal, which is usually called a discrete-moment model, claims that

all incoming information is processed in nonoverlapping (i.e., temporarily discrete) samples

or scans and that the temporal order of stimuli within a scan is not preserved.”




abc

http://www.softwarerenovation.com/ABCMetric.pdf

Assignment – an explicit transfer of data into a variableBranch – an explicit forward program branch out of scope

Condition – a logical/Boolean test

|ABC| = sqrt((A*A)+(B*B)+(C*C))




Cyclomatic Complexity

M = E − N + 2P

E = the number of edges of the graphN = the number of nodes of the graphP = the number of connected components


Basis Path Testing

http://users.csc.calpoly.edu/~jdalbey/206/Lectures/BasisPathTutorial/index.html




Complexity Conservation

“One of McCabe's original applications was to limit the complexity of routines during program development; he recommended that programmers should count the complexity of the modules they are developing, and split them into smaller modules whenever the cyclomatic complexity of the module exceeded 10.”

http://en.wikipedia.org/wiki/Cyclomatic_complexity#cite_note-mccabe76-2

NO MORE THAN TEN GALLONS OF CYCLOMATIC COMPLEXITY PER MODULE!!!!!




Complexity & Cohesion

“Our results support the idea that there is a relationship between complexity and cohesion, basically that a lack of

cohesion is associated with high complexity”

Exploring the Relationship between Cohesion and Complexity - Stein, Cox, Etzkorn - 2005


saikuro

http://saikuro.rubyforge.org/Thursday, February 23, 12

http://saikuro.rubyforge.org/

http://saikuro.rubyforge.org/

yup


flog

• Written by Ryan Davis

• opinionated

• ruby specific


Score Notes

1-2 easily understood by an average Visual Basic developer

3-10 easily understood by PHP developers

10-20 best understood by Perl, Smalltalk, or Lisp developers

20+ rich hickey

Flog Method Reading Ease *

* all totally contrived


flogtime

Not quite done, but a fun quick hack

https://github.com/chad/flogtime




code coverage

https://github.com/colszowka/simplecovThursday, February 23, 12

https://github.com/colszowka/simplecov

https://github.com/colszowka/simplecov

code coverage

• C0 - every line executed

• C1 - every branch executed

• C2 - every code path across lines executed

It doesn’t matter which one. Only direction matters.


metric-fu

http://metric-fu.rubyforge.org/

roodiflogflaychurnstatsrails_best_practicesreeksaikurocoverage




Ward’s Signature Survey

http://c2.com/doc/SignatureSurvey/Thursday, February 23, 12

http://c2.com/doc/SignatureSurvey/

http://c2.com/doc/SignatureSurvey/

something stupid


Bug Prediction at Google

http://google-engtools.blogspot.com/2011/12/bug-prediction-at-google.html

“Here at Google, we have thousands of engineers working on our code base every day. In fact, as previously noted, 50% of the Google code base changes every month.”




http://www.infoq.com/presentations/Development-at-Google

http://www.infoq.com/presentations/Development-at-Google

Bug Prediction at Google

https://github.com/igrigorik/bugspots




$> cd /your/git/repo

$> git bugspots -d 500

.. example output ..

Scanning /git/eventmachine repo

Found 31 bugfix commits, with 23 hotspots:

Fixes:

- Revert "Write maximum of 16KB of data to an SSL connection per tick (fixes #233)" for #273

- Do not close attached sockets (fixes #200)

- Write maximum of 16KB of data to an SSL connection per tick (fixes #233)

- Merge branch 'master' into close_schedule_fix

- Remove dependency on readbytes.rb for ruby 1.9 (fixes #167, #234)

- Fix compilation on MSVC2008 (fixes #253)

- EM::Deferrable#(callback|errback|timeout) now return self so you can chain them (closes #177)

- Make EventMachine::Connection#get_peername and #get_sockname valid for IPv6 (closes #132)

- reconnect DNS socket if closed

- Use String#bytesize in EM::Connection#send_datagram for 1.9 (closes #153)


Hotspots: 0.9723 - ext/ed.cpp 0.3311 - ext/ed.h 0.3271 - ext/em.cpp 0.3034 - lib/eventmachine.rb 0.2433 - lib/em/protocols/postgres3.rb 0.2403 - ext/project.h 0.0431 - lib/em/deferrable.rb 0.029 - ext/cmain.cpp 0.0278 - ext/rubymain.cpp 0.0277 - ext/eventmachine.h 0.0241 - lib/em/resolver.rb 0.0241 - tests/test_resolver.rb 0.0225 - lib/em/connection.rb 0.0013 - lib/em/protocols/smtpserver.rb 0.0003 - ext/extconf.rb 0.0002 - tests/test_basic.rb 0.0001 - ext/em.h 0.0001 - ext/cplusplus.cpp


turbulence

https://github.com/chad/turbulence

http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDyn=2






turbulencehighest complexityhighest turbulence


gitstats

http://gitstats.sourceforge.net/Thursday, February 23, 12

http://gitstats.sourceforge.net/

http://gitstats.sourceforge.net/

gitstats


The Church of Graphs


graphite

http://graphite.wikidot.com/

Highly scalable open source real time graphing engine.




statsd

https://github.com/etsy/statsd

Open source node-based daemon for stats aggregation. UDP front-end for fault-

tolerance*.

Pushes aggregated stats into graphite.




statsd gem


Resque failures


Error rates that matter


various key stats overlaid


looking at pictures only gets you so far


Shewhart Charts

Measure process stability


doesn’t work for data like this though


Double Exponential Smoothing (Holt-Winters)

http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdf




What is QA?

50 releases per day redefines “QA”Thursday, February 23, 12

Thanks!


Measuring

Technology

Transcript of Measuring