SHARPEN THE AXE. Efficiency is doing things right. Effectiveness is doing the right thing!
Doing Analytics Right - Building the Analytics Environment
Transcript of Doing Analytics Right - Building the Analytics Environment
![Page 1: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/1.jpg)
Building the Analytics Environment
![Page 2: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/2.jpg)
Look Whose Talking
@tasktop
• Nicole Bryan, VP of Product Management, Tasktop– Passionate about improving the
experience of how software is delivered– Former Director at Borland Software– [email protected] |
@nicolebryan
• Dr Murray Cantor – Senior Consultant, Cutter Consortium – Working to improve our industry with
metrics– Former IBM Distinguished Engineer– [email protected] | @murraycantor
![Page 3: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/3.jpg)
What we’ve learned so far….
• Webinar 1: There is no “one size fits all” metric nirvana
• Webinar 2: Use GQM to design the metrics that are right for your mix of development
Today…
It’s all about the execution! Let’s get practical!
![Page 4: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/4.jpg)
©2015 Murray Cantor
Choosing metrics big picture
Agree on goals
- Depends on the levels and mixture of work
Agree on the how they fit into the loop
1. “How would we know we are achieving the goal”
2.” What response we should take?”
Determine the measures needed to answer the questions
- Apply the Einstein test (as simple as possible, but no simpler)
Specify the data needed to answer the questions
Automate collection and staging of the data
4
Today
![Page 5: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/5.jpg)
©2015 Murray Cantor
From Goals to Measures to Data (GQM-ish)
1. Identify a set of corporate, division and project business goals and associated measurement goals.
2. Specify a sense-and-respond loop to steer to the goal.
3. Generate questions based on the goal that if answered:
• Let you know have achieved, are trending to \ the goal?
• Provide the level of detail necessary to take action
– Where is the problem, bottleneck?
• Communicate progress to stakeholders
– Summaries, rollups
4. Select or specify data needed to answer the questions in terms of state transitions of the relevant artifacts
5. Study the data to specify the data set and statistic needed to be collected to answer those questions and track process and
product conformance to the goals.
6. Develop automated mechanisms for data collection.
7. Collect, validate and analyze the data in real identify patterns to diagnose organization situation and provide suggestions for
corrective actions.
8. Analyze the data in a post mortem fashion to assess conformance to the goals and to make recommendations for future
improvements.
5
![Page 6: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/6.jpg)
The “Last Mile Problem”
A phrase used in the telecommunications and technology industries to describe the
technologies and processes used to connect the end customer to a communications
network. The last mile is often stated in terms of the "last-mile problem", because
the end link between consumers and connectivity has proved to be
disproportionately expensive to solve.
Read more: http://www.investopedia.com/terms/l/lastmile.asp#ixzz3dAdJpzAQ
![Page 7: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/7.jpg)
The Last Mile Problem
![Page 8: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/8.jpg)
Aspiration without execution is useless!
No wait … It’s actually worse than useless…
![Page 9: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/9.jpg)
– If execution for your analytics solution is difficult it can quickly leads to “The Light is Brighter Here” anti-pattern
Danger!!!!
![Page 10: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/10.jpg)
How Do I Unlock All This Goodness?
Po
rtfo
lio M
gm
tAgile PM
Requirements
TestDev
Op
eratio
ns
![Page 11: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/11.jpg)
Why So Difficult?
– Tool Reality
• You have lots of them! So it’s not one ETL, its many ETLs! That gets very hard to maintain.
• You’ve got disparate tools but your GQM needs single source fed by variety of tools
– I’ve got defects in HP QC, Rally and JIRA – how do I calculate cycle time!!!
• Yes, tool vendors have analytics solutions…. and these solutions are focused on their particular areas of specialization
![Page 12: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/12.jpg)
Why So Difficult?
– Logistics problems
• SaaS problem – sometimes data only available for limited time
• Transaction based data vs. reporting based data
• Many of the smaller more purpose built tools have no thought that the transactional data they are producing needs to participate in a larger analytics strategy
• You say tomato, I say tomato
Remember – you want your point tools to stay focused on their domain expertise
![Page 13: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/13.jpg)
What is the solution?
– Collated data across tools
– Abstraction away from specific tool representations of artifacts
– Near real time access
– Mix of simplicity so that you can just “get going” combined with the ability to “get sophisticated” when you need to/are ready to
![Page 14: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/14.jpg)
Powering software lifecycle analytics
![Page 15: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/15.jpg)
0
2
4
6
8
May June July Aug
0
2
4
6
ETL
Customer Data
Warehouse
“Raw” Data Storage in customer Database
(etc.)
![Page 16: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/16.jpg)
![Page 17: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/17.jpg)
Remember what Murray taught us?
![Page 18: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/18.jpg)
©2015 Murray Cantor
Kinds of Development Efforts: What is your mix?
18
1. Low innovation/high
certainty
• Detailed understanding
of the requirements
• Well understood code
2. Some innovation/
some uncertainty
• Architecture/Design in
place
• Some discovery required
to have confidence in
requirements
• Some
refactoring/evolution of
design might be required
3. High innovation/Low
Uncertainty
• Requirements not fully
understood, some
experimentation might be
required
• May be alternatives in choice
of technology
• No initial design/architecture
![Page 19: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/19.jpg)
©2015 Murray Cantor
Descriptive example: Cycle times
19
![Page 20: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/20.jpg)
Let’s Bring Cycle Time to Life!!!!!!!!!!!
![Page 21: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/21.jpg)
First, some key concepts of Tasktop Data
![Page 22: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/22.jpg)
Defects
Requirements
Test CasesTimesheets
A tangible by-product produced during the development of software.
Artifacts CollectionsA set of artifacts from your repository
Collection #1
Collection #2
![Page 23: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/23.jpg)
JIRA Defects Collection
Priority• High• Medium• Low• Trivial
Summary
Fix Version
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
Project #1
Project #2
Project #3
![Page 24: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/24.jpg)
HP Defects Collection
Priority• 1• 2• 3• 4
Description
Release
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
Project #A
Project #B
Project #C
![Page 25: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/25.jpg)
Event Collection
Priority• High• Medium• Low
Description
Released In
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
* Raw database collections are a little bit special
![Page 26: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/26.jpg)
Reporting Integration
Flow Specification
![Page 27: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/27.jpg)
And it will results in a database table like below
Another way of looking at it…Use this Model feeding defects from JIRA, HP, etc
![Page 28: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/28.jpg)
Artifact ID
Project Type Created Modified Severity Priority Status Release Assignee
DEF-1 Project A Defect 1/1/15 1/1/15 1 High Open
DEF-1 Project A Defect 1/1/15 1/2/15 1 High In Progress
John
DEF-1 Project A Defect 1/1/15 1/5/15 1 Med In Progress
John
DEF-1 Project A Defect 1/1/15 1/7/15 1 Med Shipped 1.0.0.1 John
1 Artifact, 4 Rows in Database
Event Log Concept
![Page 29: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/29.jpg)
And once you’ve got that, you can easily get things like this….
![Page 30: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/30.jpg)
Demo
![Page 31: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/31.jpg)
(2) Create or reuse a model
(3) Create collections(And Map the Collection to the model)
(4) Create an integration
Four Easy Steps
(1) Connect to your system
1234
![Page 32: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/32.jpg)
(1) Connect To Your System
![Page 33: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/33.jpg)
(2) Create or reuse a model
• Identify the fields to flow• Configure to Normalize the Data
![Page 34: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/34.jpg)
(3) Create Collections (and map them)
![Page 35: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/35.jpg)
(3) Create Collections (and map them)
One Core Artifact Type
Sourced from One Repository
Many Projects
Mapped to One Model
![Page 36: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/36.jpg)
• Configure fields and field values to conform to the normalized model values
• Transform values
Mapping Artifact to Model
![Page 37: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/37.jpg)
(4) Create an Integration
![Page 38: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/38.jpg)
Solves the Last Mile Problem
– Collated data across tools
– Abstraction away from specific tool representations of artifacts
– Near real time access
– Mix of simplicity so that you can just “get going” combined with the ability to “get sophisticated” when you need to/are ready to
![Page 39: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/39.jpg)
![Page 41: Doing Analytics Right - Building the Analytics Environment](https://reader031.fdocuments.net/reader031/viewer/2022030304/58761ced1a28ab306c8b7b83/html5/thumbnails/41.jpg)
@tasktop@cuttertweets