Why ask why? Try agile BI!

26
VOTE on the Biggest BI Challenges Welcome! As you’re getting settled, take a minute to put a dot on the board by what you think are the three biggest BI challenges.

description

Sara Handel's presentation from AgileDC on October 21, 2014.

Transcript of Why ask why? Try agile BI!

Page 1: Why ask why? Try agile BI!

VOTE on the Biggest BI Challenges

Welcome!

As you’re getting settled, take a

minute to put a dot on the board

by what you think are the three

biggest BI challenges.

Page 2: Why ask why? Try agile BI!

Why Ask WhyTry Agile BI

November 5, 2014

Page 3: Why ask why? Try agile BI!

Agenda

• Agile BI Definition

• Assess Your Current State

• Then…

– Select an Agile Methodology

– Have a Kickoff

– Inspect and Adapt

Page 4: Why ask why? Try agile BI!

BI

BI encompasses all aspects of a system

needed to produce meaningful information

to drive data driven decision making

– Data Processing (Cleansing, Transforming,

Loading)

– Data Architecture and Warehousing

– Data Analysis and Visualization Tools

Page 5: Why ask why? Try agile BI!

Agile

Page 6: Why ask why? Try agile BI!

Agile BI

Applying an Agile mindset to business intelligence

• Using an iterative, incremental, evolutionary

approach

• Focusing on value-driven development

• Delivering production quality applications

• Using barely sufficient processes

• Automating everything

• Collaborating with the customer

• Encouraging self-organizing and self managing

teams

Page 7: Why ask why? Try agile BI!

Why Agile?

Source: http://www.versionone.com/Agile101/Agile-Software-Development-Benefits/

The people who need

the data see the data

You need different

data? Sure!

The data source

changed? We’re on it!

Business Intelligence,

enough said

Fail fast

Page 8: Why ask why? Try agile BI!

Agile BI Maturity ModelTeam Roles /

Skill Sets

Technical

Architecture

Engineering

Practices

*All team members can independently

complete any task from database

design to report creation

*It’s not about getting your job done it’s

about getting the job done

*Increased collaboration

Example: ETL developers work with

data modelers to come up with a

database design that balances the

tradeoffs between reporting and

loading

*Decreased formality in interactions

across skill sets

*Collaboration among people with the

same skill set - Example: data

modelers work with other data

modelers

*Official transitions and likely

disagreement across skill sets -

Example: ETL developers are given

source to target mappings when the

data modelers complete the database

design and are upset that the design is

hard to load

*Clear understanding of data’s

business value

*Clear understanding of the purpose for

each component of the technical

architecture

*Active effort to clarify understanding of

data’s business value

*Streamlined architecture where

possible

*Process to deprecate unused

components

*Numerous (possibly) redundant layers

(staging, ODS, EDW, data marts, etc.)

*Inclusion of data with no clear

business value

*Lingering tables, reports, ETL scripts,

with no known purpose

*End-to-end use of optimal engineering

practices

*Team self-enforces usage through

criteria for completing work

*Some configuration management

(SQL scripts to create all db objects are

under CM, but not ETL and report

information)

*Some automation is in place (perhaps

to promote new objects or code to

another environment or to test ETL)

Leve

l 3

Leve

l 2

Leve

l 1

Page 9: Why ask why? Try agile BI!

Assess Your Current State

• How well is your team setup for

collaboration and change?

Page 10: Why ask why? Try agile BI!

Agile BI Maturity Model

Team Roles / Skill Sets

It’s not about getting your job done

it’s about getting the job doneLevel

3

Level

2

Level

1

Collaboration Formality

ETL ReportingData

ModelersDBAs

etc.

https://www.castlellc.com/collaboration.aspx

Page 11: Why ask why? Try agile BI!

Agile BI Maturity Model

Create a dedicated team with skills needed to get data

into the hands of end users to make decisions

Team Roles / Skill Sets

Support self organized culture

- Let the team define their own success criteria

- Avoid saying HOW things must be done

Fill skill set gaps with external training, cross training,

lunch and learns and more

Level

3

Level

2

Level

1

Page 12: Why ask why? Try agile BI!

Assess Your Current State

• How well is your team setup for

collaboration and change?

Page 13: Why ask why? Try agile BI!

Assess Your Current State

• What is your current technical

architecture? What aspects present the

biggest challenges to incremental

evolution and change?

Page 14: Why ask why? Try agile BI!

Avoidable Inevitable

Change Is…• Grain of fact table

• New type 2 attribute

• Change from type 1 to type 2

• Multi-purpose column or table

• Redundant data

• Tables with too many columns or rows

• “Smart” columns

• Complex ETL objects

• Large SQL modules

• Unconformed Dimensions

• Indiscriminate use of materialized views

• Underutilization of materialized views

• Overreliance on documentation

Page 15: Why ask why? Try agile BI!

Agile BI Maturity Model

Technical Architecture

Level

3

Level

2

Level

1 Purpose?? Value??

Purpose? Value?

Purpose! Value!

Page 16: Why ask why? Try agile BI!

Agile BI Maturity Model

* Identify redundancy

* Combine or streamline things where possible

Technical Architecture

Keep it up! Don’t let complexity creep in.

*Create a central repository

*Get rid of things that are no longer being used

Level

3

Level

2

Level

1

Page 17: Why ask why? Try agile BI!

Assess Your Current State

• What is your current technical

architecture? What aspects present the

biggest challenges to incremental

evolution and change?

Page 18: Why ask why? Try agile BI!

Assess Your Current State

• Do you follow technical practices that can

enable agility?

Page 19: Why ask why? Try agile BI!

Agile BI Maturity Model

Engineering Practices

Level

3

Level

2

Level

1

• No central location for system building

blocks

• Manual push between environments

• Some configuration management

• Some automation

End-to-end use of optimal engineering practices

Page 20: Why ask why? Try agile BI!

Agile BI Maturity Model

*Start putting files into a configuration management

system

*Work out the kinks of your deployment process

Engineering Practices

*Hold yourselves accountable for maintaining high

standards for new efforts

*Reduce technical debt each iteration

*Start creating automated tests

Level

3

Level

2

Level

1

Page 21: Why ask why? Try agile BI!

Assess Your Current State

• Do you follow technical practices that can

enable agility?

Page 22: Why ask why? Try agile BI!

Agile BI Maturity ModelTeam Roles /

Skill Sets

Technical

Architecture

Engineering

Practices

*All team members can independently

complete any task from database

design to report creation

*It’s not about getting your job done it’s

about getting the job done

*Increased collaboration

Example: ETL developers work with

data modelers to come up with a

database design that balances the

tradeoffs between reporting and

loading

*Decreased formality in interactions

across skill sets

*Collaboration among people with the

same skill set - Example: data modelers

work with other data modelers

*Official transitions and likely

disagreement across skill sets - Example:

ETL developers are given source to

target mappings when the data modelers

complete the database design and are

upset that the design is hard to load

*Clear understanding of data’s

business value

*Clear understanding of the purpose for

each component of the technical

architecture

*Active effort to clarify understanding of

data’s business value

*Streamlined architecture where

possible

*Process to deprecate unused

components

*Numerous (possibly) redundant layers

(staging, ODS, EDW, data marts, etc.)

*Inclusion of data with no clear

business value

*Lingering tables, reports, ETL scripts,

with no known purpose

*End-to-end use of optimal engineering

practices

*Team self-enforces usage through

criteria for completing work

*Some configuration management

(SQL scripts to create all db objects are

under CM, but not ETL and report

information)

*Some automation is in place (perhaps

to promote new objects or code to

another environment or to test ETL)

*Building blocks of the system (db

create scripts, ETL packages, report

files, etc.) are not maintained in any

central location nor are they under

configuration management

*Files are manually copied from one

environment to another

Leve

l 3

Leve

l 2

Leve

l 1

Page 23: Why ask why? Try agile BI!

Select an Agile Methodology

Scrum

Kanban

RU

P

XP

BDD

And so

on…

Scrum

Page 24: Why ask why? Try agile BI!

Have a Kick Off

Page 25: Why ask why? Try agile BI!

Inspect and Adapt

Page 26: Why ask why? Try agile BI!

Contact InformationSara Handel

[email protected]

Agile BI Training – December 11, 2014

Check out Wyn Van Devanter next

A Thin Automation Framework for Manageable

Automated Acceptance Testing