SmarterWorkingDataTeam

30
But can your data tool make your data team work smarter? 1

Transcript of SmarterWorkingDataTeam

Page 1: SmarterWorkingDataTeam

1

But can your data tool make your data team

work smarter?

Page 2: SmarterWorkingDataTeam

2

A typical member of a data delivery teamis a single point of failure.

Why?

Page 3: SmarterWorkingDataTeam

3

Because she has unique knowledge of• Specific data domains• Specific processes that pipe the data to

a specific client• Specific applications for specific clients

to use the data

Page 4: SmarterWorkingDataTeam

4

She has this unique knowledge because she’s the one who personally created the experience based on her• Personal style of cleaning data, and only to a

minimum as driven by the specific client• Personal naming conventions, development style• Personal coding ability • Personal level of need for documentation

• —of course all team members will be different

Page 5: SmarterWorkingDataTeam

5

Your team should be using data that is standardized—

There are at least 4 major data suppliers to your data delivery team:• The upstream, even bigger data team• The team who developed that custom app for your line of business• The other team who developed that other custom app for your line of business• That major core corporate system that is now in the process of being replaced

• But with all these suppliers, can you honestly hope that data will

arrive the same way always?

Page 6: SmarterWorkingDataTeam

6

On your current corporate-approved platform:• There is a data handling limit (2 gigabyte), compared

to the actual size of the data• Or there is no data handling limit, but usability is so

clunky that users prefer to code over drag-and-drop (and they are proud of it)

• There are several versions of the corporate-approved platform in production

• Or if you have more than one platform, the dueling platforms have a you-say-potAYtoe-I-say-potAHtoe problem.

Page 7: SmarterWorkingDataTeam

7

Natural Personal Variation

Standard Corporat

e Platform Constrain

ts

Natural Data

Variation

These 3 elements combine, creating the existing work environment

for your data delivery team

Page 8: SmarterWorkingDataTeam

8

Natural Personal Variation

Standard Corporat

e Platform Constrain

ts

Natural Data

Variation

There other are side effects emanating from this environment

single points of failure

instability

furniture

inefficiency

behind the times in terms of techniques

A data delivery process that the client does not use to make decisions

Page 9: SmarterWorkingDataTeam

9

Natural Personal Variation

Standard Corporat

e Platform Constrain

ts

Natural Data

Variation

Some things you can’t change. What can you change to eliminate the side effects?

single points of failure

instability

furniture

inefficiency

behind the times in terms of techniquesthe tool

Page 10: SmarterWorkingDataTeam

10

You’ve always said the corporate-approved platform was “free”, but how expensive are the

negative side effects?

Page 11: SmarterWorkingDataTeam

11

What positive side effects do you want?

What features would yield these?

Page 12: SmarterWorkingDataTeam

12

1Eliminate

Data Chunking 2

Less Data

Processing or

Coding

3 More Work

Recycling

4Data

Consistency a Done

Deal

5 Intra-Team

Communi-cation

6 Unrestriced Data Shaping

7 Higher Order

Functions

Page 13: SmarterWorkingDataTeam

13

1 Eliminate Data Chunking

A better tool would allow you to handle any size data.• Then you wouldn’t have to build and manage

processes to chunk it• Then you wouldn’t have to manage data on either

side of the chunking line• You would not need to make tables and take up so

much server space

Page 14: SmarterWorkingDataTeam

14

Here is the property sheet for Input Data

Here is the Input Data icon

Notice all the supported file types

Page 15: SmarterWorkingDataTeam

15

2 Less Data Processing

If you did not have to habitually make tables:• You wouldn’t employ other action queries like

append, delete, and update• Then you wouldn’t have to wait for all that

processing• The standard tool wasn’t designed or optimized

for that kind of processing anyway

Page 16: SmarterWorkingDataTeam

16

Here is the Browse icon

Here is the view of the data.

When you are finished developed, remove or disable Browses for optimization

Page 17: SmarterWorkingDataTeam

17

2 Less Coding

Coding is necessary the further up the data stream, because it is physically necessary.

But coding is a hindrance the further down the data stream, as more and more human factors come in to play.

Page 18: SmarterWorkingDataTeam

18

3 More Work Recycling

A better tool would package macros so that they can be exported and imported from project to project, regardless of domain• Team members swapping packages• Packages available from the internet• All new work builds on previous work, lowering

the risk of furniture

Page 19: SmarterWorkingDataTeam

19

You can do Events.

Here is the property sheet for an entire module.

Here is where you import or export an entire module

You can download someone else’s module off the internet. You are not transporting data, just a program.

Page 20: SmarterWorkingDataTeam

20

4 Data Consistency a Done

DealInstead of enforcing data consistency across 4 or more data offices, you need a tool with which you write the

once-and-for-all program to enforce data consistency within your walls, regardless of which data source or which team member.

Page 21: SmarterWorkingDataTeam

21

Also, you can gauge efficiency and reasonable output of your module.

This is the property sheet for Dynamic Input

Here is the Dynamic Input icon

Page 22: SmarterWorkingDataTeam

22

5 Intra-Team

CommunicationA better tool would use standard graphics while applications were created in such a way that documentation was inherent.• Drag and drop workflows• Annotation, even default annotation• Icons, not uniquely named queries, sets, or

programs

Page 23: SmarterWorkingDataTeam

23

This is a Union icon

This Filter icon, with possible T or F outputs, could have been inserted as an afterthought.

This note on this Sort icon was automatic, too.

This is a Join icon with possible Left, Inner, or Right join options. Make the flow emate from the correct one. Check your decision with a Browse.

Page 24: SmarterWorkingDataTeam

24

6Unrestricted Data

ReshapingA better tool would go further than field manipulation.• Not just crosstab, but also transpose• Apply a formula to multiple fields• Do operations on previous row or next row

Page 25: SmarterWorkingDataTeam

25

Transpose

Cross Tab

The Multi Row Formula function means you can manipulate based on previous or subsequent rows.

Page 26: SmarterWorkingDataTeam

26

7Higher Order Functions

A better tool would offer higher order functions for everyday data situations, saving time.• Cleaning? Try a Parsing function, then a Regular

Expression (pattern matching function, but then a string function only as last resort

• Matching? Try Fuzzy Matching were “Rob Snow” and “Robert Snow” match

Page 27: SmarterWorkingDataTeam

27

This is the language for Regular Expressions. It is vast, and there are many “RegEx” icons for different little jobs.

Parsing text to columns is an everyday data chore which can be done with string functions, but not nearly as efficiently as a function built specifically for the job.

Page 28: SmarterWorkingDataTeam

1Eliminate

Data Chunking 2

Less Data

Processing or

Coding

3 More Work

Recycling

4Data

Consistency a Done

Deal

5 Intra-Team

Communication

6 Unrestricted Data Shaping

7 Higher Order

Functions

*or love, as needed or preferred

Team actually a team

Data stable and big

glory*

Efficiency

Advanced analytics

Low risk of furniture

Page 29: SmarterWorkingDataTeam

29

1Eliminate

Data Chunking 2

Less Data

Processing or

Coding

3 More Work

Recycling

4Data

Consistency a Done

Deal

5 Intra-Team

Communication

6 Unrestricted Data Shaping

7 Higher Order

Functions