INFINITELY SCALABLE DATA MANAGEMENT

GETTING OUR HEADS INTO THE CLOUDS

THIS IS THE MOST IMPORTANT SLIDE OF THE DECK. REALLY.

IF YOU REMEMBER NOTHING ELSE

▸ Cloud technologies have already changed everything - hardware and processing constraints are disappearing

▸ The technologies and tools have evolved from what we have used for decades

▸ This is mostly a people and organizational leadership challenge

▸ The virtuous cycle:

▸ Measure - Identify Improvements - Improve - Repeat

▸ It is not too late to evolve our businesses, and it is up to us

2

HELLO CLASS. MY NAME IS ANTHONY, AND I AM A DATA GUY.

WHO AM I▸ Early career doing data development,

architecture, warehousing and business intelligence in trading industry

▸ MBA from Kellogg

▸ Data management and strategy consultant

▸ First CDO for the Chicago Transit Authority

▸ Senior Partner and Chief Data Architect for Uturn Data Solutions, a company helping businesses improve with data and cloud technologies

3

aalgmin@uturndata.com312-957-8527@AJAlgmin

HELP ME HELP YOU HELP ME GIVE YOU THE MOST I CAN DURING THE CLASS.

WHO ARE YOU

▸ Do you identify as IT or Business?

▸ What kinds of roles?

▸ Industries?

▸ Size of organization?

▸ Data management / data governance maturity?

▸ Where are you in your relationship with the cloud?

▸ Heard of it, but done nothing substantial

▸ Dev/Test or selected workflows?

▸ Production Systems / Core Operations

4

AVOIDING CLOUD TODAY IS LIKE AVOIDING COMPUTERS IN THE 1980’S. GOOD LUCK WITH THE PENCIL AND PAPER.

TAKEAWAY THIS! 5

HELLO CLOUD, IT’S NICE TO MEET YOU

INTRODUCING THE CLOUD

▸ What is the cloud?

▸ The cloud is simply the abstraction of infrastructure and compute capabilities

▸ Where do you buy the cloud?

▸ Amazon is many times larger than the rest of the market

▸ Microsoft, Google are growing quickly

▸ Everybody else

6

IT’S CHANGING EVERYTHING FOR A REASON, ACTUALLY A LOT OF REASONS

WHY THE CLOUD

▸ Power

▸ Cost

▸ Scalability

▸ Security

▸ Flexibility

▸ Speed to delivery

7

JUST BECAUSE THE EXCUSE IS COMMON DOESN’T MEAN IT IS GOOD

WHY RESIST CLOUD

▸ Security

▸ Loss of control

▸ Lack of information

▸ Lacking technology skills

▸ Misaligned incentives

▸ Vendor Lock-In

▸ Change is hard

8

FORGET WHAT YOU KNEW ABOUT THE CLOUD 3 YEARS AGO. TODAY’S CLOUD TECHNOLOGIES OFFER RECENTLY- UNBELIEVABLE CAPABILITIES.

TAKEAWAY THIS! 9

EVER BEEN IN A SALES PRESENTATION WHERE THEY REFUSE TO ACTUALLY SHOW YOU THE TOOL? I HATE THAT.

DEMO: LET’S LOOK AT THE CLOUD

▸ The AWS Console

▸ Compute

▸ Database

▸ Workflows

▸ Automation

10

WHOA, LOOK AT THAT DATA — IT’S ENORMOUS

BIG DATA

▸ Not the term, but when did big data first occur?

▸ What constitutes big data?

▸ Volume - Variety - Velocity - Veracity - Value - Verifiability

▸ Somebody was trying too hard

▸ Big data is a scale challenge

▸ How many organizations are perfect leveraging data that does not have “big data” scale challenges?

11

“BIG DATA” IS A RELATIVE TERM. IT HAS ALWAYS EXISTED. TODAY’S BIG DATA IS TOMORROW’S AFTERTHOUGHT.

TAKEAWAY THIS! 12

THERE’S NOTHING LIKE SWIMMING IN A DIRTY DATA LAKE

DATA LAKES

▸ When do you perform data governance, data quality, etc.

▸ Ingest or Use?

▸ Data lakes are a dumping ground for unrefined, minimally-governed data

▸ Is this bad or good?

▸ Depending on how you draw the lines, we have all been using data lakes for a long time

13

DATA LAKES SHIFT GOVERNANCE AND REFINEMENT FROM INGEST TO TIME-OF-USE.

TAKEAWAY THIS! 14

BECAUSE “STATISTICS” IS THE “PATAGONIAN TOOTH FISH” OF BIG DATA

DATA SCIENCE

▸ Data science has almost as many variants as actual science

▸ Data scientist identifies potential value derived from data

▸ Where are the data scientists in your organization?

▸ What a data scientist does:

▸ Statistics (regressions, predictive analytics), data wrangling, data modeling, reporting, business analysis, programming, others?

15

A DATA SCIENTIST IDENTIFIES POTENTIAL VALUE DERIVED FROM DATA.

TAKEAWAY THIS! 16

IF REAL CLOUDS COULD BE PRIVATE, LARRY ELLISON WOULD LOCK THEM ALL UP ON HIS ISLAND.

PRIVATE CLOUD

▸ “Private cloud” is when a single organization deploys computing resources structured to mimic the engagement model of the cloud

▸ What are the benefits?

▸ Full Control

▸ Arguably more security, if you believe you can do it better than Amazon, Microsoft, or Google

17

I’M SURE MOST OF YOU DIDN’T FIND THAT LAST ONE FUNNY, BUT I THOUGHT IT WAS HILARIOUS.

PRIVATE CLOUD (CONTINUED)

▸ What are the drawbacks?

▸ Power

▸ Cost

▸ Scalability

▸ Security

▸ Flexibility

▸ Speed to delivery

▸ Private clouds diminish or eliminate all the benefits of the cloud!

18

THERE IS NO SUCH THING AS A PRIVATE CLOUD.

TAKEAWAY THIS! 19

YOU THINK WE HAVE A LOT OF DATA NOW? YOU AIN’T SEEN NOTHING YET.

INTERNET OF THINGS

▸ This is what comes after “big data”

▸ Sensors, logs, events, locations, beacons

▸ This is granularity-driven evolution

▸ Where does IoT fall on the structured/unstructured spectrum?

20

INTERNET OF THINGS IS ALL ABOUT EXPONENTIAL GROWTH IN SMALLNESS.

TAKEAWAY THIS! 21

YOU DON’T TELL THE PERSON WITH THE BIGGEST GUN TO STOP AIMING

GOVERNING DATA IN THE CLOUD

▸ Data governance has never been more important

▸ Tools evolve, hopefully principles last

▸ The people challenge is the toughest

▸ The goal of data governance should be:

▸ Add more benefit than the cost

▸ Shouldn’t that be everybody’s goal?

22

“DATA” GOVERNANCE IS MISLEADING. DATA CAN’T BE GOVERNED: IT HAS NO FREE WILL.

TAKEAWAY THIS! 23

IN A PERFECT WORLD THIS SLIDE WOULD ALSO BE UNNECESSARY. UNFORTUNATELY, OUR ENTERPRISES OFTEN MISS THIS STUFF.

OPEN SOURCE

▸ Who uses open source software?

▸ Is anybody prohibited from using open source in their organization?

▸ Who uses GitHub? Why?

▸ As we move into evolving our businesses, we need to question policies and find efficiencies everywhere we can

24

THE PROBLEM YOU ARE SOLVING HAS BEEN MOSTLY-SOLVED ALREADY.

TAKEAWAY THIS! 25

PEOPLE LOVE TO CLAIM AGILE WHEN THEY ARE SO NOT. IT’S LIKE THE OPPOSITE OF EVERYONE WHO INSISTS THEY DO NOT MICROMANAGE

AGILE

▸ Agile is a philosophy promoting iterative design, development, and collaboration to promote a more nimble and responsive SDLC

▸ Whose organizations are “agile” shops?

▸ Whose organizations are “waterfall” shops?

▸ Are there any other SDLC management standards in use?

▸ Do we think Agile has any more or less impact/value for cloud environments?

26

SORRY IF THIS LOOKS LIKE A TYPO. IT REALLY ISN’T.

DEVOPS

▸ What is DevOps?

▸ How many of us work in shops with a heavy DevOps emphasis?

▸ How does data stuff fit in with DevOps?

▸ Book recommendation: “The Phoenix Project,” by Gene Kim, Kevin Behr, and George Spafford

27

DEVOPS TREATS INFORMATION TECHNOLOGIES LIKE THINGS THAT MATTER TO THE BUSINESS. NOVEL CONCEPT, THAT.

TAKEAWAY THIS! 28

BUSINESSES ARE EITHER DATA-DRIVEN…OR DOOMED

TODAY’S DATA-DRIVEN BUSINESS

▸ How do our businesses use data?

▸ Value of Data

▸ Increase Top Line

▸ Decrease Bottom Line

▸ Manage Risk

▸ My favorite question

▸ If I give you this, what will you do differently?

29

DATA’S MOST IMPORTANT ROLE IS TO INFORM YOUR BUSINESS ON HOW TO GET BETTER AT WHAT IT DOES.

TAKEAWAY THIS! 30

A VIRTUOUS CYCLE IS THE OPPOSITE OF A DEATH SPIRAL

THE VIRTUOUS CYCLE OF THE CLOUD

31

MEASURE

IDENTIFY IMPROVEMENTSIMPROVE

CLOUD DONE RIGHT CREATES A VIRTUOUS CYCLE THAT WILL TRANSFORM YOUR BUSINESS.

TAKEAWAY THIS! 32

THERE ARE ONLY TWO CHOICES: CHANGE OR FAIL

OVERCOMING OBJECTIONS

▸ “We need to focus on operations”

▸ “We can’t put that much data in the cloud”

▸ “The security concerns make this a non-starter”

▸ “Costs will be uncontrollable”

▸ “We want to avoid vendor lock-in”

▸ “We already invested a lot in our infrastructure”

33

THE SUCCESS OR FAILURE OF YOUR CLOUD INITIATIVES WILL RELY MOST ON YOUR UNDERSTANDING AND APPEALING TO EACH STAKEHOLDER’S PERSONAL INTERESTS.

TAKEAWAY THIS! 34

WE ARE REALLY CLOSE TO Q&A AND CUTE PICTURES OF MY KIDS

PUTTING IT ALL TOGETHER

▸ Is cloud the future?

▸ What are our responsibilities in all this?

▸ What is really holding us back?

▸ Find ways to stop talking about working and start working

▸ Start by doing something small

35

THIS IS STILL THE MOST IMPORTANT SLIDE OF THE DECK

RECALL: IF YOU REMEMBER NOTHING ELSE

▸ Cloud technologies have already changed everything - hardware and processing constraints are disappearing

▸ The technologies and tools have evolved from what we have used for decades

▸ This is mostly a people and organizational leadership challenge

▸ The virtuous cycle:

▸ Measure - Identify Improvements - Improve - Repeat

▸ It is not too late to evolve our businesses, and it is up to us

36

37

ANY QUESTIONS?

GETTING OUR HEADS INTO THE CLOUDS: INFINITELY SCALABLE DATA MANAGEMENT

Somebusinessesaredealingwithmountainsofpoor-qualitydata.Othersaredealingwithmountainsofpoorly-defineddata.Andothersaredealingwithactualmountains.Thougheachorganiza?onfacesuniquechallenges,allorganiza?onsneedtobeusingdatatogetbeAeratwhattheydo.Andasdata-drivenprofessionals,wemusthelpourorganiza?ons’mountainsreachtheclouds.

Thecloudisnotthefutureofenterprisecompu?ng:it’stheNOWofenterprisecompu?ng.

Weareinthemidstofadataawarenessrenaissance,andourorganiza?onsaredemandingthatwebuildanaly?cscapabili?estodrivefuturesuccess.Cloudtechnologiesarepartoftheanswer,buttofullyrespondtothesechallengesweneedtobringnewmindsetsandtechniquestodatamanagement.

It’suptousEDWaAendeestobringthesecapabili?estoourbusinesses.Ifyourbusinessisnotalreadythere,itneedstobe.Cometothissessiontolearnhowtogettothecloud,andhowourdatamanagementprac?cesmustevolvetoharnessthepower.

Thissessionwillcover:

• Apragma?cfounda?onalperspec?veonhotdatatermsyouhaveheardabout,like:

o BigData-arela?vescalechallengethathasalwaysexisted

o DataLakes-op?mizingdatastructuringenergyacrossplace,?me,andpurpose

o TheCloud-forgetwhatyouknewaboutitthreeyearsago

o PrivateCloud-likebuyinganairplaneversusflyingcommercial

o InternetofThings-thiswillmakecurrentBigDatascalelaughableinthenextfewyears

o DataScience-acapabilityfartooimportanttobeleQen?relytodatascien?sts

• Whythecloudmakessomenewthingspossible,manythingseasier,andafewthingsmuchharder–andhowweneedtoadaptourdatamanagementprac?cesinresponse

• WhybusinessimpactistheonlythingthatreallymaAers

• Whatallthismeansforbusinessasusual,andhowtobuildahigh-performanceorganiza?ondrivenbydatainthecloud

38