LOPSA East 2013 - Monitoring for the Public Good

Monitoring for the Public Good

Mike Julian

Friday, May 3, 13

Who am I?

• ORNL operations staff

• LOPSA Technical Staff

• Generalist IT engineer for ~10 years

Friday, May 3, 13

Who am I?




• I am obsessed with monitoring.

Friday, May 3, 13

Who am I?




Friday, May 3, 13

Who am I?




• I am obsessed with data.

Friday, May 3, 13

A dashboard is a visual display of the most important information needed to

achieve one or more objectives; consolidated and

arranged on a single screen so the information can be monitored at a glance.

What is a dashboard?

* Stephen Few, Intelligent Enterprise

Friday, May 3, 13

The highlighted bits are what I think are the most important points to remember.

But first...

You need some data!

Friday, May 3, 13

Visualization is a lot of fun, but it’s no use if you don’t have the right data to display. Finding the data is even more fun, and potentially much more challenging.

So, what data should you get. That depends on what you need to show.

Let’s say you wanted to show...

• Bandwidth usage: border router(s), SNMP query

• Peak usage hours: databases, network traffic, VPN usage

• User demographics: lots of database work.

• There’s always the case that the data may not exist.

Friday, May 3, 13

These are just a couple examples of information you might want to show, and where you might find the data for them.

Your data is all over the place

• ERP, CRM

• Your ticket system

• Product databases

• Google Analytics

• Operational monitoring systems (eg, Nagios)

Friday, May 3, 13

Your data is stored in lots of places. Google Analytics, your operational monitoring systems, various databases...

This is where the engineer in you shines: use your language of choice, SQL, and knowledge of your infrastructure to find the data and do the necessary magic on it to turn it into meaningful information.

Building information dashboards: an exercise in not thinking like an ops

engineer.

An exercise in not thinking Ops

Friday, May 3, 13

“What gets measured, gets managed.” - Peter Drucker, management guru

Friday, May 3, 13

Key Performance Indicator (KPI)A way to measure the progress

of a service, program, or initiativetowards the goals.

Friday, May 3, 13

The purpose of the KPI is to drive change.How do you know you’re doing better or worse without the numbers?

KPIs differ depending on use. Bandwidth utilization is a KPI for a datacenter, but perhaps not for a doctor’s office.

In fact, they differ within companies. New customer acquisitions is a KPI for marketing and sales, but not for IT. Likewise, Finance doesn’t care about utilization metrics of the ERP system.

KPI examples

Friday, May 3, 13

You are the CEO of a web hosting company. What are your KPIs?

KPI examples

Friday, May 3, 13


•Total sales (dollar amount)•Total expenses (dollar amount)•Profit margin (percentage)•Number of new customers•Number of cancellations

KPI examples

Friday, May 3, 13



You are the customer service manager for a call center.

What are your KPIs?

KPI examples

Friday, May 3, 13



You are the customer service manager for a call center.

What are your KPIs?•Call volume•Call length•Queue wait time•Number of call abandons

KPI examples

Friday, May 3, 13

Let’s build a dashboard!

Friday, May 3, 13

Let’s build a dashboard!LOPSA Board of Directors Dashboard

Friday, May 3, 13


KPIs

Friday, May 3, 13


Membership•New members•Lapsed members•Member satisfaction

KPIs

Friday, May 3, 13



Financial•Program income•Contribution income•Program expenses•Operating expenses

KPIs

Friday, May 3, 13




Locals Program•Chapter attendance•New chapters•Conversions

KPIs

Friday, May 3, 13




Locals Program•Chapter attendance•New chapters•Conversions

Mentorship Program•Pairing requests•Pairing assignments•Pair ratings

KPIs

Friday, May 3, 13

Dashboard design tips

• Know your audience

• Only display relevant data

• Both to your audience and your topic

• Watch your granularity

• Keep it simple

Friday, May 3, 13

Your audiences care about different things. Even among IT teams, different metrics matter. Keep this in mind.

Granularity: a dashboard is high-level. Look for the balance between showing enough information and too much information.

Keep it simple. Eye-candy is great, but don’t overdo it: the purpose of a dashboard is to answer questions quickly.

(numbers used not actual values)

Friday, May 3, 13

Visualization was done with D3.js

What does YOUR boss care about?

Friday, May 3, 13

Let’s try another...

Friday, May 3, 13

KPIs do not have to be quantitative. Sometimes they’re qualitative.

Think of things this way: what information would your boss wish he had in a staff meeting with other managers? What information shows how well your department is doing?

Let’s try another...An IT Manager’s dashboard

Friday, May 3, 13



KPIs:•Support request volume•Incident resolution time•Customer satisfaction (survey)•Infrastructure availability %•Significant system events•Upcoming changes

Let’s try another...An IT Manager’s dashboard

Friday, May 3, 13



That’s cool and all, but we’re engineers...

Friday, May 3, 13

But Mike, we’re engineers! What has all this got to do with us?

No problem!

Friday, May 3, 13

It’s got everything to do with you!

This has given you some insight into how your boss and executives think. If you start answering their questions before they can even ask them, what do you think they’re going to do? You’ve just become more valuable to them, and you’ve developed a crucial skill.

Remember, IT supports the business.

While you’re there... (or: interesting data)

What else is in your data that no one has thought to ask about?

Friday, May 3, 13

In the course of working on LOPSA projects, I realized an interesting project could be built.

LOPSA Membership Heatmap

* No PII was used in the creation of this map

Friday, May 3, 13

Don’t worry, no personally identifiable information was used in the creation of this map.

This map is fairly obvious: a heat map of active LOPSA members. It’s done in the Google Maps API. On the live version, you can zoom in and see more granularity. At this zoom level, LA and San Diego are just one blob, while they are clearly distinct at a closer zoom.

I just used a SQL query, Python, and Javascript for this.

Resources• Information Dashboard Design by Stephen Few

• Actually, anything by Stephen Few.

• What Your CEO Wants You to Know by Ram Charan

• For learning how to think like management

• A Practical Intro to Data Science - Zipfian Academy blog

• Deep intro into data science--for those who really want to get into it

• D3.js (d3js.org)

• Awesome visualization library

Friday, May 3, 13

That’s all, [email protected]

Friday, May 3, 13

mailto:[email protected]

mailto:[email protected]

Thank You for Attending LOPSA-East '13

Please fill out the Trainer Evaluation

http://lopsa-east.org/2013/training-survey

Rate LOPSA-East '13

http://www.lopsa-east.org/2013/rate-lopsa-east-13

Friday, May 3, 13





LOPSA East 2013 - Monitoring for the Public Good

Technology

Transcript of LOPSA East 2013 - Monitoring for the Public Good