Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns,...
-
Upload
arline-campbell -
Category
Documents
-
view
226 -
download
0
Transcript of Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns,...
![Page 1: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/1.jpg)
Measuring What MattersEducause 2007
Craig Blaha, Information Technology ServicesDavid Burns, McCombs School of Business
Copyright Craig Blaha and David Burns, 2007. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright
statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
![Page 2: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/2.jpg)
Abstract
Determining what to measure, how to measure it, and to whom to report the results can be more of an art than a science. In this session, four major research universities
will address how to communicate the importance of IT to administration, outreach, teaching and learning, and
research.• Project Overview•Lessons Learned•Moving Forward
![Page 3: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/3.jpg)
Who We Are• Brian Busby
Collaborative Applications ManagerUniversity of Wisconsin-Madison
• Phil DevanSr. Director Network Research ServicesThe Pennsylvania State University
• Geoff LakemanDirector, Network Engineering & Technical ServicesUniversity of Washington
• Jim LoweChief Information Security OfficerUniversity of Wisconsin-Madison
• Shanna Smith, PhDDivision of Statistics and Scientific ComputationUniversity of Texas
![Page 4: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/4.jpg)
What We Did
• Ambitious Plan– Account for all IT spending – Develop service catalogs– Identify key metrics
• Focus On Central IT Organizations• 4 Stakeholders Of Higher Education
– Administration, UW-Madison– Outreach, Penn State– Research, University of Washington– Teaching & Learning, University of Texas
![Page 5: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/5.jpg)
Approach
• Interview key stakeholders outside of IT1. From your perspective what are the goals, strategies or
objectives you are striving to achieve to make the university a better learning and working environment (what matters)?
2. What measurements are you using to gauge the progress towards achieving these goals (how do you know you are succeeding)?
3. From your perspective, how does IT help you make your work successful (role of IT)?
4. What do you think IT should be measuring and why?
…This would be where things went off the rails.
![Page 6: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/6.jpg)
What Happened
• Discoveries– People hadn’t had a conversation like this with
someone from IT… EVER!– Metrics don’t matter?!?
• Goals• Barriers• Role of IT
![Page 7: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/7.jpg)
Goals
• Teaching and Learning– Reasoning , moral & ethical
thinking– Computer competency– Problem solving– Teamwork and
communication
• Outreach– Research, teaching, service– Leverage talents of
university
• Research– Research grants & contracts,
published work– Positive educational &
economic impact– Collaboration
• Administration– Efficiency, transparency,
integrity– Collaboration and
communication– Savings
![Page 8: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/8.jpg)
Barriers
• Teaching and Learning– Many faculty married to
old learning models– Individualized, flexible
learning model is difficult with one teacher and many students
• Administration– Resistance to change– External forces– Politics and minority
stakeholders
• Research– Funding– Improving access to
resources– Available infrastructure– Highly competitive
• Outreach– Funding– Marketing/perception– Geography– Little involvement locally
![Page 9: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/9.jpg)
Role of IT
• Teaching and Learning– Accessibility– Resource-saving– Interaction & collaboration– On-the-spot information– Student perceptions
• Administration– It’s not the technology – it’s
what we do with it– IT helps us provide services– Stakes are high - success of
IT is tied to success of administrative organization
• Outreach– Enabler and enhancer
• Videoconferencing for problem solving
• Fundamental to World Campus– Much of it should be common
good• IT is fundamental• Like a utility
• Research– Research grants depend on
technology infrastructure– Sharing IT resources from
around the world promotes big science
![Page 10: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/10.jpg)
“Not in the least bit interested, metrics become the goal”
![Page 11: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/11.jpg)
• Themes– Communication– Collaborative Metrics– Alignment
![Page 12: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/12.jpg)
Communication
![Page 13: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/13.jpg)
Communication
• “No news is good news”• Better communications from IT
– What’s going on?– Lay language– “IT has A real marketing problem”
• Listen– Understand the real mission
• Laying the groundwork– Establishing a relationship– Working towards goals, metrics, and alignment
![Page 14: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/14.jpg)
Essential Questions• What is the value derived from IT?• What are the results that matter?
– Measuring performance against desired outcomes– Converting expectations to service standards
• How are these results measured?– Types of performance measures
• Reliability measures• Responsiveness measures• Project measures• Utilization/adoption• Client satisfaction
![Page 15: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/15.jpg)
A Path Forward
![Page 16: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/16.jpg)
Collaborating On Metrics
• Leadership from IT required• Starting the conversation about metrics
• Present a metric• Discuss that metric regularly
– Especially progress or problems
• See if it ends up being useful• Wash, rinse, repeat.
• It doesn’t matter unless you measure it• If it doesn’t matter – DON’T measure it!
![Page 17: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/17.jpg)
Money Matters
Worth The
Cost?
Measures of IT changes
Measures of mission changes
Itemized IT Expenses
![Page 18: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/18.jpg)
Current Metrics
• Reliability Measures– Quality = no errors– Uptime
• Utilization/Adoption– Blackboard usage statistics
• Client Satisfaction– Blackboard satisfaction– Annual survey
• Responsiveness Measures– Desktop support
• Project Measures– Increased funding– Technology licensing– Budgets and timelines– Statements of work
![Page 19: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/19.jpg)
Goals and Measures
• Goals– Operational excellence– Communication– Fiscal responsibility
• Recommended Metrics– Project based metrics are fine– Qualitative better than quantitative
![Page 20: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/20.jpg)
Barriers
• Behavior beats metrics• Metrics that don’t matter• Defensive metrics• Fear of accountability
![Page 21: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/21.jpg)
Finding Alignment
![Page 22: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/22.jpg)
Alignment Process
![Page 23: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/23.jpg)
Alignment Process
• Identify Stakeholders– Planning that leaves out the middle
• Their Goals and Measures– What is important to them– How do they measure themselves
• Mapping IT Services and Metrics– What do they use– What measures do you have for those services
• Feedback and Refine– Create ongoing dialog and engagement
![Page 24: Measuring What Matters Educause 2007 Craig Blaha, Information Technology Services David Burns, McCombs School of Business Copyright Craig Blaha and David.](https://reader035.fdocuments.net/reader035/viewer/2022062221/56649d995503460f94a838d8/html5/thumbnails/24.jpg)
Key Points
• Focus on the mission• Different strokes…• Understand customer needs• Metrics are collaborative• Active listening is communication• Strategic planning – mind the gap• Sell IT!