Measurement as-a-service a new way of organizing metrics programs - wilhelm meding

Slide title 70 pt

CAPITALS

Slide subtitle minimum 30 pt

Miroslaw Staron, Associate Professor, Software Engineering Wilhelm Meding, Senior Measurement Program Leader, Ericsson

Measurement As A Service

Slide title 44 pt

Text and bullet level 1 minimum 24 pt

Bullets level 2-5 minimum 20 pt

Characters for Embedded font: !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™ĀĀĂĂĄĄĆĆĊĊČČĎĎĐĐĒĒĖĖĘĘĚĚĞĞĠĠĢĢĪĪĮĮİĶĶĹĹĻĻĽĽŃŃŅŅŇŇŌŌŐŐŔŔŖŖŘŘŚŚŞŞŢŢŤŤŪŪŮŮŰŰŲŲŴŴŶŶŹŹŻŻȘș−≤≥fifl

ΆΈΉΊΌΎΏΐΑΒΓΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΪΫΆΈΉΊΰαβγδεζηθικλνξορςΣΤΥΦΧΨΩΪΫΌΎΏ

ЁЂЃЄЅІЇЈЉЊЋЌЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯЁЂЃЄЅІЇЈЉЊЋЌЎЏѢѢѲѲѴѴҐҐəәǽẀẁẂẃẄẅỲỳ№

Do not add objects or text in the footer area

Measurement As A Service | 2015-09-25 | (25)

› Measurement As A Service – Overview › Benefits of MaaS › Enablers of MaaS › Sum-up

Content

Slide title 44 pt








› How to support the company’s core business processes, by optimizing its measurement program?

› MaaS is a measurement licensing and delivery model, in which metrics are licensed on a subscription basis, centrally hosted, collected and delivered on demand

› By having MaaS in place we: – reduced the cost of measurement by 50% – reduced the reporting time by 95% – increased uptime to 100%

Addressing the challenge of optimizing measurement programs

Slide title 44 pt







Measurement As A Service | 2015-09-25 | (25) Measurement-as-a-Service

Requirements Development Release

Business processes

Extracts

Measurements

Metrics team

Develops and

maintains

Provide measurements

Stakeholder

Knowledge base

Uses and maintains

MetricsCloud - Metrics, infrastructure, licensing, delivery

Slide title 44 pt








Benefits of MaaS

›  Unification –  Common Metrics language –  Common view of what main metrology terms stand for –  Measurement process –  Infrastructure

›  Scalability –  Small team supporting large organization

›  Many measurement systems developed and maintained ›  Centralization of storage ›  Deliveries ›  Accessibility ›  Maintenance ›  Support

›  Agility –  Keeping up with the evolution of technology and ways of working

›  Elevated company metrics competence –  Knowledge base –  Value proposition –  Advanced analytics

Slide title 44 pt








Enablers of MaaS

›  International standards, e.g. ISO/IEC 15939, ISO 25000 › Automation

– Information collection – Information processing – Self-healing

› Trust – Information Quality – Security – Dependable Information products

› Cloud based storage and dissemination mechanism › Dedicated metrics team

Slide title 44 pt








Working with standards e.g. ISO/IEC 15939

›  ISO/IEC 15939:2007(E) –  Identifies activities for collecting, analyzing, reporting and evaluating measurement data –  Provides definitions of measurement terms –  Supports effective management of the measurement process –  Is adopted by the IEEE Std 15939 – 2008

Activity

Data store

Flow

Legend

Measurement Process Model

* ISO/IEC 15939:2007 (E)

Slide title 44 pt








Measurement Information Model

Attribute Attribute

Measurement Method

Measurement Method

Base Measure

Base Measure

Measurement Function

Derived Measure

Derived Measure

(analysis) Model

Indicator

Interpretation

Information Product

Measurement Concept

Entity

Information Need

The outcome of the measurement process that satisfies the information needs

Explanation relating to quantitative information in the indicator to the information needs in the language of the measurement users

Variable assigned a value by applying the analysis model to base and/or derived measures

Algorithm for combining measures and decision criteria

Variable assigned a value by applying the measurement function to two or more values of base measures

Algorithm for combining two or more base measures

Variable assigned a value by applying the method to one attribute

Operations mapping an attribute to a scale

Property relevant to information needs

Object that is to be characterized by measuring its attributes

Insight necessary to manage objectives, goals, risks and problems

Measurement Information Model

- “~ is a structure linking information needs to the relevant entities and attributes of concern” – from Annex A (ISO/IEC 15939)

- Defines measurement constructs which link information needs with attributes of entities

Slide title 44 pt








Example of an instance of the Measurement Information Model

Slide title 44 pt








› Metrology – measurement theory – Sound scientific foundation

› Measurement Information Model

– Generic code ›  New concepts established

in the organization – Stakeholder – Information Need – Indicator

Planning & Developing standardized

Slide title 44 pt








Project “X” status report: April

Main areas Status Comment Overall Planning We keep the time plan so far. Test status may cause delays.

Requirements X1 out of Y1 requirements have been reviewed X2 out of Y2 requirements are linked to test cases X3 out of Y3 requirements have test cases in ”passed”

Configuration Management

Bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, bla, bla,

Defect status

Defect backlog not decreasing

Test progress Function testing: according to plans System testing: behind schedule Network testing: lack of resources

Costs Within budget

Slide title 44 pt








Project “X” status report: April

Slide title 44 pt








How do we know that “green” is “green”?

Attribute Attribute

Measurement method

Measurement method

Base Measure

Base Measure


Derived Measure

Derived Measure

(analysis) Model

Indicator

Interpretation

Information

Measurement Concept

Entity

Information Need

Check the sorting criteria

Check that the export file was updated Check the structure of the database export Check that export scripts were run without errors

Check that there is a value Check that the value is within stated boundaries

Check access to all BMs

Check that there is a value

Boundary conflicts / model consistency

Check that there is a value Check access to all DMs Check that the correct color is set

Attribute Attribute

Measurement method

Measurement method

Base Measure

Base Measure


Derived Measure

Derived Measure

(analysis) Model

Indicator

Interpretation

Information Product

Measurement Concept

Entity

Information Need

Check that the value is within stated boundaries

Check that the value is within stated boundaries

“Trust” indicator “Trust” indicator

Check that databases were updated

Check access to database export

Check access to database

We check “states” and “transitions”

We summarize the “reliability” in one binary indicator

Slide title 44 pt








Measurement Information Products

Slide title 44 pt








Source files Collection

Information

process

Information

Product

Source files

Source files

Many data points Few data points

Measurement System

Measurement Program

People

Projects

Products

Slide title 44 pt








Simple measures

Complex measures

Signals

db

1

db

x

db

2

db

3 ...

Stakeholders

Measurement Program, cont.

Slide title 44 pt








Infrastructure and common language set-up à Simplistic measurement system

›  Measurement system is a set of interrelated and interacting elements necessary to achieve metrological confirmation and continual control of measurement processes

›  Can be realized using simple tools as MS Excel

›  Can be standardized using ISO 15939

Slide title 44 pt








Execution environment, storage, configuration

Measurement Execution Software System (MESS)

Web server

Measurement system

Information product

Raw data files

Databases

Products

People

Execute Update status

Base measures

Derived measur

es

Indicators

Green

Red Yellow

Measurement Program, cont.

Slide title 44 pt








Self-healing measurement infrastructure

›  Algorithms that automatically recover metrics

›  Increased throughput of metric teams

›  Increased data quality ›  Reduced maintenance cost

Slide title 44 pt








Automation Simplified maintenance

Slide title 44 pt








Storage and access

Execution and Information Quality

Information product provisioning and licensing

DB Web server Storage

Slide title 44 pt








Research Development

Maintenance

Dedicated Metrics Team The Ericsson metrics team

Support

Slide title 44 pt








› MaaS is a measurement licensing and delivery model, in which metrics are licensed on a subscription basis, centrally hosted, collected and delivered on demand

›  Benefits

– Higher quality of metrics – Lower cost – Faster adoption of new metrics

›  Bare minimum – Standards – Automation – Company specific

Measurement as a Service Sum-up

Measurement as-a-service a new way of organizing metrics programs - wilhelm meding

Software

Transcript of Measurement as-a-service a new way of organizing metrics programs - wilhelm meding