Process Metrics & KPI’s – The vital link to evaluating process effectiveness

Angelo Scangas, President4 Essex Center Drive #3971

Peabody, MA 01960978-430-7611

angelo@qualitysupportgroup.com

©2014 QSG, Inc.

©2014 QSG, Inc.

1. Business & Process Improvement2. PDCA Continual Improvement3. Risk Management4. Process Measurement5. Types of Measures6. Best Practices7. Cynosure - Quality & Regulatory Metrics

Agenda

Business Process ImprovementBusiness process improvement (BPI) is a systematic approach to help an organization optimize its underlying processes to achieve more efficient results.

KPIs – Track Business Process Improvement

Source: Wikipedia

Business KPIsKPIs – Track Business Process ImprovementExamples:

1. Product Margins/Profitability2. Customer Satisfaction3. OTD4. Inventory Management5. Safety6. ?????

Process ImprovementProcess improvement is a series of actions taken by a process owner to identify, analyze and improve existing business processes within an organization to meet goals and objectives, such as increasing profits and performance, reducing costs and accelerating schedules. These actions often follow a specific methodology or strategy to increase the likelihood of successful results.

Source: Wikipedia

KPIs and metrics (drivers)

It Helps to Understand Drivers that Influence Performance of a KPI. Which may be other Metrics

and/or other factors.

A Driver Metric (or other factor) may influence more than one KPI.

10/10/14 QSG Meeting Presentation – G. Lane

Internal Business Processes

Innovation processes

Operation Processes

Quality/Regulatoryprocesses

Sales/Marketing processes

Measures

Standard Business and Process Measures

8

Business Process Measures

Measures of Affectivity

On Time Release for Source Inspection

On Time Kit ReleaseCycle Time for Processes A, C, T, R

% of Documentation available at Release

On Time DeliveryOn Time Delivery to MRP

On Time ReleaseOn Time Approval

Customer SatisfactionOn Time Delivery to Contract

EscapesCustomer CARs

Critical Process ManagementCritical Items Identified

Key Characteristics IdentifiedVariability Management in Place

Product Yield

Process YieldsTest Yields

Inspection YieldsRolling Throughput Yields

PDCA

PDCA (plan–do–check–act or plan–do–check–adjust) is an iterative four-step management method used in business for the control and continuous improvement of processes and products. It is also known as the Deming circle or the Shewhartcycle.

Source: Wikipedia

4.1.3 Process ApproachISO/DIS 9001, clause 0.4, Plan-Do-Check-Act cycle

CustomerSatisfaction

Customers& other

relevantinterested

parties

5 Leadership

8 Operations

6 Planning 9 Performanceevaluation

4.1, 4.2, 4.3Establish context,

define relevantinterested parties& scope of QMS

10 Continual Improvement

7 Support Processes

4.4 QMS- GeneralProcess - Approach

OutputsInputs Products &ServicesRequirements

*

*

4.8 Performance evaluation (section 9)

• Organizations need to plan how they’re going to monitor, measure, analyze, and evaluate their QMS, and then

• Monitor, measure, analyze, and evaluate their organization's QMS

• Clause “8.4 Analysis of data” was substituted with clause “9.1.3 Analysis and evaluation”. It’s obvious from sub-clause title that now beside analysis it is required to do evaluation as well.

ISO/DIS 9001, clause 9.1

4.4.1 Actions to address risks and opportunities

ISO/DIS 9001, clause 6.1

One of the key changes in theISO/DIS 9001 standard is toestablish a systematic approachto risk.

By preventing or reducing the undesired effects, the organization becomes proactive. Preventive action is automatic in risk-based approach system.

Instead of being onlypart of the preventiveaction procedure, risk isconsidered thoughoutthe quality managementsystem.

1

3 2

Pane 2 – Why are we here?

$0.00

$50,000.00

$100,000.00

$150,000.00

$200,000.00

$250,000.00

$300,000.00

$350,000.00

1 2 3 4 5 6 7 8 9

Pane 1 – Where are we?

0.00%

1.00%

2.00%

3.00%

4.00%

5.00%

6.00%

Month

Perc

enta

ge o

f Pai

d Sh

ipm

ents

with

Exc

eptio

n

Run Chart Data Exceptions by Month

Month Paid shipments # Exceptions Exceptions Goal

September FY05 886 43 4.85% 2.50%October FY05 1139 32 2.81% 2.50%November FY05 1103 45 4.08% 2.50%December FY05 1346 48 3.57% 2.50%January FY05 993 38 3.83% 2.50%February FY05 856 36 4.21% 2.50%March FY05 1056 44 4.17% 2.50%April FY06 923 33 3.58% 2.50%May FY06 877 13 1.48% 2.50%June FY06 891 14 1.57% 2.50%July FY06 779 9 1.16% 2.50%Aug FY06 923 7 0.76% 2.50%

Pane 3 – How did we get here? Pane 4 – What do we do now?

Action Plan

Action Resp Date Comment

PDCA - Example

13

CustomersProcess

Products or Services

MeasurementProcess

Dealing withthe Future

Dealing withthe Past

Product Controland Disposition

Process Controland Improvement

Use of Measures

Lead Indicators

• Common examples are:– Market share, backlog (book-to-bill ratio),

new product introductions, new product development lead times, product quality, customer satisfaction, employee morale, personnel development, inventory turnover, bad debt ratio, or safety

Lagging Indicators• In contrast to lead indicators, lag indicators

are measures that point to earlier plans and their execution.

• Financial performances are lag indicators.• Many times, financial performances are too

late to affect future products and services.

• Understand Stakeholder needs/requirements – business, customer, regulatory, legal, etc.

• Use value stream mapping to identify key processes/points for measures.

• Develop a process for collecting, recording, and analyzing data.

• Try the measure and adjust as necessary.Remember – A measure is as good as the actions it identifies for improvement

Developing Measures

• Where can we measure the process?

• What are pros and cons of the above locations?

• Where should we start??

Selecting MeasuresOutputs

CustomersProcessProducts or Services

GaugeGauge Gauge

In-Process Final Product Customer Satisfaction

Gauge

Inputs

Measures that do more harm then good

???

Thank you !!

Angelo Scangas, President4 Essex Center Drive #3971

Peabody, MA 01960978-430-7611

angelo@qualitysupportgroup.com

©2014 QSG, Inc.

The Power of MetricsHow Data Drives Decisions

Connie HoyJanuary 2015

Why?

• Cynosure uses metrics for 2 reasons:

– Drive Business decisions• Where do we need new employees?• What products need to have reliability improvements?• Do we have the right tools to drive training?

– Measure how we comply with the quality policy

Cynosure, Inc. strives to achieve satisfactionfor our customers and patients through ourinnovative products and services. Cynosure isa leader in the design and manufacture ofhigh quality, reliable products that are safeand effective. Cynosure is committed tocomplying with regulatory requirements andmaintaining and improving the effectivenessof our quality management system.

Quality Policy

Mapped to objectives and metrics

Cynosure, Inc. strives to achieve satisfactionfor our customers and patients through ourinnovative products and services. Cynosure isa leader in the design and manufacture ofhigh quality, reliable products that are safeand effective. Cynosure is committed tocomplying with regulatory requirements andmaintaining and improving the effectivenessof our quality management system.

Satisfied Customers

Service Response Time (All Laser Consoles)

Objective: Respond under 2 Days

Service First-Call Resolution (All Laser Consoles)

84%

84%

85%

85%

86%

86%

87%

87%

88%

88%

89%

Q1 2012 Q1 2013 Q1 2014

1st Call Resolution RateQ1 2012 vs. Q1 2013 vs. Q1 2014 1st Call Resolution Rate

Service Calls per Warranty Laser (Not real data)Q1 2013 YoY Q1 2014

*Cynergy Units under Warranty: 4

How do we use this data?

• Drives the decision on what the sustaining team should work on

• Prioritizes the sustaining projects based on:• Safety• Customer Nuisance• Cost of issue

Satisfied Patients

• MDRs• 18 MDRs submitted to the FDA in 2013

• Of these, 1 was also reported to Canada (MPR)• 16 patient-related• 2 product-related (CAPAs issued)

• 8 MDRs submitted to the FDA in Q1 2014• All patient-related

• Implemented improvements in the reportabilityreview process, including clearer guidelines to ensure reportability determinations are accurate or otherwise justified.

20 MDRs in this period7 due to product issue

13 due to user error and/or serious injury

Monthly MDRs Submitted

High Quality and Reliablility

92.6%96.5%

85.3% 87.6%93.3% 94.5% 93.3%

96.9%

83.8%

71.9% 73.1%

64.9% 67.6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

50%

60%

70%

80%

90%

100%

Q2 - 12 Q3 - 12 Q4 - 12 Q1 - 13 Q2 - 13 Q3 - 13 Q4 - 13 Q1 - 14

Overall Cynosure Training Status(North American Employees)

Overall Completion Status Complete Within 30 Days

• Note: Began measuring “on-time” rates in Jan 2013. Status is # of completed requirements against total active training requirements. Cycle is % completed “on time” (within 30 days / per due date).

Target = 100%

Material Review Board (MRB) Process (Westford and Fremont, Burlington Dec ‘13 to Present)

Total Disposition Dollars:Q4 2013 = $190,508.30

Total Disposition Dollars:Q1 2014 = $268,496.95

Accolade, Cynergy/Vstar, Elite MPX, PicoSure Lasers (Westford)

6) Process/Product/Supplier PerformanceAcclaim, Apogee/Elite MD, Apogee/Elite+ (Westford/Cogmedix)

Safe

Patient Impact Complaints –not real data

• 63 complaints (2013) vs. 69 complaints (2012)• 38 complaints (Q1 2014)• Improvements to complaint investigation process: QA and Clinical working closely

together to gather and evaluate patient data.

Laser Type 2012 2013 Q1 2014Accolade 1

Affirm 4 2Cynergy 4 1Elite + 2

Elite MD 23 20 6Elite MPX 10 17 7

Pico 1Medlite C6 3 3 1Revlite/SI 1 5Affinity 1

CO2 3SLT II 9 9 5

SmartSkin 1 2SmartXide 1

Adivive 1Icon 1 8

StarLux 500 2 1Vectus 2

8) Customer Feedback

• MDRs• 18 MDRs submitted to the FDA in 2013

• Of these, 1 was also reported to Canada (MPR)• 16 patient-related• 2 product-related (CAPAs issued)

• 8 MDRs submitted to the FDA in Q1 2014• All patient-related

• Implemented improvements in the reportabilityreview process, including clearer guidelines to ensure reportability determinations are accurate or otherwise justified.

20 MDRs in this period7 due to product issue

13 due to user error and/or serious injury

Monthly MDRs Submitted

Compliant

Q4 2013 – Q1 2014: CAPA Distribution to FDA 21 CFR 820 (QSR)(All Sites)

N = 30

End