Strategies for Implementing Successful

Laboratory Automation

Maximizing Return on Investment

Agilent Automation Solutions

Ian YatesGlobal Sales Support and Applications DirectorApril, 13, 2011

ian.yates@agilent.com

April 21, 20111

Introduction

Have been involved in laboratory automation for twenty years• GlaxoWellcome (UK), AstraZeneca (Sweden), Velocity11 (US), Agilent

(Canada) and consulting

- Experience with many automation vendors• Beckman, CRS (Thermo), Tecan, Cybio, RTS, Velocity11 (Agilent), Zymark (Caliper),

and many instrument providers

• Have been part of many (150+) successful implementations…..

• … but have also seen and made my share of mistakes!

Disclaimer

Based on a true story

The identity of people, places and organizations may have been changed to protect the innocent!

Workshop Overview

Why, When and What to automate?• Benefits of automation

• The “real” benefit - Return on Investment

• Industry perception of ROI

How to automate?• Tips for maximizing ROI

Questions / Discussion

Key Benefits of Automation

Why Automate?

At some point, you cannot do it well without automation!• Maintain data quality with increased sample capacity.• Consistency scales better with automation.

Benefits of Automation

Benefits of Automation

Enabling Science

See the whole interview: http://www.chem.agilent.com/en-us/products/instruments/automation/pages/biocel1800-free-cd.aspx

“We can tackle questions we couldn’t physically do before in terms of scale, efficiency, and, really, the depth of the scientific question….”

~Todd Kinsella, Director of Functional Genomics, Rigel PharmaceuticalsOn the benefit of BioCel automation with integrated FACS to his lab.

Benefits of Automation

Workflow coverage

E.g. NGS Sample Prep

Combination of multiple sample prep steps

Combined workflows

E.g. Stem cell culture and high content screening system

Benefits of Automation

Data qualityPeople make mistakes, no matter how well trained, experienced, motivated, alert and focused they are.

A1 !!!

Number of Errors / Events per Month**

Error / event classification

Pre FE500 Post FE500*** % Reduction

Sorting and routine errors

10,192 2,038 80.0

Pour-off errors 3,072 614 80.0

Labeling errors 9,011 1,800 80.0

Biohazard exposure events

3,126 24 99.3

* Study performed at Milton S. Hershey Medical Center, Pennsylvania State University.** Estimated for one month’s period of data collection.*** Errors recorded in this category were from manually-handled samples for STAT testing plus those samples processed for microbiology and virology testing.

.

Clin Chem 48(3):540-548, March 2002

Error ratesStudied mainly in clinical setting

Data consistency

Production of much more comparable data• Quantification of bacterial growth in urine samples*

– Manual method (2 operators) – 263/303 (87%) of duplicate samples gave the same result

– Automated method – 303/303 (100%) of duplicate samples gave the same result

*Comparison of results from automated inoculating and streaking of urine specimens versus manual methodsY. A. Lue, C. J. Krause and A. B. John Quest Diagnostics Incorporated Teterboro, New JerseyOriginally presented at the 2002 American Society of Microbiology Annual General Meeting in Salt Lake City

Benefits of Automation

Reduce cycle times

Evolution• Speeding up current processes

– E.g. HTS throughput• 1,000,000 member collection for

HTS• 100 (20 weeks) days at 10k/day• 10 days (2 weeks) at 100k/day• Retest• Follow-up screening

Revolution• Enabling new workflows

– E.g. Iterative screening• 25k samples in triplicate = 1day• Analyse results• 10k samples in triplicate = 1day• Follow-up screening

Two possibilities:

The “real” reason to automate

Return on Investment (ROI)

Time

The ratio of money gained or lost on an investment relative to the amount of money invested • $100 earned on a $1000 investment = ROI of 10%• $25 earned on $100 investment = ROI of 25%

• Payback Period - The amount of time required for the benefits to pay back the cost of the project.

• Net Present Value (NPV) - The value of future benefits restated in terms of today’s money.

• Internal Rate of Return (IRR) - The benefits restated as an interest rate.

What is ROI?

So why isn’t it that easy for lab automation?

Can be many degrees of separation between automation and ultimate return (revenues/profits)• Other influencing factors• Time• People

– E.g. reliance on short-term measures– Limited feedback loop

Possible conflicts of interest due to the individuals involved• Scientists not accountants• “Human factors”

– Career progression, politics, “latest and greatest” syndrome, etc.

April 21, 2011Confidentiality Label

22

Industry Perception of ROI

How much has automation played a role in helping your organization discover new drugs faster or file more patents?

Don't Know, 4

Not at all, 13

Some, 33

Substantially, 50

Don't know, 16

No, 13

Yes, 71

Industry Perception of ROI (i)

Has automating your process met with your expectations of results?

Don't Know, 12

Not at all, 7

2-4 fold, 45

5-9 fold, 21

10+ fold, 15

Don't know, 4 More

applications, 7

Cost savings, 14

Faster processing,

16

Result quality, 22

Time savings, 37

Industry Perception of ROI (ii)

How much has automating your applications increased your research productivity?

What is the top benefit of automating your applications?

0% 10% 20% 30% 40% 50% 60%

Don't know

No need for increased productivity

None - I have been able to automate processes as needed

Training time to learn software

Applications not able to be automated

Cost of automation

What one factor limits the adoption of more automation in your laboratory the most?

Industry Perception of ROI (iii)

In summary

83% said lab automation has “somewhat” or “substantially” helped them discover new drugs faster of file more patents81% said automation has increased research productivity by at least 2-fold

71% said automation has met their expectations with regards to results

Key benefits were saving time, saving money and increasing qualityHowever…..

…the majority stated that cost is the biggest hurdle to implementing lab automation ?

April 21, 2011Confidentiality Label

27

Possible explanations

The advantages of laboratory automation do not make financial sense• i.e. the ROI is negative

The respondents overestimate the value of automation• Perhaps because they work with it?

People always say things cost too much?

April 21, 2011Confidentiality Label

28

= There is scope for understanding, using and maximizing ROI more effectively

How to maximize ROI

Nine ways to maximize ROI

Understanding Cost vs Quality vs TimeThe Big PictureSelecting the right supplierSelecting the right systemCapturing design detailsPurchasing automationImplementing automationUsing automationPeople issues and change management

Cost vs Quality vs Time

The three main parameters when choosing how to implement an automated system

Very important to decide which is the key factor for a specific implementation

This will to a large degree dictate at least one, and possibly both, of the other two

Time Cost

Quality

Cost vs Quality vs Time

Quality

Cost vs Quality vs Time

CostTime

Quality

CostTime

Cost vs Quality vs Time

Time Cost

Quality

Cost vs Quality vs Time

Time Cost

Quality

Cost vs Quality vs Time

CostTime

Quality

Cost vs Quality vs Time

CostTime

Quality

Cost vs Quality vs Time

The Big Picture

A common mistake is to consider the process(es) to be automated in isolation• In reality, automating a process may give very limited benefits if the big

picture is not seen• Take the opportunity to review

– Current Process• Improvements• Stability

– Up and downstream processes• Compatibility• Bottlenecks

– Future directions• Scalability• Flexibility

April 21, 2011Confidentiality Label

39

ProduceAssayPlates

ProduceWorkingPlates

PrimaryScreen

CherryPick

Actives

Activity Confirm-

ation

Follow-up screens

CherryPick

ConfirmedActives

The Big Picture – The Wrong Way

– The automation drives the process

– Or it becomes a white elephant

Current processMap, improve and ensure stability

Up and downstream processesBottlenecks Compatibility

Physical, process and IS/ITFuture directions

ScalabilityFlexibility

ProduceAssayPlates

ProduceWorkingPlates

PrimaryScreen

CherryPick

Actives

Activity Confirm-

ation

Follow-up screens

CherryPick

ConfirmedActives

The Big Picture – The Right Way

– The process drives the automation

Selecting the Right Supplier

Selecting the Right SupplierClosely linked to system requirements• Often a fairly limited choice• Important factors when there is a choice• Track record – systems and delivery

– Talk to and preferably visit reference sites– Talk to users and managers!– Use industry contacts (LRIG, SLAS, Linked-In, etc.)

• Relationship– Most important when issues arise

• Engineering skills– Software and hardware

• Project Management skills– Sometimes not emphasized at vendors

• Post-implementation support– Technical support– Service

Selecting the Right Supplier

The vendor should be viewed as a partner• If you’re happy, they’re happy• The most successful implementations I have seen use this approach

– And some of the worst when the relationship is not strong– (obviously it’s easier to be a partner when things are going well)– There will always be “give and take”

April 21, 2011

Confidentiality Label

44

“Oh, did we forget to mention we’d like to shake the plates while incubating them?”

“Our project manager just got married and is going on his honeymoon!”

“Our key user is tied up on another project – can we

delay acceptance testing by 3 weeks?”

“We’re now moving to 1536-well plates – can it dispense in those

as well?”

“We’re having trouble handling one of your plate types”

“The testing is taking longer

than expected”

“Can you get it ready early for our CEO visit?”

“Our building is flooded!”

Selecting the Right System

Must have features• Match application requirements (fit for purpose)• Match cost vs quality vs time constraints• Match size and facility constraints• Safety

Other important features• Reliability• Scalability• Flexibility• Usability

Software

Selecting the Right System - Reliability

Think about what you really need• 100% reliability is often asked for, but attempting it is hugely expensive,

and achieving it is impossible!

Different ways of measuring and comparing• Up-time• Mean Time Between Failure • Mean Operations Between Failure• Mean Plates/Samples Between Failure

Many factors affect observed reliability• Labware• Changing applications• User skill level• Building services

Selecting the Right System - Reliability

• Important distinction between failure and error– Balance between walk away time and fail-safe– Ability to switch between attended and unattended modes ideal

– Other users of similar systems likely to be the best source of information on reliability

– Details are important– Labware, users, protocols, etc.

• Keep it simple

Selecting the Right System - Scalability

Initial costSize and facility requirements

Total cost (?)Delivery timeline

+

Risk that the system capacity and/or throughput quickly is proven not to be adequate

• Capacity requirement under-estimated• Capability over-estimated• Availability leads to greater demand

– E.g. cell culture, cherry-picking, sample prep, etc.• Various approaches

Selecting the Right System - Flexibility

Two main types of flexibility• Extensibility

– Re-configurable– Adaptable to new demands

• Versatility– Can do many tasks or processes– Work with many labware types

• Both are important and many systems provide a large degree of flexibility in an effective way…

……..but

April 21, 2011Confidentiality Label

49

Selecting the Right System - Flexibility

…be aware of…increased complexity

Increased complexity =• Increased cost• Increased delivery times• Reduced relative reliability

+ + + +

Selecting the Right System - Flexibility

…be aware of…reduced effectiveness

The right level should be judged on a system by system basis– Consider the cost / benefit ratio

+ ++

• The user experience– Influenced by personal preference

• Software interaction– Different user levels key– Intuitiveness / ease of use DILBERT © (1994) Scott Adams. Used by permission of Universal Uclick. All rights reserved.

• Hardware interaction – Loading plates and reagents, removing waste, etc. – Robot teaching– Error monitoring and handling– Ease of off-line instrument use– Safety and guarding– Access for service and maintenance– Ergonomics– Cleaning or sterilization

Selecting the Right System - Usability

Selecting the right system - software

Equally as important as the hardware• And perhaps, harder to get right• Key aspects to look for

– Scheduling engine• Static vs dynamic vs event-driven, simulation abilities, scalable

– User interface• User roles, layers, intuitiveness

– Configurability• Device addition, level of control, expandability

– Error handling• Ease, consistency, remote control

– Integration• Link to databases, workflow control systems, LIMS, other control software

April 21, 2011Confidentiality Label

53

Capturing system design details

The procurement process is often under time pressure from both the vendor and the customer

In any case, it is unlikely that all the details will be ironed out ahead of an order

But, expectations need to be set correctly• The automation needs to perform the workflow it was designed to address

When is the right time to discuss/finalize details• Dependent on complexity (technical and application)

– Some example approaches:

April 21, 2011Confidentiality Label

54

Capturing the details

Key features Config. Order Delivery

URS Scope / Proposal Order FDS ATP Delivery

Full URS FDS Order DDS ATP Delivery

Pre-order -possibly multiple vendors

Start-up support

Low complexity

Med. complexity

High complexity

Acronym recap

URS – User Requirements Specification• What the customer wants

FDS – Functional Design Specification• What the vendor will deliver

DDS – Detailed Design Specification• How the vendor will deliver functionality

ATP – Acceptance Test Plan• How the vendor will demonstrate functionality

April 21, 2011Confidentiality Label

56

Purchasing Automation

Build a good relationshipAvoid unnecessary time pressure

Be aware of possible false economies• Sourcing 3rd party devices• Re-using old equipmentEnsure internal budget available• Services, training, travel, labwareInvolve other stakeholders early• IS/IT department• Building and facilities• Health and Safety

April 21, 2011Confidentiality Label

57

Implementing Automation

Ensuring expectations converge

Active participation during the entire project• Regular communication

– Phone, email and visits– Be proactive

• Prioritize and contribute fully to documentation– A chore for many

• Involve internal users and customers• Consider internal project manager for complex projects

There should be no surprises when the automation arrives

April 21, 2011Confidentiality Label

58

Using Automation

Take full advantage of training and vendor presence on site

Start to use system immediately, even if only for testing and/or familiarization

Identify a “super-user”

Have a QC procedure in place and use it!

Consider professional services (start-up support)

Maintain system performance logs– Up-time (or down-time)– Utilization– Throughputs– Errors and breakdowns

April 21, 2011Confidentiality Label

59

System Performance Logs

People Issues

Introducing automation and/or new processes has a large impact on people• Users• Customers

Important to be sensitive to these effects

May be worth implementing a change management program for larger installations and/or process changes

April 21, 2011Confidentiality Label

61

People issues – Reaction to change

Time

Invo

lvem

ent

SHOCK

DENIAL

ANGER

BARGAINING

DEPRESSION

ACCEPTANCE

People issues – Reaction to change

ANGER - “%$@^##& car!", “You’re a piece of %#^@." Slam your hand on the steering wheel.

BARGAINING - (realizing that you're going to be late for work)..., "Oh please, please, please, start – I’ll change your battery tomorrow”.

DEPRESSION - "Oh No, what am I going to do. I'm going to be late for work. I give up. I don’t really care about the BioCel meeting anyway!

ACCEPTANCE - "Ok. It's dead. Guess I had better call the garage and cycle to work instead

SHOCK / IMMOBILISATION -“Uuhh, what”. Sit and stare out the windshield.

DENIAL - “It can’t be”, try again, and again, and again. Turn off the lights and… try again!

Another change model

April 21, 2011Confidentiality Label

64

People Issues

• W• I• I• F• M

April 21, 2011Confidentiality Label

65

People Issues

• What’s• In• It• For• Me !

April 21, 2011Confidentiality Label

66

Ensure the operators / customers see the benefits to them

Involve the negative people as much, if not more, than the positive ones

People Issues

Understanding change

Time

Und

erst

andi

ng

Communicate progress and updates on a regular basis

Three take home messages

The ultimate benefit of laboratory automation should be a compelling Return On Investment

Take the time to understand your expectations and then measure the outcomes against them

Remember the people. They are the ones who will ultimately determine the success of the automation

April 21, 2011Confidentiality Label

68

Thank-you for your attention

Questions?