11/14/2016

1

Implementation Insights

Vancouver General Hospital, BC

Challenges we faced

Will vary depending on what your starting point is (SBT vs Serology)

• Space

• Training

- Bench and analysis

• Computerization

• Integration into LIMS

Work-flow integration

• How much change do you need to take the lab from molecular to NGS?

➡ still need pre- and post-pcr areas, so not much change from current set up

➡ tissue or sample collection will probably stay the same as current methods (ACD blood, BMT with multiple transfusions?)

➡DNA extractions will also probably stay the same (note the use of EDTA in elution buffer)

➡ computerization might be an issue

11/14/2016

2

Challenges with space• Currently we are still running SSO/SSP, our sequencing

core has moved into the old serology lab

➡ Bench height and width is different

➡ Benches dimension need to be able to take equipment (size and weight…a MiSeq weighs more than Terasaki typing plates)!

• Freezer/fridge space

➡ Do you keep individual loci amplicon? Or only pooled samples? How long are libraries good for?

Challenges with Training

• Staff training involves;

➡ bench/hands on training

➡ analysis resulting training

➡ Also, disseminating information to clinics and physicians.

Challenges with trainingBench work;

• Remarkably similar to current molecular techniques.

• Micro-volumes require additional experience to achieve consistency

• Multiplex of high number of samples can be confusing and will require more care awareness during processing

• Many more steps involved in library preparation (compared to SSO/SSP), time-management is more relevant

11/14/2016

3

Bench Training

• Walk through protocol and technology

• 3 test runs (non-valuable samples; with smaller batch sizes initially),

• 3 real runs (with standard batch size, concordance with SSO/SSP)

• All QC tests passed for all 6 runs

• Training schedule ~5 months

Actions to overcome bench challenges

• Practice good micro-pipetting technique on non-valuable samples

• Develop monitoring procedures, and good practice, to ensure no sample mix up when handling many samples

• Protocols and workflows which are more flexible will permit technicians to develop a time management plan that works for them

Challenges with training

Analysis;

• Previous experience with SBT data and the associated software makes the learning curve less steep

• level of competence comes with analysis of a POOR run not a good run!

• Simplified workflow of analysis is essential to maintain consistency across calls

11/14/2016

4

Actions to overcome challenges with analysis training

• Familiarization with software

• Lab meetings to show how the software looks, how to navigate through analysis pipeline, examples of results, examples of reports, integration into current reporting procedure

• Clear SOP (even better, cheat sheets!)

• Practise datasets (good and bad data). Compare analysis with trainee and trainer, work through challenging calls together

Challenges with equipment

• Due to the significantly higher throughput capabilities of the technology, even simple things will need to be re-assessed;

• ordering reagents and supplies will be more frequent. Plan accordingly

• Multichannel pipettes will have to be maintained regularly and calibrated (which is an ASHI standard)

CONNECTIVITY

• Computer specification for software will vary depending on software used;

• Most can be run on a stand alone desktop, either as individual licenses or as client servers

• Some will require a dedicated server to run software

11/14/2016

5

Challenges with connectivity

• Standard hospital computers are not 64-bit.

• Data storage requirements for an Immunology lab for the last 40 years ~ 500mb/month, current data storage requirements for an Immunology lab ~100gb/month

• Data storage security concerns

Optimal multiplexing strategy

• Optimal multiplexing strategy;

➡expected sample throughput,

➡ frequency of testing,

➡ the assay TAT,

➡ the degree of batching (for regions of high genomic complexity, higher read depth can be used to compensate to improve accuracy)

• for batching samples, there must exist guidelines for standard multiplexing and read depth to ensure equivalence of test results.

• Validation of barcodes

DATA REPORTING

• Clinical interpretation: Seamless, unambiguous, clinically relevant interpretation for end user

• Clinical report generation; ‘What to include and what not to include’…intronic differences, variants of unknown significance

11/14/2016

6

Integration with LIMS

• Automatic transfer of fastq files to data analysis software

• Direct import from analysis software to Histotrac (or equivalent)

• Update of

Tie- breakers

• All methods tested were excellent and easily implementable in the lab

• Customer support is critical

• Although cost is of course important, should not be the deciding factor

• Tech feedback

Top 5 reasons to go NGS

• The learning curve might be steep, depending on where you start from (serology, SBT), but easily streamlined for a medium-large throughput lab

• One pass testing; no need to go back and order additional tests (careful with those class 2’s!)

• Maintain autonomy from other sequencing labs, business model

• Creating a pre-to-post transplant continuum

• Cos all your friends are doing it!

11/14/2016

7

THANK YOU

Introduction

• For clinical purposes, NGS has all but replaced it’s methodological predecessor, Sanger sequencing.

• It’s faster, cheaper but ALONE, is it sensitive and specific enough to catch every variant while avoiding false-positives?

Is cost a barrier to introduction of NGS in HLA labs?

•

Pro’s Con’s

Pushing Transplant diagnostics into the 21st century

High set up costs

Comparable to current molecular testing costs

More complex training than current molecular testing techniques

Fewer supplementary assaysMinor pipetting error could result in loss of amplicon and consequently

no data

Limits repeated testing (especially hi-res repeats when Ab detected)

Steep learning curve for data analysis

11/14/2016

8

Interesting article

• The Scientist Q&A: Confirming NGS results with Sanger (Ambry Genetics CEO, Aaron Elliot)

• http://www.the-scientist.com/?articles.view/articleNo/47224/title/Q-A--Confirming-Next-Gen-Sequencing-Results-with-Sanger/

Is cost a barrier to introduction of NGS in HLA labs?

• multi-plexing is a double edged sword; if it works, drastically lowers cost, but if theres an error, lose a lot in terms of reagent and time

• higher start-up costs

• cost of positive/negative controls (for each library preparation as well as for each sequencing instrument run)

11/14/2016

9

Costs

• Evaluation costs

• Validation/accreditation costs

• Capital equipment costs

• Tech costs…training costs

Capital Equipment InvestmentInstrument Cost*

Illumina MiSeq $100,000

Illumina MiSeq Support (Years 2 & 3) $34,000

Size selection equipment (Optional) $10,000

Plate fluorometer (Optional) $20,000

qPCR machine (Optional) $30,000

5x PCR machine $35,000

64-bit computer with 16+ GB RAM $3,000

1-2x Liquid handler (Optional, pre & post PCR) $50,000+ (each)

*Based on figures from four US Labs, rounded to $1,000

workflow integration• Upstream integration

• Sample collection and DNA extractions

• Library preparation workflows

• Sample batching

• Liquid handling automation

• Data processing and analysis workflows

• Data transfer automation

• Analysis automation

• Long term data storage

• Approval workflow and tracking

• LIMS integration

• Export to NMDP standard

• Import directly into LIMS

11/14/2016

10

Work-flow integration

• multiple QC steps; initially time-consuming but in the long run, will save bench time. Eventually QC steps can be pared down to minimum.

• to confirm or not to confirm…with different assay

• typing takes longer than antibodies…consequence on activation of patient

• labs not previously engaged in HLA typing could become involved in testing, as the workflow is fairly simple and indeed not drastically different for a wide range of diagnostic tests (cancer panel vs hla panel)…however interpretation is KEY

Automation

11/14/2016

11

Work-flow integration

• WHAT IS OUR CURRENT TAT FOR NGS?

• Currently not suitable for deceased donor typing

Challenges in data interpretation

• Can be summarised in 3 words; TIME, WORKLOAD, AMBIGUITIES

• Average time spent analysing and interpreting NGS data…time allotment

• data analysis is still a bottle-neck…partly because batches can be so large

• very few ambiguities…gone are the days of ‘chains’

Challenges in data interpretation

• determining QC characteristics

• associations

11/14/2016

12

Challenges in data interpretation

• loss of 2nd allele in homozygous!

• determining true homozygousity

• drop outs…

• amplicon coverage- accurate pooling required

Challenges in data interpretation

• Submission of new alleles to CTR dictionary

• getting used to no chains!

• G group problem is now the reverse of the problem before (cant match too specific vs cant match not specific enough)

NGS Data Analysis Approaches

• Reference- bases alignment, followed by HLA call based on the variants detected during alignment

• call is only as good at the alignment, coverage often insufficient

• Database based approach against known alleles

• novel allele detection and homozygous alleles are difficult to detect

11/14/2016

13

Complex genetics

• Duplications

• High level of segmental duplications

• Lots of similar genes and lots of similar pseudogenes

• Duplicated segments can be more similar within an individual than they are to the corresponding segments of a reference genome

Complex genetics

• Particularly DRB

• Tandem repeats can still be problematic (however no other technique has overcome this issue fully)

However…

• Many alleles are already known (even partially), both in terms of sequence and frequencies within a population

• HLA regions is relatively small, with a high degree of linkage disequilibrim, therefore many haplotypes have been described

11/14/2016

14

Validation requirements

• 50 samples (plus 20 blind), over 3 runs

• Set up as you would routine testing (run size, sample type)

• Evaluation of;

• Run to run variability

• Tech to tech variability

Quality Testing

• Must evaluate potential allele dropouts

• Documentation for;

• Sample preparation

• Monitor fidelity of barcoding methods

• rotate control samples with different barcode sequences

• Instrument performance measures

• internal control sample for instrument run performance

Quality Testing

• Analytic performance criteria

• Incorporating vendor specifications

• Base quality, read length, average coverage, uniformity of coverage

• Independently validate software program

• Ensure genotyping algorithms are appropriate for sequencing strategy and sequencing error modalities

11/14/2016

15

• KNOW YOUR ASSAY’S LIMITATIONS!

Issues to consider

• inclusion of positive control sample, library QC

• use of phiX for cross lot QC, run QC

Top 5 reasons to go NGS

• The learning curve might be steep, depending on where you start from (serology, SBT), but easily streamlined for a medium-large throughput lab

• one pass testing; no need to go back and order additional tests (careful with those class 2’s!), auto-antibody issue

• Cos all your friends are doing it

• potential for …epitope etc, rejection assay that detects donor-derived cell-free DNA in recipient plasma (Journal of Molecular Diagnostics)

Circulating cell-free DNA enables noninvasive diagnosis of heart transplant rejection. De Vlaminck et al. Sci Transl Med. 2014 Jun 18;6(241)

• ?maintain autonomy from (cancer) other sequencing labs, business model

• Creating a pre-to-post transplant continuum?

11/14/2016

16

Single Molecule, Real-Time (SMRT®) Sequencing

Unique capability to reveal the unknown

High

Accuracy� Achieves >99.999% consensus accuracy

� Lack of systematic sequencing errors

2 High Coverage

Uniformity

Lack of GC content or sequence

complexity bias

3 Sequence

native DNA� No DNA amplification

� Epigenome characterization

4Very Long

Read lengthsSingle Molecule Real Time Sequencing is

capable of generating long reads

to enable resolution of variation

1

Roche’s Single Molecule Real-Time Sequencer

Accessible Benefits of Single Molecule Real-Time Sequencing

Products in development and not available for global distribution. For Research Use Only.

Not for use in diagnostic procedures.

SMRT® is a trademark of Pacific Biosciences

� Long read lengths

� High accuracy

� Sensitivity to detect minor variants at low frequencies

� Simultaneous base-modification detection

� High coverage uniformity

� Short run times (minutes to few hours)

� Analytically validated workflows and analysis algorithms

� An optional, simplified supplied and supported compute

pipeline

Designed To Deliver:

11/14/2016

17

LOCATION

SPACE• ASHI Standards require (wipe tests for both);

• Pre-PCR and

• Thermocyclers, vortexer, centrifuge (no change from current molecular requirements)

• Post-PCR areas

• DNA quantitation equipment, thermocyclers, vortexer, centrifuge, library preparation auxiliary equipment, sequencer

• DNA extraction area

EQUIPMENT (specific)

• Auxiliary equipment (not just the sequencer) is required for any HLA kit on the market

• DNA quantitation (and qualitation) equipment (fluorometers, spectrophotometers, gel equipment, RT-PCR etc)

• Library preparation equipment (magnetic stands for beads (remarkably expensive!), Pippin prep), thermocyclers)

11/14/2016

18

Challenges with space• Currently we are still running SSO/SSP, our sequencing

core has moved into the old serology lab

• Bench height is different

• Benches dimension need to be able to take equipment (size and weight…a MiSeq weighs more than Terisaki plates)!

• Freezer/fridge space

• Do you keep individual loci amplicon? Or only pooled samples? How long are libraries good for?

DNA quantitation equipment

Equipment (standard)

• Vortexers (plate and tube)

• Centrifuges (plate and tube)

• Amplification thermocyclers

• Multichannel pipette

11/14/2016

19

Other equipment

Challenges with equipment

• Due to the significantly higher throughput capabilities of the technology, even simple things will need to be re-assessed;

• ordering reagents and supplies will be more frequent. Plan accordingly

• Multichannel pipettes will have to be maintained regularly and calibrated (which is an ASHI standard), but when pipetting for heavy loads and with microvolumes, any tiny variability will cause samples to fail

Equipment (general)

• Liquid handlers are highly recommended for both Pre- and Post-PCR steps

• Budget allowing (otherwise, get the 1 for Pre-PCR)

• Are programmable for other uses within the laboratory

11/14/2016

20

Challenges with equipment

• Sequencers and liquid handlers have relatively large footprints, and can be very heavy and sensitive

• Must be on weight bearing bench

• No located on the same bench as a centrifuge or high-traffic area as vibrations can affect both the sequencer and the liquid handler

• Highly recommended NOT to move sequencers (alignment of internal optics) or liquid handlers (alignment and very heavy) unless absolute need..pick a permanent place

Equipment (specific)

TRAINING

• Staff training involves;

• bench/hands on training

• analysis resulting training

• Also, disseminating information to clinics and physician. Whilst the reported typings (if remaining at 4 digits) will look the same, the loss of chains and the ambiguities (no more XX) will need to be clarified with the programs

11/14/2016

21

Challenges with training

• Bench work;

• Remarkably similar to current molecular techniques. Micro-volumes require additional experience to achieve consistency

• Multiplex of high number of samples can be confusing and will require more care awareness during processing

• Many more steps involved in library preparation (compared to SSO/SSP), time-management is more relevant

Actions to overcome bench challenges

• Practice good micro-pipetting technique on non-valuable samples

• Develop monitoring procedures, and good practice, to ensure no sample mix up when handling many samples (do not overwhelm the tech…identify a suitable number of multiplex-able sample and use that as a standard)

• Protocols and workflows which are more flexible will permit technicians to develop a time management plan that works for them

Challenges with training

• Analysis;

• Previous experience with SBT data and the associated software makes the learning curve less steep

• level of competence comes with analysis of a POOR run not a good run!

• Simplified workflow of analysis is essential to maintain consistency across calls

11/14/2016

22

Actions to overcome challenges with analysis training

• Familiarization with software

• Lab meetings to show how the software looks, how to navigate through analysis pipeline, examples of results, examples of reports, integration into current reporting procedure

• Clear SOP (even better, cheat sheets!)

• Practise datasets (good and bad data). Compare analysis with trainee and trainer, work through challenging calls together

ADDITIONAL

• During the validation phase of assay development, it is important to test assay performance using reference material

• Positive control material

• Must establish what constitutes a ‘fail’ or a ‘pass’ positive control

• Requirement to include a Negative control

11/14/2016

23

Bench Training

• Walk through protocol and technology

• 3 test runs (non-valuable samples; with smaller batch sizes initially),

• 3 real runs (with standard batch size, concordance with SSO/SSP)

• All QC tests passed for all 6 runs

Equipment Training

• Monitoring sequencing runs

• SOP will outline run metrics, positive control for sequencing portion of protocol (cross run variability)

• Maintenance of sequencers (do not knock bench while machine is running etc, wash maintenance schedule)

CONNECTIVITY

• Computer specification for software will vary depending on software used;

• Most can be run on a stand alone desktop, either as individual licenses or as client servers

• Some will require a dedicated server to run software

11/14/2016

24

Challenges with connectivity

• Standard hospital computers are not 64-bit.

• Data storage requirements for an Immunology lab for the last 40 years ~ 500mb/month, current data storage requirements for an Immunology lab ~350gb/month

• Data storage security concerns (there is a paper published that individuals could be identified from anonymous NGS data; Science 2013. 339:321-4. ‘Identifying personal genomes by surname inference)

BIOINFORMATICS

• essential to have a strategy to detect and investigate false…

Challenges of Bioinformatics

11/14/2016

25

COST COMPARISON

• Demonstrating cost effectiveness of clinical NGS is key to payor reimbursement, hospital uptake

OTHER CHALLENGES

• The question is not technological ‘can it be done’ but rather practical ‘how can NGS technology be developed into a mainstream clinical typing tool?’

• Quality control standardisation…lack thereof…current NGS technologies have higher error rates and novel error modes compared to traditional sequencing…

• what is an appropriate orthogonal technology? i mean, we ‘assume’ SSO/SSP is ‘right’. What is the true measure of truth

11/14/2016

26

• BMT samples; the way we request samples may need to change…what if the patient has had multiple transfusions and we are not detecting patient cells but donor DNA

• DNA often low, cell count low in certain types malignancies

How we overcame challenges

FLOW DIAGRAMS OF VENDORS

11/14/2016

27

Illumina TruSight

Life Technologies (Thermo Fisher) NXType

11/14/2016

28

FLOW DIAGRAMS IN PRACTICE

• CAP NGS checklist recommends orthogonal analytical confirmation of all encountered mutations from an assay before the mutation is reported as clinically actionable (is this relevant for HLA…possibly not)

• FDA-Cleared instrumentation…

• Optimal multiplexing strategy; includes taking into account the following …expected sample throughput, frequency of testing, the assay TAT, the degree of batching (for regions of high genomic complexity, higher read depth can be used to compensate to improve accuracy)…for batching samples, there must exist guidelines for standard multiplexing and read depth to ensure equivalence of test results.

11/14/2016

29

DATA REPORTING

• Clinical interpretation: Seamless, unabiguous, clinically relevant interpretation for end user

• Clinical report generation; ‘What to include and what not to include’…intronic differences, variants of unknown significance

Raw reads (DAT files, BCL files)

Deconvoluted reads (FASTQ files)

Alignment to reference genome

Aligned reads (BAM files)Coverage calculations, local alignment

Demultiplexing

On target alignment

Variation analysis

Filtration

Clinical report

‘Truth’

Parallel analysis of control samples

QC raw and QC passed run yield,read quality, run parameters

QC barcoding deconvolution, sample read distribution

QC alignment (mapping quality),library complexity

QC coverage depth, coverage uniformity,allellelic frequency,

strand bias, GC content

QC controls