Automated quality control and system suitability in...

©2012, Genentech

Automated quality control and system suitability in Panorama for

peptide and small molecule analysis

2018 June 03

Paul AugerSkyline User Group Meeting

©2012, Genentech

Clinical biomarker assays require high levels of confidence in the high volumes of data produced

Drug developmentResearch Pre-IND Phase I Phase

IIPhase III(Pivotal)

Clinical Practice

Biomarker Discovery

Clinical Use

Biomarker development towards clinical assays

Clinical Qualification

Untargeted discovery

Targeted Quantitation(3-5-plex MRM assay)

Targeted Discovery(Multiplex MRM assay)

Each stage of development increases the need for assay robustness and rigor = reliable QC data

©2012, Genentech

Highly multiplexed assays accentuate challenges for clinical biomarker implementation

Ideal Quality Control data system would be:

● Fully Automated● Flexible● Reproducible● Real Time

Mass Spec lab

Peptide Small Molecule/Metabolite

©2012, Genentech

Panorama is a solution to large scale data storage and analysisPanorama is an open source bioinformatics data management platform– Module within LabKey server– Searchable database storing Skyline data files– Creates a shareable environment for processed Skyline files– Supports large scale method development and refinement– Enables assay and instrument tracking

ASMS 2013

©2012, Genentech

Panorama Partnership enables use of Panorama in secure environment with expanded capabilities

● Collaborate directly with MacCoss lab and LabKey teams● Installed Labkey server on-site

○ Allows for custom analysis and reporting tools● Guide Skyline & Panorama software development priorities● All partners contribute to common development fund● All partners give input on priorities and design decisions

Current Partners: Roche/Genentech (since 2013), Merck & Co., Inc. (since 2014), NCI-CPTAC (since 2017), and Amgen (since 2017)

https://panoramaweb.org/wiki/home/page.view?name=panoramaPartnersProgram

©2012, Genentech

Can we do QC analysis on Panorama?QC samples run

regularly on mass spec(Biognosys iRT mix)

Import into skyline file Publish to ‘System suitability’ Panorama folder

Desired features• QC graphs and metrics automatically

updated• Line charts with multiple peptides• Calculate mean, Std. dev and %CV, and

plot with run value relative to mean over time

• Metrics:• Peak area, peak height, peak width,

retention time• Automatic flagging when outside of

pre-defined thresholds• + instrument log – add info on service,

cleaning etc. and use to annotate graphsSProCoP style with Outlier ID

Panorama Partners: Origins of QC folder (2014)

MS Bereman et al.. JASMS 2014, 25 (4), 581-587

©2012, Genentech

Panorama Partners QC folder

Development of QC folder to monitor instrument performance

View retention time, peak area, H/L ratio, etc.

Add notes

Zoom to details

©2012, Genentech

Define reference regions for QC statisticsPanorama Partners QC folder

QC – Levey-Jennings Plots

©2012, Genentech

Determine which feature drives variability in methodPanorama Partners QC folder

Retention time variable

©2012, Genentech

Transition to Automatic upload and QC analysis

AutoQC Workflow

Users browse, query, and export data

Panorama creates interactive plots

AutoQC uploads results to Panorama

SkylineRunner/Skyline process file

AutoQC detects file

Instrument acquires raw data file

Panorama Partners AutoQC (2016)

©2012, Genentech

Loss of instrument sensitivity detected due to real time peak area monitoring System Suitability

Easy identification of loss of signal across all iRT peptides flags the system and allows analyst to troubleshoot and repair any issues quickly and in real time.

More insidious gradual loss of signal across multiple iRT peptides is often difficult to spot without a quality QC monitoring tool

©2012, Genentech

AutoQC enables the tracking of multiple metrics in real time saving valuable samples

Assay Quality Control

Retention time shift of hydrophilic peptide mimics important analyte. RT shift signals need to replace trap column

Reduction in peak area for same peptide informs that peak is no longer being fully retained at the correct amount

©2012, Genentech

CONFIDENTIAL INFORMATION – DO NOT COPY OR FORWARD – INTERNAL USE ONLY

Panorama and skyline simplify data management and analysis for targeted MS assays

©2012, Genentech

What’s next for AutoQC and the Panorama Partners

Upcoming features● Email alerts● Customizable views

○ Adjustable y-axis on plots○ User-defined guide set ranges (set min/max values for outliers,

instead of being based on mean +/- 3 SD)○ Customizable metrics○ More summary visualizations to see multiple metrics/peptides at

the same time

©2012, Genentech


Acknowledgements

GenentechBiomarker DevelopmentKristin WildsmithJulie LeeShadi EshghiVeronica AnaniaKent WongRod Mathews

Information TechnologiesJerome RaineyJon Sasano

RocheTom DunkleyMichel Petrovic

Panorama PartnersJosh Eckels (Labkey)Brendan MacLean (UW)Vagisha Sharma (UW)

©2012, Genentech

Can we do QC analysis on Panorama?QC samples run

regularly on mass spec(Biognosys iRT mix)

Import into skyline file Publish to ‘System suitability’ Panorama folder

Desired features• QC graphs and metrics automatically updated• Line charts with multiple peptides• Calculate mean, Std. dev and %CV, and plot

with run value relative to mean over time • Metrics:

• Peak area, peak height, peak width, retention time

• Automatic flagging when outside of pre-defined thresholds

• + instrument log – add info on service, cleaning etc. and use to annotate graphs

QC Graph Ideas

SProCoP style with Outlier ID

Panorama Partners: Origins of QC folder (2014)

week1 week2 Week3…..

Skyline style bar charts

©2012, Genentech

Automatically upload QC runs to PanoramaPanorama Partners QC folder

Vagisha Sharma, Skyline team, MacCoss Lab, UW

Automated quality control and system suitability in...

Documents

Transcript of Automated quality control and system suitability in...