Evaluating Stochastic Thresholds for Six Amplification...

© 2011 The Bode Technology Group, Inc.

Evaluating Stochastic Thresholds

for Six Amplification Kits

Presented by: Stephanie Sivak


Outline

• Study performed by Bode in conjunction

with VCU

• Purpose

• Stochastic drop out

• Study design

• Results


Stochastic drop-out

• Defined in this study:

• An event where a pair of alleles exhibit one allele above an established analytical threshold and one allele below – Sister allele could also

drop out completely

75RFU


Stochastic drop-out

• May be affected by a number of

contributing factors in the laboratory

• Amplification conditions: template input,

reaction volume and cycle number

• Sequencing conditions: injection time and

voltage


Purpose

• Problem: Analyst variability with mixture

interpretation

• NIST Mix 05 Study

– Confirmed the issue


Purpose

• Baby Steps: Stochastic threshold

evaluation

• Original validation studies- homozygote

threshold

• Increased sensitivity

– Type of samples testing

– Optimization techniques


Purpose

• 2009 FBI/QAS requirement to set stochastic threshold – “The laboratory establishes a stochastic threshold based on

empirical data derived within the laboratory and specific to the quantitation and amplification systems (e.g., kits) and the detection instrumentation used.”

• 2010 SWGDAM requirement to set stochastic threshold – “Internal validation studies conducted after the date of this

revision shall include as applicable: database-type samples, reproducibility and precision, sensitivity and stochastic studies, and contamination assessment. Internal validation studies shall be documented and summarized. The technical leader shall approve the internal validation studies.”


Experimental Design

• Evaluate single source and mixture samples

– First 4 amplification kits: P+, CO, ID, PP16

• Use of single source

– Amp at different low level concentrations

– Determine a peak detection threshold

– Evaluate analysis threshold

– Suggestions for stochastic thresholds

• Does the introduction of more than one person’s DNA affect stochastic threshold


Experimental design

• Evaluate mixture profiles at levels where

interpretation is difficult due to the

possibility of drop out

• Based on this data make

recommendations for a stochastic

threshold that would aid in interpretation


Experimental Design

• Evaluated 75 potential donors

– Consent

– Level of heterozygosity

– Rarity of alleles in profile

– Gender

• Initial studies based on individual displaying the most heterozygosity

• Chose 13 donors to create 9 two-person mixture samples


Experimental Design Chosen donor alleles:


Experiment Design

• Peak Detection

– Level of DNA needed to generate detectable results from low template samples

– Targeted: 50pg, 100pg, 150pg, 200pg

– Amplified with 4 amplification kits • Profiler Plus

• COfiler

• Identifiler

• PowerPlex 16


• Samples were extracted on an EZ1 and

quantified with Quantifiler

• All samples were amplified in 25ul reaction

volumes

• P+,CO, ID = 28 cycles

• PP16= 30 cycles

• All samples were injected at (3kV 10s)

Methods


• Analyze data at the baseline- 1 RFU

• Any peaks attributed to real data deleted

• Calculate average peak height for

remaining data points (noise)

• x 3 standard deviations

Method-Peak Detection


Results-Peak Detection

Amplification Kit Peak Detection

Profiler Plus ID 20 RFU

COfiler 20 RFU

Identifiler 25 RFU

PowerPlex 16 30 RFU

• Minimum value we can start to determine real

data from background noise

• Minimum value to look for drop out peaks


Stochastic Threshold

• Evaluation proposed in different ways

– Logistic regression

– Above highest peak in the most severe

imbalance

– Plotting PHR v Avg RFU


Sources

• Gill, Peter et al. FSI: Genetics 3 (2009) 104-111

• Puch-Solis R. et al. FSI: Genetics Supplement Series 2 (2009) 460-461

• Bright Jo-Anne et al. FSI: Genetics 4 (2010) 111-114

• Tvedebrink Torben et al. FSI: Genetics 3 (2009) 222-226

• Balding, David J and Buckleton, John. FSI: Genetics 4 (2009) 1-10

• Budowle B. et al. J Forensic Sci 54(4)(2009) 810–821

• NIST presentations

• User Forums


PHR v Avg RFU

NIST presentation- how to determine the stochastic threshold

• Examine intensity and peak height ratios of samples at different low level concentrations

• Observe variation in peak height ratio and peak intensity

• The stochastic threshold is the point at which variation begins a rapid increase


Results- Peak Height Ratio

vs RFU

• At which RFU value does the majority of the peak height ratios fall below our expected balance – P+,CO, ID= 60%

– PP16= 50%

– Using an analytical threshold of 75 RFU

• Plotted PHR v Avg RFU for 50, 100, 150, 200pg

• Zoom in on PHR below expected balance % – Heights could be considered unreliable under this

value and drop out may be possible


Comparison of Signal Strength with Peak Height Ratio for Profiler

Plus 25ul Reactions at 28 cycles using 50, 100, 150 and 200pg of

Template DNA

0

20

40

60

80

100

120

0 50 100 150 200 250 300 350

Average RFU per Allele

Peak H

eig

ht

Rati

o (

sh

ort

er

peak/t

all

er

peak)


Comparison of Signal Strength with Peak Height Ratio for COfiler

25ul Reactions at 28 cycles using 50, 100, 150 and 200pg of Template

0

20

40

60

80

100

120

0 100 200 300 400 500


Peak H

eig

ht

Rati

o (

Sh

ort

er

peak/t

all

er

peak)


Comparison of Signal Strength with Peak Height Ratio for Identifiler

25ul Reactions at 28 cycles using 50, 100, 150 and 200pg of Template

DNA

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

0 50 100 150 200 250 300 350 400


Peak H

eig

ht

Rati

o (

sh

ort

er

peak/t

all

er

peak)


Comparison of Signal Strength with Peak height Ratio for

PowerPlex 16 25ul Reactions at 30 cycles using 50, 100, 150 and

200pg of Template DNA

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

0 100 200 300 400 500 600 700 800 900


Peak H

eig

ht

Rati

o (

sh

ort

er

peak/t

all

er

peak)


• Based on one sample

– Triplicate amplifications at 4 concentrations

– May be able to lower for ID

– Suggest increasing above 200 rfu for PP16

• Determine targeted amounts for mixture study

• What influences stochastic thresholds?

– DNA quantity?

– Multiple contributors?

Summary


Mixture Study • Target amount of minor contributor for

mixture study

– PP16

• 100pg

• 75pg 50:50

– ID, P+, CO

• 150pg

• 100pg 50:50


Experiment Design

• Two-person mixtures

– Paired 7 females with 6 males to make 9 two-person mixtures


Experiment Design

• Different ratios F:M

• 8 ratio

comparison/pair

• 9 pairs

• 3 amplifications/ea

• 4 amplification kits

• 864 total profiles

Donor Ratio Concentration (pg)

Female 95% 2850

Male 5% 150

Female 90% 1350

Male 10% 150

Female 80% 600

Male 20% 150

Female 50% 100

Male 50% 100

Female 50% 150

Male 50% 150

Female 20% 150

Male 80% 600

Female 10% 150

Male 90% 1350

Female 5% 150

Male 95% 2850


Experiment Design

• Each profile evaluated to determine threshold above highest peak in most severe imbalance allele pair – 1 allele ≥150RFU and the other allele is between 20-

74RFU

– Determine if we dropped our analytical threshold could we drop our stochastic

• Plotted PHR of minor contributor and major contributor vs RFU


Profiler Plus Results Profiler Plus™

Sample Name Locus Peak Height 1 Peak Height 2

F1_50_M1_50_100pg FGA 38 153

F1_50_M1_50_100pg D18S51 48 165

F6_50_M1_50_150pg D18S51 51 155

F3_50_M3_50_150pg vWA 53 203

F5_50_M5_50_150pg D7S820 55 172

F3_80_M3_20 D21S11 55 210

F2_50_M2_50_100pg D18S51 55 240

F6_50_M1_50_150pg D21S11 58 173

F1_50_M1_50_150pg vWA 59 209

F1_50_M2_50_150pg FGA 61 166

F7_50_M6_50_150pg D5S818 65 152

F1_10_M1_90 D18S51 68 154

F1_5_M1_95 D18S51 69 211


COfiler Results

COfiler™


F2_50_M3_50_150pg D16S539 24 176

F2_90_M2_10 TPOX 48 185

F2_50_M2_50_100pg D16S539 49 156

F2_95_M2_5 TPOX 50 155

F1_20_M1_80 D3S1358 51 167

F2_50_M2_50_150pg D16S539 55 158

F1_50_M2_50_150pg D16S539 62 240

F7_50_M6_50_100pg D16S539 71 217

F3_20_M3_80_CO D16S539 74 157


Identifiler Results Identifiler™


F7_50_M6_50_150pg D16S539 43 164

F1_50_M2_50_150pg D21S11 43 172

F3_50_M3_50_150pg CSF1PO 44 180

F4_20_M4_80 TPOX 49 179

F1_20_M2_80 D16S539 50 173

F5_50_M5_50 D16S539 58 176

F2_50_M3_50 D8S1179 59 200

F4_80_M4_20 D19S433 60 166

F5_50_M5_50_150pg D16S539 62 172

F4_50_M4_50_100pg TPOX 66 172

F5_50_M5_50_150pg CSF1PO 67 156

F2_80_M2_20 FGA 68 153

F3_50_M3_50_150pg CSF1PO 70 222

F1_10_M2_90 D16S539 70 234


PowerPlex® 16


F2_10_M2_90 D18S51 35 222

F2_50_M2_50_75pg D16S539 40 236

F2_50_M2_50_100pg D16S539 42 209

F1_50_M2_50_100pg D5S818 43 184

F1_50_M1_50_75pg D7S820 43 161

F2_80_M2_20 Penta E 44 247

F1_50_M2_50_100pg D16S539 44 251

F1_80_M1_20 Penta D 45 178

F2_50_M2_50_75pg D16S539 46 195

F2_50_M2_50_75pg D18S51 50 184

F2_95_M2_5 TPOX 51 212

F1_50_M1_50_75pg D18S51 59 154

F3_5_M3_95 Penta E 60 210

F6_50_M1_50_75pg Penta E 60 156

F1_50_M2_50_75pg D16S539 60 168

F2_50_M2_50_100pg Penta E 65 155

F1_50_M2_50_75pg Penta E 65 266

F3_20_M3_80 D21S11 66 179

F1_20_M2_80 FGA 66 167

F6_50_M1_50_75pg D18S51 72 193

F2_20_M2_80 Penta E 72 179

F2_5_M2_95 Penta E 74 184


Summary • ABI kits appeared consistent between each other but the

stochastic variation depending on the analytical

threshold – 160RFU - 240RFU

• Promega PP16 kit produced the most stochastic

occurrences but the stochastic variation was more

consistent and not as dependent on the analytical

threshold – 250RFU - 260RFU

• Did not want to implement so many different thresholds

or rely on outlier data – Cannot account for every genetic anomaly


Results Heterozygous Balance for Identifiler - 28 cycles, 25 ul reactions

Run on ABI 3100 Sequencer

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

0 100 200 300 400 500 600 700

RFU per Allele

He

tero

zy

go

us

Ba

lan

ce

(%

)


Summary • Began implementing use of stochastic

thresholds in mixture interpretation in 2009.

• If a locus exhibits alleles below the stochastic threshold then drop out may be possible

– Increase in inconclusive results

– Analysts consistency greatly improved


Recommendations

• Single source profile with high level of

heterozygosity

• Get data points from ~3-5 individuals

• Plot PHR vs Avg RFU

Evaluation of Identifiler Plus

and PowerPlex 16 HS

• Evaluated new Identifiler Plus and

PowerPlex 16 HS amp kits

– Prior to manufacture, companies stated these

kits would provide greater sensitivity and

improve mixture interpretation

• Better balance = lower analytical threshold



Experimental Design

• Evaluate single source samples only

– ID+ and PP16 HS

• Amp at different low level concentrations


– Evaluate analysis thresholds

– Suggest stochastic thresholds


• Samples previously extracted on EZ1s

and quantified with Quantifiler

• All samples were amplified in 25ul reaction

volumes

• ID += 28 cycles

• PP16 HS= 30 cycles

Methods


Results- Peak Height Ratio

vs RFU

• At which RFU value does the majority of the peak height ratios fall below our expected balance – ID+= 60%

– PP16 HS= 50%

– Using an analytical threshold of 75 RFU

• Plotted PHR v Avg RFU for 50, 100, 150, 200pg

• Zoom in on PHR below expected balance % – Heights could be considered unreliable under this

value and drop out may be possible


Results Comparison of Segnal Strength with Peak Height Ratio for Identifiler Plus 25ul Reactions at

28 cycles using 25, 50, 100 and 200 pg of Template DNA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 100 200 300 400 500 600 700

Average Peak Height per Allele (RFU)

Pea

k H

eig

ht

Rat

io (

Max

/Min

)

Comparison of Signal Strength with Peak Height Ratio for Identifiler Plus

25uL Reactions at 28 cycles using 25, 50, 100 and 200 pg of Template DNA


Results Comparison of Signal Strength with Peak Height Ratio for PowerPlex 16 HS

25uL Reactions at 28 cycles using 25, 50, 100 and 200 pg of Template DNA Comparison of Signal Strength with Peak Height Ration for PowerPlex 16 HS 25ul Reactions

at 30 amplification cycles using 25, 50, 200 and 200 pg of Template DNA

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 50 100 150 200 250 300 350 400

Average Peak Height per Allele (RFU)

Pea

k H

eig

ht

Rat

io (

min

pea

k/m

ax p

eak)


Results Summary

• Table shows highest peak threshold observed

with sister allele < analytical threshold – Again, with ABI’s ID+, decreasing analytical threshold means

that you can decrease stochastic threshold

– Promega’s PP16 HS stochastic threshold not affected by

changing analytical threshold

Analytical Threshold 50RFU 75RFU

ID + 158 222

PP16 HS 129 129


Comparison of ID/ID+ and

PP16/PP16 HS

• ID+

– Greater stochastic threshold than ID (250RFU vs 200RFU)

– Other studies performed by Bode confirmed ID+ is more

sensitive, which could aid interpretation

• PP16 HS

– Decreased stochastic threshold than PP16 (200RFU vs

250RFU)

– Other studies performed by Bode confirmed PP16 HS is only

slightly more sensitive, which probably will not greatly aid

interpretation


Summary

• Data from 2009 study examining four kits proved to be solid basis for study this year on two additional kits – Stochastic thresholds still in same 200-250rfu range

• Work in progress – Looking into increasing PP16 stochastic threshold to 250 rfu also

– Use in mixture interpretation software for HTP lab (need macros/software!!!)

– Use of different types of statistics to allow use of more loci


Recommendations

• Single source profile with high level of

heterozygosity

• Get data points from ~3-5 individuals



Many Thanks!

• Jon Davoren, Bode

• Katherine Butler, GWU

• Hallie Garofalo, Bode (VCU)

• Mike Cariola, Bode

• Amy Jeanguenat, Bode

[email protected]

Evaluating Stochastic Thresholds for Six Amplification...

Documents

Transcript of Evaluating Stochastic Thresholds for Six Amplification...