Diagnostic Testing

Diagnostic testingDiagnostic testing

Diagnostic testingDiagnostic testing

Relevant to:Relevant to:1.1. Control and eradication campaignsControl and eradication campaigns

(e.g., serological screening; (e.g., serological screening; genetic screening; genetic screening; herd and flock accreditation) herd and flock accreditation)

2.2. Individual-animal clinical diagnosisIndividual-animal clinical diagnosis

Diagnosis of infectious disease Diagnosis of infectious disease in populationsin populations

1. Current infection1. Current infection• Clinical signsClinical signs• Isolation of agentIsolation of agent

• Molecular changes Molecular changes (surface antigens, integration)(surface antigens, integration)

IMMUNE RESPONSEIMMUNE RESPONSE

• Biochemical changesBiochemical changes

Diagnosis of infectious disease Diagnosis of infectious disease in populationsin populations

2. Past infection2. Past infection

• Clinical historyClinical history

• Pathognomonic changesPathognomonic changes

IMMUNE RESPONSEIMMUNE RESPONSE

SEROLOGICAL EPIDEMIOLOGYSEROLOGICAL EPIDEMIOLOGY

Strictly: any variable in serumStrictly: any variable in serum(e.g., metabolites)(e.g., metabolites)

Conventionally: antibodiesConventionally: antibodies

Nature of veterinary dataNature of veterinary data

Problem ofProblem of variabilityvariability

E.g. Milk yieldE.g. Milk yieldWeightWeightHeightHeightAntibody titresAntibody titres

DataData

QuantitativeQuantitative

NominalNominal OrdinalOrdinal DiscreteDiscrete(counts(counts))

ContinuousContinuous(measurements)(measurements)

IntervalInterval RatioRatio

QualitativeQualitative(categorical)(categorical)

Variability:Variability:

• Demonstrated in Demonstrated in singlesingle samples samples for continuous data (and ordinal for continuous data (and ordinal data)data)

16

14

12

10

8

6

4

2

01 2 3 4 5 6 7

Weight (kg)

Num

ber o

f pig

s

Median (QMedian (Q22) = 50) = 50thth percentile percentile

Standard deviation (Standard deviation (ss))

Central tendancyCentral tendancy

SpreadSpread

= (x) n

Mean (Mean (xx) )

Where s2 = (x- )2

(n-1)xx

Semi-interquartile range (SIR)

SIR = SIR = (Q(Q33-Q-Q11)) 2 2

Where Where QQ11 = 25 = 25thth percentile (lower quartile) percentile (lower quartile)QQ33 = 75 = 75thth percentile (upper quartile) percentile (upper quartile)

-3-3 -2-2 -1-1 00 11 22 33

Pro

porti

on o

f val

ues

of x

/uni

t

0.10.1

0.20.2

0.30.3

0.40.4

0.50.5

0.60.6

0.70.7

xx

Normal distributionNormal distributiont t distributiondistributionlognormal distributionlognormal distribution

Antibody dilution:Antibody dilution:• Usually with a constant ratio Usually with a constant ratio

between dilutions between dilutions (i.e. geometric)(i.e. geometric)• This allows measurement on a This allows measurement on a

logarithmic scalelogarithmic scale, because:, because:

1.1. Antibodies are lognormal Antibodies are lognormal

2.2. LogsLogs22 can be used as coded values can be used as coded values

Num

ber o

f bird

sN

umbe

r of b

irds

Antibody titres (logarithmic scale)Antibody titres (logarithmic scale)

00<10<10

35 -35 -

40 -40 -

25 -25 -

20 -20 -

15 -15 -

5-5-

30 -30 -

10 -10 -

1010 2020 4040 8080 160160 320320 640640 12801280 25602560 >2560>2560

Antibody titres expressed as reciprocal Antibody titres expressed as reciprocal dilutions (X) and coded titres (logdilutions (X) and coded titres (log22X)X)

Reciprocal dilution Reciprocal dilution (X)(X)

Coded titre Coded titre (log(log22X)X)

1 (undiluted serum)1 (undiluted serum) 00 22 11 44 22 88 331616 443232 556464 66

Mean titresMean titresArithmetic meanArithmetic meanMean of coded titres:Mean of coded titres: E.g. five titres 1/2, 1/4, 1/2, 1/8, 1/4E.g. five titres 1/2, 1/4, 1/2, 1/8, 1/4 Coded as: 1, 2, 1, 3, 2Coded as: 1, 2, 1, 3, 2 Arithmetic meanArithmetic mean = (1+2+1+3+2)/5= (1+2+1+3+2)/5

= 1.8= 1.8Geometric meanGeometric meanAntilogAntilog22 of arithmetic mean of arithmetic meani.e. 2i.e. 21.81.8 = 3.48 = 3.48

NB Sernonegative animals CANNOT be includedNB Sernonegative animals CANNOT be included

Why can’t seronegative Why can’t seronegative animals be included?animals be included?

Because the logarithm of zero is:Because the logarithm of zero is:

- -

INTERPRETATION OF TESTSINTERPRETATION OF TESTS

• Refinement Refinement (Analytical specificity)(Analytical specificity)

• AccuracyAccuracy

The classification of some influenza A viruses The classification of some influenza A viruses (based on Murphy and Webster, 1990)(based on Murphy and Webster, 1990)

Haemagglutinins (H) and Haemagglutinins (H) and neuraminidases (N)neuraminidases (N)

StrainsStrains

H1 N1H1 N1 PR/8/34PR/8/34H1 N1H1 N1 Sw/Ia/15/30Sw/Ia/15/30H2 N2H2 N2 Sing/1/57Sing/1/57H3 N2H3 N2 HK/1/68HK/1/68H3 N2H3 N2 Sw/Taiwan/70Sw/Taiwan/70H3 N8H3 N8 Eq/Miami/1/63Eq/Miami/1/63H4 N6H4 N6 Dk/Cz/56Dk/Cz/56H5 N3H5 N3 Tern/S.A./61Tern/S.A./61H6 N2H6 N2 Ty/Mass/3740/65Ty/Mass/3740/65H7 N7H7 N7 Eq/Prague/1/56Eq/Prague/1/56

AccuracyAccuracy(Results in the (Results in the individualindividual animal) animal)

Positive results:Positive results:Actual infectionActual infection True +veTrue +veCross-reactionsCross-reactions False +veFalse +veNegative results:Negative results:Absence of infectionAbsence of infection True -veTrue -veImproper testImproper test False -veFalse -veImproper timingImproper timing ....ImmunotoleranceImmunotolerance ....InhibitorsInhibitors ....Insensitive testsInsensitive tests** ....

* Analytical sensitivity* Analytical sensitivity

EVALUATION OF DIAGNOSTIC TESTSEVALUATION OF DIAGNOSTIC TESTS

• ValidityValidity• Reliability Reliability (repeatability)(repeatability)

Both are Both are long-runlong-run properties properties

1 2 3 4 5 6 7 8 9 10

123456789

123456789

123456789

1 2 3 4 5 6 7 8 9 10

123456789

123456789

123456789

1 2 3 4 5 6 7 8 9 10

123456789

123456789

123456789

1 2 3 4 5 6 7 8 9 10

123456789

123456789

123456789

ValidityValidityHighHighH

igh

Hig

hLowLow

Low

Low

AA BB

CC DD

Test statusTest status True True +ve+ve

Status -veStatus -ve TotalTotal

+ve+ve aa bb a + ba + b- ve- ve cc dd c + dc + dTotalTotal a + ca + c b + db + d

Sensitivity* Sensitivity* = = a/aa/a + + ccSpecificity* Specificity* = = d/bd/b + + dd

Fully:Fully:DiagnosticDiagnostic sensitivity sensitivityDiagnosticDiagnostic specificity specificity

OrganOrgan ConditionCondition Sensitivity Sensitivity (%)(%)

LiverLiver FascioliasisFascioliasis 19.0719.07

LungLung TuberculosisTuberculosis 5.555.55

Tongue and Tongue and musclemuscle

CysticercosisCysticercosis 0.000.00

Sensitivity of beef inspection Sensitivity of beef inspection procedures (Nigerian abattoir)procedures (Nigerian abattoir)

Test statusTest status True statusTrue status TotalsTotals

PositivePositive NegativeNegative

Flotation positiveFlotation positive 22(22(aa)) 1(1(bb)) 2323Flotation negativeFlotation negative 14(14(cc)) 43(43(dd)) 5757

TotalsTotals 3636 4444

Possible results of a diagnostic test exemplified by Possible results of a diagnostic test exemplified by application of a centrifugation/flotation technique to application of a centrifugation/flotation technique to

horses of known tapeworm status horses of known tapeworm status (from Proudman and Edwards, 1992)(from Proudman and Edwards, 1992)

Sensitivity = Sensitivity = a/a/((aa + + cc) = 22/36 = 0.61 (61%)) = 22/36 = 0.61 (61%)

Specificity = Specificity = d/d/((bb + + dd) = 43/44 = 0.98 (98%)) = 43/44 = 0.98 (98%)

TestTest Sensitivity Sensitivity SpecificitySpecificity

2G6 Ag-ELISA2G6 Ag-ELISA 71%71% 75%75%

Tr7 Ag-ELISATr7 Ag-ELISA 81%81% 78%78%

IgG ELISAIgG ELISA 89%89% 92%92%

Trypanosoma evansiTrypanosoma evansiDiagnostic testsDiagnostic tests

Microhaematocrit/mouse inoculation/ Microhaematocrit/mouse inoculation/ card agglutination test/card agglutination test/ELISAsELISAs

NB.NB. TwoTwo types of sensitivity types of sensitivity and specificityand specificity

1.1. DiagnosticDiagnostic sensitivity and sensitivity and Diagnostic Diagnostic specificityspecificity

Potential for confusion!Potential for confusion!

2.2. AnalyticalAnalytical sensitivity and sensitivity and Analytical Analytical specificityspecificity

Prevalence correction for Prevalence correction for misclassificationmisclassification

Test prevalence = PTest prevalence = PTT

True prevalence = PTrue prevalence = P

P =P = PPT T + specificity - 1+ specificity - 1Sensitivity + specificity - 1Sensitivity + specificity - 1

Test statusTest status

1.1. Discrete categoryDiscrete category(e.g. +ve or –ve to radiography; (e.g. +ve or –ve to radiography; presence or absence of a clinical sign)presence or absence of a clinical sign)

May be based on:May be based on:

2.2. Continuous variableContinuous variable (e.g. blood enzyme level; antibody (e.g. blood enzyme level; antibody titre)titre)

Num

ber o

f bird

sN

umbe

r of b

irds

Antibody titres (logarithmic scale)Antibody titres (logarithmic scale)

00<10<10

35 -35 -

40 -40 -

25 -25 -

20 -20 -

15 -15 -

5-5-

30 -30 -

10 -10 -

1010 2020 4040 8080 160160 320320 640640 12801280 25602560 >2560>2560

Continuous variables:Continuous variables:

Require definition of a Require definition of a ‘cut-off’‘cut-off’ value value

FMD: OIE protocolFMD: OIE protocol

Duplicate wells:Duplicate wells:Blocking ELISA:Blocking ELISA:

Positive if mean Positive if mean 1:45 1:45E.g. Well 1:1/32E.g. Well 1:1/32

Coded titre = 5Coded titre = 5Well 2:1/64Well 2:1/64Coded titre = 6Coded titre = 6

AMT = (5+6)/2AMT = (5+6)/2= 5.5= 5.5

GMT = 2GMT = 25.55.5

= 45.25= 45.25

Mean titre = 1:45Mean titre = 1:45

SpecificitySpecificity

SensitivitySensitivity

CC

Effect of various positive thresholds on the serodiagnostic Effect of various positive thresholds on the serodiagnostic interpretation of the enzyme immunoassay for the detection interpretation of the enzyme immunoassay for the detection

of bovine antibody to of bovine antibody to Brucella abortusBrucella abortusPositive Positive thresholdthreshold(OD 414 nm)(OD 414 nm)

SensitivitySensitivity SpecificitySpecificity Prevalence Prevalence of diseaseof disease

Predictive valuePredictive value

Non-vacNon-vac Vac Vac (S19)(S19)

+ve test+ve test -ve test-ve testEitherEither

Non-vacNon-vac VaccVacc

0.2200.220 0.9600.960 0.9900.990 0.8520.852 0.10.1 0.010.01 0.0010.001

0.920.920.500.500.090.09

0.420.420.060.060.010.01

1.001.001.001.001.001.00

0.2600.260 0.9430.943 0.9950.995 0.9300.930 0.10.1 0.010.01 0.0010.001

0.950.950.640.640.150.15

0.600.600.120.120.010.01

0.990.991.001.001.001.00

0.3000.300 0.9370.937 0.9980.998 0.9480.948 0.10.1 0.010.01 0.0010.001

0.980.980.840.840.340.34

0.670.670.150.150.020.02

0.990.991.001.001.001.00

0.3400.340 0.9200.920 0.9990.999 0.9690.969 0.10.1 0.010.01 0.0010.001

0.990.990.900.900.480.48

0.770.770.230.230.030.03

0.990.991.001.001.001.00

Defining a cut-off point for continuous Defining a cut-off point for continuous and ordinal test variablesand ordinal test variables

1.1. 2 or 3 standard deviations greater than the mean 2 or 3 standard deviations greater than the mean in unaffected animals in unaffected animals (does not consider (does not consider distribution in affected population)distribution in affected population)

Several approaches:Several approaches:

2.2. 55thth percentile in affected animals percentile in affected animals (does not (does not consider distribution in healthy population)consider distribution in healthy population)

3.3. Minimum number of incorrect diagnosesMinimum number of incorrect diagnoses

4.4. Minimum cost of incorrect diagnosesMinimum cost of incorrect diagnoses

ROC curveROC curve

(100-Specificity) (%)

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Sens

itivi

ty (%

)

AA

Definition of true statusDefinition of true status

True positiveTrue positive

Requires a ‘gold standard’Requires a ‘gold standard’

1.1. Experimental infectionsExperimental infections2.2. Several tests (+ve toSeveral tests (+ve to all all))

True negativesTrue negatives

1.1. Animals from countries where infection is known Animals from countries where infection is known to be absentto be absent

2.2. Several tests (-ve to Several tests (-ve to allall))

Sensitivity and specificitySensitivity and specificity

PrecisionPrecision

Required (confidence intervals)Required (confidence intervals)

‘‘Rule of thumb’:Rule of thumb’:

100 True +ves100 True +ves

200 True -ves200 True -ves

Possible results of a diagnostic testPossible results of a diagnostic test


DiseasedDiseased Not diseasedNot diseased

DiseasedDiseased aa bb a + ba + b

Not diseasedNot diseased cc dd c + dc + d

TotalsTotals a + ca + c b + db + d a + b + c + da + b + c + d

Test statusTest status True statusTrue status TotalsTotals+ve+ve -ve-ve

+ve+ve aa bb a + ba + b-ve-ve cc dd c + dc + dTotalsTotals a + ca + c b + db + d

SensitivitySensitivity = = a/a/((a +a + cc))SpecificitySpecificity = = d/d/((b + db + d))Predictive valuesPredictive values+ve test result+ve test result = = aa/(/(aa + + bb))-ve test result-ve test result = = dd/(/(cc + + dd))

Sensitivity and specificity of three Sensitivity and specificity of three screening tests for bovine brucellosisscreening tests for bovine brucellosis

Sensitivity Sensitivity (%)(%)

Specificity Specificity (%)(%)

Tube agglutination testTube agglutination test 62.062.0 99.599.5

Complement fixation testComplement fixation test 97.597.5 99.099.0

Brewer card testBrewer card test 95.295.2 98.598.5

Predictive value (positive result) of the tube agglutination test Predictive value (positive result) of the tube agglutination test for bovine brucellosis at three different prevalence levelsfor bovine brucellosis at three different prevalence levels

Test statusTest status True statusTrue status TotalsTotalsBrucellosis Brucellosis

presentpresentBrucellosis Brucellosis

absentabsentBrucellosis Brucellosis presentpresent

1860 (1860 (aa)) 485 (485 (bb)) 23452345

Brucellosis Brucellosis absentabsent

1140 (1140 (cc)) 96515 (96515 (dd)) 9765597655

TotalsTotals 30003000 9700097000 100000100000

Predictive value (positive result) =Predictive value (positive result) =

(Sensitivity = 62% Specificity = 99.5%)(Sensitivity = 62% Specificity = 99.5%)(Prevalence of brucellosis: 3%(Prevalence of brucellosis: 3%

aaaa + + bb

= 79.5%= 79.5%

Prevalence of brucellosis: 0.1%Prevalence of brucellosis: 0.1%


Brucellosis Brucellosis presentpresent



6060 500500 560560


4040 9940099400 9944099440

TotalsTotals 100100 9990099900 100000100000

Predictive value (positive result) = 10.7% Predictive value (positive result) = 10.7%

Prevalence of brucellosis: 0.01%Prevalence of brucellosis: 0.01%





66 500500 506506


44 9949099490 9949499494

TotalsTotals 1010 9999099990 100000100000

Predictive value (positive result) = 1.2% Predictive value (positive result) = 1.2%

Aggregate (herd-level) testingAggregate (herd-level) testing(1) Aggregate sensitivity(1) Aggregate sensitivitySeSeaggagg = 1-(1-P= 1-(1-PTT))nn

PPTT = = test prevalencetest prevalence

E.g.:E.g.: True prevalence, True prevalence, PP,, = 0.25= 0.25nn = = number of animals samplednumber of animals sampled

SeSe = 0.97,= 0.97,SpSp = 0.98,= 0.98,nn = 4= 4

PPTT = {= {PP x sensitivity} + {(1 – x sensitivity} + {(1 – PP) x (1 – specificity)}) x (1 – specificity)}= {0.25 x 0.97} + {(1 – 0.25) x (1 – 0.98)}= {0.25 x 0.97} + {(1 – 0.25) x (1 – 0.98)}= 0.2575= 0.2575

SeSeaggagg = 1 – (1 – 012575)= 1 – (1 – 012575)44

= 1 – 0.7425= 1 – 0.742544

= 1 – 0.3039= 1 – 0.3039= 0.70= 0.70

I.e. 70% of all affected herds have one or more test-positive I.e. 70% of all affected herds have one or more test-positive animals (30% of affected herds are missed)animals (30% of affected herds are missed)

Aggregate (herd-level) testingAggregate (herd-level) testing(2) Aggregate specificity(2) Aggregate specificitySpSpaggagg = (Specificity)= (Specificity)nn

E.g.:E.g.: SpSp = 0.98= 0.98nn = 4= 4

SpSpaggagg = 0.98= 0.9844

= 0.92= 0.92I.e. 92% of all unaffected herds have no test-positive I.e. 92% of all unaffected herds have no test-positive animals in each sample of 4 animals. animals in each sample of 4 animals. (8% of unaffected herds will test positive)(8% of unaffected herds will test positive)

Tests at aggregate (herd) levelTests at aggregate (herd) level

Net effectsNet effectsAs the number of animals tested increasesAs the number of animals tested increases

• Aggregate (herd) specificity Aggregate (herd) specificity • Aggregate (herd) sensitivity Aggregate (herd) sensitivity

Multiple testing strategiesMultiple testing strategies

Two main typesTwo main types

• ParallelParallel

• SerialSerial


ParallelParallel

• Positive to Positive to anyany test teste.g. Brucellosise.g. Brucellosis

Tests:Tests:1.1. Bacterial cultureBacterial culture2.2. Rose Bengal testRose Bengal test3.3. Milk ring testMilk ring test


SerialSerial

• Positive to Positive to allall tests testse.g. Swine dysenterye.g. Swine dysentery

Tests:Tests:First First fluorescent antibody test on faecal fluorescent antibody test on faecal or mucosal smearsor mucosal smearsThen Then positive confirmed by bacterial positive confirmed by bacterial cultureculture

Multiple testing strategies:Multiple testing strategies:sample calculationssample calculations

ExampleExample

Test A: Se 80%; Sp 60%Test A: Se 80%; Sp 60%Test B: Se 90; Sp 90%Test B: Se 90; Sp 90%True prevalence = 0.20 (20%)True prevalence = 0.20 (20%)True proportion negative = 80%True proportion negative = 80%

Multiple testing strategies:Multiple testing strategies:sample calculations: sample calculations: PARALLELPARALLEL

SensitivitySensitivityTest A detects 80%, leaving 20% undetectedTest A detects 80%, leaving 20% undetectedTest B detects 90% of the remaining 20% = 18%Test B detects 90% of the remaining 20% = 18%Therefore total +ve to Therefore total +ve to eithereither test = 80% + 18% test = 80% + 18%= 98% = = 98% = parallelparallel Sensitivity Sensitivity

SpecificitySpecificityTest A categorises 60% of true negatives as negativeTest A categorises 60% of true negatives as negativeTest B categorises only 90% of these as negativeTest B categorises only 90% of these as negativei.e. 90 x 60% = 54%, negative to i.e. 90 x 60% = 54%, negative to bothboth= = parallelparallel Specificity Specificity


Parallel testingParallel testingComplete the contingency table:Complete the contingency table:

Positive predictive value:Positive predictive value: 19.6/56.4 = 35% 19.6/56.4 = 35%

True +True +(%)(%)

True -True -(%)(%)

TotalTotal

Test +(either) (%)Test +(either) (%) 19.619.6 36.836.8 56.456.4

Test - (both) (%)Test - (both) (%) 0.40.4 43.243.2 43.643.6

TotalTotal 2020 8080 100100

Negative predictive value:Negative predictive value: 43.2/43.6 = 99% 43.2/43.6 = 99%

Multiple testing strategies:Multiple testing strategies:sample calculations:sample calculations:SERIALSERIAL

SensitivitySensitivityTest A detects 80%Test A detects 80%

Test B detects 90% of these as positiveTest B detects 90% of these as positive

Therefore % positive to Therefore % positive to bothboth tests tests

= 80% x 90% = 72% = 80% x 90% = 72% == SerialSerial Sensitivity Sensitivity

SpecificitySpecificityTest A correctly categorises 60% of true negatives, Test A correctly categorises 60% of true negatives, leaving 40%leaving 40%Test B categorises 90% of these as negative Test B categorises 90% of these as negative

= 90% x 40% = 36%, negative= 90% x 40% = 36%, negative

= = SerialSerial Specificity Specificity

Therefore total number negative to either testTherefore total number negative to either test= 60% + 36% = 96%= 60% + 36% = 96%


Serial testingSerial testingComplete the contingency table:Complete the contingency table:

Positive predictive value:Positive predictive value: 14.4/17.6 = 82% 14.4/17.6 = 82%

True +True +(%)(%)

True -True -(%)(%)

TotalTotal

Test +(both) (%)Test +(both) (%) 14.414.4 3.23.2 17.617.6

Test - (either) (%)Test - (either) (%) 5.65.6 76.876.8 82.482.4

TotalTotal 2020 8080 100100

Negative predictive value:Negative predictive value: 76.8/82.4 = 93% 76.8/82.4 = 93%

Multiple testing strategies:Multiple testing strategies:

• Third typeThird typeRepeated testing (negative-herd retesting)Repeated testing (negative-herd retesting)

• Exclusive to veterinary medicineExclusive to veterinary medicine• Applied to Applied to herds herds not individualsnot individuals• Using the Using the samesame test e.g. tuberculosis test e.g. tuberculosis

E.g., Comparative intradermal tuberculin test:E.g., Comparative intradermal tuberculin test:1.1. Reactors culled;Reactors culled;2.2. -ve and inconclusive reactors tested every 60 -ve and inconclusive reactors tested every 60

days until two clear herd tests are achieved;days until two clear herd tests are achieved;3.3. Then return to 4-year testing cycleThen return to 4-year testing cycle

Diagnostic Testing

Documents

Transcript of Diagnostic Testing