Comparing the reliability and diagnostic accuracy of routine NHS laboratories

and a research laboratory in the Diagnosis of Urinary Tract infection in

Young children (DUTY) study

Kim Harman on behalf of the DUTY study team

Research aim

Aim

To compare the reliability and diagnostic accuracy of two different types of laboratory in the urinalysis with regard to UTI

BackgroundThere is a ‘National standard method’ for urinalysis but it is unknown how much this is adhered to.

Few studies have compared routine NHS laboratory reporting of results against a single standardised laboratory.

Evidence for laboratory processes is weak.

Whole study design• Cross sectional diagnostic accuracy study with prospective

f/u• ‘Open’ inclusion criteria

– Children <5 years old– Presenting to primary care with an acute (<28 days) illness

• Constitutionally unwell and / or • Urinary symptoms, even if clinician ‘knows’ the diagnosis

• Excluded– Neurogenic bladder– Antibiotics <7 days– Trauma as main concern

Data collected

• Data collection– Baseline demographics, symptoms and signs (n=63) rated

none/mild/moderate or severe

• Urine sample– Preference clean catch, then Newcastle Nappy pad

– Urine dipstick testing

• Urine sample split and sent to– Local NHS laboratory for routine microscopy and culture

– More detailed research laboratory (RL) analysis

Methods (1)Definitions of UTI

• NHS laboratory– Report of pure/predominant growth of a uropathogen (e.g. e.coli,

proteus, etc) at ≥105 cfu/ml

• Research laboratory – Growth of ≥105 cfu/ml of a single uropathogen (“pure growth”) – Or growth of ≥105 of a uropathogen with ≥3 log10 (1000-fold)

difference between growth of this and the next species (“predominant growth”)

Methods(2)

Association with a priori specified variables clearly associated with the presence of UTI from the literature

Pain/crying when passing urinePassing urine more oftenChanges in urine appearanceTemperature ≥ 39°CDipstick nitrite positiveDipstick leukocyte positive

(Hay AD, Whiting P, Butler CC. How best to diagnose urinary tract infection in preschool children in primary care? BMJ 2011; 343:d6316.)

General results

1. 69 Local NHS Labs involved

2. 6,079 Local NHS Laboratory urines

3. 5,107 Research Laboratory urines

4. 4,808 children with predictors from CRF & urine at both labs

Study participantsCategory Age <3 years Age 3-5 years

Gender Male 1439 (49.9%) 919 (47.8%)Female 1445 (50.1%) 1005 (52.2%)

Age (years) 0 to <1 1016 (35.2%)1 to <2 942 (32.7%)2 to <3 926 (32.1%)3 to <4 1099 (57.1%)4 to <5 825 (42.9%)

Sample method Clean catch 758 (26.3%) 1861 (96.7%)Nappy pad 2126 (73.7%) 63 (3.3%)

Urine sample results

Result Age <3 years Age 3-5 years

NHS laboratory Positive 189 (6.6%) 62 (3.2%)

Research

laboratoryPositive 51 (1.8%) 37 (1.9%)

Kappa 95% CIBoth collection methods 0.36 (0.29, 0.43)

Clean catch 0.54 (0.45, 0.63)Nappy pad 0.20 (0.12, 0.28)

≥ 3 years 0.53 (0.41, 0.65)Clean catch 0.55 (0.43, 0.67)Nappy pad N/A

<3 years 0.29 (0.21, 0.36)Clean catch 0.52 (0.37, 0.67)Nappy pad 0.20 (0.12, 0.28)

Similar patterns were seen when comparisons were further stratified into age groups <2 and ≥2 to <3 years. N/A is not available – too few results

Between lab comparisons

Associations with variablesClean catch Nappy pad

Research laboratory AUC (95% CI) 0.86 (0.79, 0.92) 0.79 (0.70,0.88)

NHS laboratoriesAUC (95% CI) 0.75 (0.69,0.80) 0.65 (0.61,0.70)

00

.51

1 0.5 0

Clean catch

UTI + ve in NHS

00

.51

1 0.5 0

Nappy pad

00

.51

1 0.5 0

UTI +ve in RL

00

.51

1 0.5 0

Se

nsitiv

ity

Specificity

Between lab comparisons

Summary of main results

• There is worse than expected agreement between the labs and between collection methods

• Associations of microbiological positivity with pre-specified signs/symptoms were lower for:

– NHS positivity than for research laboratory positivity

– Nappy pad samples than for clean catch samples

Implications - microbiological

NHS labs may wish to adopt the methods of the research laboratory.

Implications – clinical

More effort should be spent on obtaining higher quality (clean catch) samples, even in the very young

Implications - research

Leading hypothesis is that for nappy pad samples the NHS labs overdiagnose UTI because of failure to identify contamination

NIHR disclaimer

This project was funded by the NIHR Health Technology Assessment Programme (project number 08/66/01) and will be published in full as a Health Technology Assessment.

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA programme, NIHR, NHS or the Department of Health.

Thank you teamInvestigatorsAlastair Hay (co-CI) Study ManagersChris Butler (co-CI) Jonathan Sterne Harriet DowningPeter Brindle Judith van der Voort Marilyn PetersBrendan Delaney Penny Whiting Kate RumsbyJan Dudley Data Manager Emma Thomas-JonesMargaret Fletcher Cathy Lisles Cherry-Ann WaldronWill Hollingworth Database manager StatisticiansKerenza Hood Stevo Durbaba Kate BirnieRobin Howe Microbiologist Michael LawtonPaul Little Mandy Wootton Tim PicklesAlasdair MacGowan Health economist DUTY nurses / CSOsKathy O’Brien John Busby

Primary care clinicians Children & their families

Associations in Research Lab

Central laboratoryClean catch Nappy pad

OR (95% CI) p OR (95% CI) p

Pain/crying passing urine 6.0 (3.0, 11.8) <0.001 1.4 (0.3, 7.0) 0.716

Passing urine more often 0.8 (0.4, 1.7) 0.543 1.2 (0.3, 4.4) 0.839

Change in urine appearance 3.1 (1.6, 6.1) 0.001 3.1 (1.2, 7.9) 0.019

Temperature ≥39°C 1.7 (0.6, 5.1) 0.333 1.1 (0.1, 8.8) 0.930

Dipstick: nitrite +ve 11.2 (5.4, 23.1) <0.001 5.2 (2.4, 11.3) <0.001

Dipstick: leukocyte +ve 5.3 (2.8, 10.0) <0.001 4.1 (1.9, 8.9) <0.001

AUC (95% CI) 0.86 (0.79 0.92) 0.79 (0.70, 0.88)

Associations in NHS labs

NHS laboratoryClean catch Nappy pad

OR (95% CI) p OR (95% CI) p

Pain/crying passing urine 2.9 (1.6, 5.1) <0.001 1.1 (0.4, 3.1) 0.838

Passing urine more often 0.6 (0.3, 1.1) 0.073 0.7 (0.3, 1.5) 0.370

Change in urine appearance 3.0 (1.8, 4.9) <0.001 2.1 (1.3, 3.5) 0.005

Temperature ≥39°C 1.7 (0.8, 3.8) 0.157 0.7 (0.2, 2.2) 0.526

Dipstick: nitrite +ve 7.6 (4.1, 14.1) <0.001 2.0 (1.4, 2.9) 0.001

Dipstick: leukocyte +ve 3.1 (1.9, 5.1) <0.001 3.1 (2.1, 4.4) <0.001

AUC (95% CI) 0.75 (0.69, 0.80) 0.65 (0.61, 0.70)

0.1

.2.3

.4s.e

. o

f ka

ppa

-.5 0 .5 1kappa

Funnel plot 95% confidence limits