Validity of Self-reported Periodontal disease:

a systematic review

Research protocol

July 2015

Hadeel M. Abbood1

Dr. Tatiana V. Macfarlane1

Dr. George Cherukara1

1 University of Aberdeen, Aberdeen Dental School and Hospital, AB25 2ZD

2

Table of contents

Introduction .............................................................................................................................. 3

Rationale: Why it is important ............................................................................................... 3

Aim ............................................................................................................................................ 4

Objectives.................................................................................................................................. 4

Method ...................................................................................................................................... 4

STUDY SELECTION ................................................................................................................................................... 4 SEARCH STRATEGY ................................................................................................................................................... 4 INCLUSION AND EXCLUSION CRITERIA .......................................................................................................................... 4 DATA EXTRACTION .................................................................................................................................................. 5 QUALITY ASSESSMENT .............................................................................................................................................. 5

Milestone ................................................................................................................................... 6

References:................................................................................................................................ 7

Table of appendixes

Appendix 1 Map of searching terms in MEDLINE ................................................................... 9 Appendix 2 Flow diagram of study selection process ............................................................. 10

Appendix3 Data extraction form ............................................................................................ 11 Appendix 4 Quality assessment tool (AMSTAR) ................................................................... 12

Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy ................................ 14

3

Introduction

Periodontal disease is a group of diseases of the periodontal tissues that lead to attachment

loss and alveolar bone destruction. The prognosis of periodontal disease in some but not all

patients may results in tooth loss, especially if there is no treatment (Löe et al. 1986).

Diagnosis depends on the findings from the history and clinical examination of the patient

(Clerehugh, Tugnait & Genco 2013), which can includes several separate indices. These

indices include Gingival Index (GI), Plaque Index (PLI), bleeding on probing (BOP), loss of

attachment (LOA), furcation involvement, and tooth mobility. The information gathered will

allow the clinician to decide on a logical basis whether additional special tests are required,

such as radiographic examination and tests to detect biomarkers in the saliva and the gingival

crevicular fluid. (Chapple, Gilbert 2002).

Self-reporting is easy to apply, cost-effective, and provides immediate results. It can be relied

upon for national public health surveys (Tourangeau, Yan 2007). Self-report has gained

popularity as an important method in screening and motivational interviewing (Lundahl,

Burke 2009). Self-rated health assessment and utilization of healthcare services are important

determinants of health, and have particular relevance for public health (Chakraborty et al.

2003). Self-report is a potent, skilled, and accepted means of assessing many diseases, such

as assessing cancer, cardiovascular disease (Newell et al. 1999), and juvenile rheumatoid

arthritis (Wright et al. 1994). It can also be used to assess risk factors for disease, such as diet

(Willett 1990, Rimm et al. 1992), physical activity (Wolf et al. 1994), high blood pressure

(Tormo et al. 2000), and general health (Sheridan, Mulhern & Martin 1998). An example for

self-reported questionnaires survey is the Queensland preventive health surveys that are

designed to make the respondent self-assess their own health (Queensland Government

2015).

The Scottish health survey (SHeS) is another example of self-reported health. It provides a

detailed picture of the health of the Scottish population in private households. It is designed

to make a major contribution to the monitoring of health in Scotland. SHeS also monitor oral

health, especially the periodontal health and the oral hygiene using a self-reported

questionnaire (the Scottish Government 2015).

Rationale: Why it is important

For public health interventions against periodontal disease to be developed, implemented, and

evaluated, an adequate monitoring of the disease is required. Clinical assessment of

periodontal disease is extremely need more resources and facilities, and cannot be used in

several state-based surveillance systems. The use of valid, economical, and low-resource self-

reported measures of periodontal disease would be of great benefit in a variety of ways.

Firstly, it would aid in studying periodontal disease epidemiologically at a larger scale than is

feasible with the present clinical measures, because using surveys can incorporate a larger

population than clinical examination. Secondly, questions regarding periodontal disease

could easily be added to ongoing studies to evaluate associations with other diseases and

conditions. The use of self-report would allow for an easier and low-cost method of obtaining

data for research and would support the evolution of oral health programs (Siegal, Martin &

Kuthy 1988, Kallio 1996). Finally, self-reported measures would allow for surveillance of the

periodontal condition of populations over time, in national, state, or regional surveillance

programs.

4

It has been over 10 years since the last systematic review of the validity of self-reported

periodontal disease was published (Blicher et al 2005). In this time, several studies using self-

reported periodontal disease have been reported.

It is the time now to conduct another systematic review, and further assess the validity of

self-reported periodontal disease. Such a review will update the previous systematic review

by Blicher et al.

Aim

To update the systematic review published in 2005 by Blicher et al. which reviewed literature

up to June (2nd

week) 2004.

Objectives

1. Summarise new findings and combine them with the previous review by Blicher et al.

2. Identify the most effective means of self-reporting of periodontitis.

Method

Study selection

The research aimed to analyse all the papers that assessed the validity of self-reported

gingival and periodontal diseases by comparing with a ‘clinical gold standard’.

Search strategy

MEDLINE search strategy was adopted from 1966-2004 review by Blicher et al. 2005 and

will be updated to include studies until 2015. Embase database will be added to the electronic

search.

The search strategy terms will be grouped into three categories. Group I terms will be related

to gingivitis and periodontal disease (gingivitis, gingival, gingival disease, periodontal,

periodontal disease, periodontitis, tooth mobility, loss of attachment, bleeding gum); group II

will include terms associated with self-reporting (questionnaire, self-assessment, self-report,

self-reported), and group III terms concerning process of validation (comparison, compared,

validity, validation) (see appendix 1). Intra-group terms will be combined with the Boolean

commands ‘OR’, and Inter-group terms will be combined with ‘AND’.

Two journals (Journal of Clinical Periodontology and Periodontology 2000) will be searched

manually from 2005-2015. Publications in languages other than English will not be included.

Inclusion and exclusion criteria

We will include studies in English language only; any other languages will be excluded. The

studies to be included should use the clinical gold standards for each questionnaire.

Studies that used self-report but did not validate these measures or validate them in overall or

in combination of more than one oral health measure will be excluded.

5

Data extraction

After completing the database search and manual search, two independent reviewers will

screen the title and the abstract of each study following the inclusion criteria. If disagreement

occurs between the two reviewers, a third reviewer will be asked consulted. Data extraction

will be done by two independent reviewers. They will extract the required data from each

study. These information include: study type, population description (sample size, mean age,

gender, method of recruitment), question used for self-report, clinical gold standard, and time

of data collection. A standardised data extraction form will be used for data extraction (see

appendix 3). The data extracted will be used to construct a 2*2 table to calculate the

sensitivity and specificity. Predictive values will also be used. If some information is missing

we will contact the authors of the studies to provide us with the required information.

The data from research studies will be used to create tables with group questions asked by

subject and contain information regarding specific phrasing, reference gold standard. Validity

of self-assessment will be quoted as reported by the authors: percentage of agreement,

sensitivity/specificity, p-value, predictive values, correlation/regression coefficient or

descriptive methods as reported by the authors, We will calculate an additional statistics

based on the data provided in the manuscript when needed and possible. Good validity level

will be arbitrarily defined as sum of either: positive and negative values, or specificity and

sensitivity, exceeding or equal to 120%. This value was arbitrary chosen by Blicher et al.

(2005). If a study compared a self-reported question against multiple clinical measures, only

the most relevant one will be chosen.

Quality assessment

The studies will be analysed according to the guidelines for systematic reviews of diagnostic

studies (Devillé et al. 2002). Each publication will be reviewed using specially designed data

collection sheets (see appendix 3) with respect to: population profile, selection criteria, type

of self-reporting, nature of questions, reference clinical gold standard, and results of

validation.

Quality assessment tool (QUADAS-2) will be used to evaluate the risk of bias and

applicability of primary diagnostic accuracy for each study. QUADAS-2 is designed to assess

the quality of primary diagnostic accuracy studies; the tool consists of four key domains

regarding patient selection, index test, reference standard, flow and timing. The tool is

completed in four phases: 1) state the review question; 2) develop review specific guidance;

3) review the published flow diagram for the primary study or construct a flow diagram if

none is reported; 4) judgement of bias and applicability. Each domain is assessed in terms of

the risk of bias and the first three are also assessed in terms of concerns regarding

applicability. To help reach a judgement on the risk of bias, signalling questions are included.

These flag aspects of study design related to the potential for bias and aim to help reviewers

make risk of bias judgments. Risk of bias is judged as “low”, “high”, or “unclear”. If all

signalling questions for a domain are answered “yes” then risk of bias can be judged “low”. If

any signalling question is answered “no” this flags the potential for bias. Review authors then

need to use the guidelines developed in phase 2 to judge risk of bias. The “unclear” category

should be used only when insufficient data are reported to permit a judgment. QUADAS-2 is

available on the link below:

http://www.bris.ac.uk/medialibrary/sites/quadas/migrated/documents/quadas2.pdf

QUADAS-2 is available on the link below:

http://www.bris.ac.uk/medialibrary/sites/quadas/migrated/documents/quadas2.pdf

6

Quality assessment tool for systematic review (AMSTAR) will be used to evaluate the

previous systematic review by Blicher et al. in yes/no fashion (Shea et al. 2007)(see appendix

4).

Study protocol will be registered with PROSPERO http://www.crd.york.ac.uk/PROSPERO/

We aim to publish this review in a peer reviewed journal.

We will follow Prisma guidelines for reporting of systematic reviews http://www.prisma-

statement.org/ .

We will register our study in systematic review data repository (SRDR)

http://www.srdr.ahrq.gov/home/index

Milestone

Protocol finalised - August 2015

Search strategy- August to September 2015

Study selection -September

Data extraction –September - October

Quality assessment-October – November

Summary of results –November

Writing final report -November

Circulation of report for comment –December

Submission for publication –January 2015

7

References:

Blicher, B., Joshipura, K. & Eke, P. 2005, "Validation of self-reported periodontal disease: a

systematic review", Journal of dental research, vol. 84, no. 10, pp. 881-890.

Chakraborty, N., Islam, M.A., Chowdhury, R.I., Bari, W. & Akhter, H.H. 2003,

"Determinants of the use of maternal health services in rural Bangladesh.", Health

Promot Int, vol. 18, no. 4, pp. 327-337.

Chapple, I.L.C. & Gilbert, A.D. 2002, "Understanding Periodontal Diseases: Assessment and

Diagnostic Procedures in Practice." in Quintessence Publishing Co., , pp. 111-128.

Clerehugh, V., Tugnait, A. & Genco, J.R. 2013, Periodontology at a glance, 4th edn, Wiley-

Blackwell.

Devillé, W.L., Buntinx, F., Bouter, L.M., Montori, V.M., de Vet, H.C., van der Windt, D.A.

& Bezemer, P.D. 2002, "Conducting systematic reviews of diagnostic studies: didactic

guidelines.", BMC Med Res Methodol, vol. 3, no. 2, pp. 9.

Kallio, P. 1996, "Self-assessed bleeding in monitoring gingival health among adolescents.",

Community Dent Oral Epidemiol, vol. 24, pp. 128-132.

Löe, H., Anerud, A., Boysen, H. & Morrison, E. 1986, "Natural history of periodontal disease

in man. Rapid, moderate and no loss of attachment in Sri Lankan laborers 14 to 46 years

of age", Journal of Clinical Periodontology, vol. 13, no. 5, pp. 431-445.

Lundahl, B. & Burke, B.L. 2009, "The effectiveness and applicability of motivational

interviewing: A practice-friendly review of four meta-analyses.", Journal of Clinical

Psychology, vol. 65, no. 11, pp. 1232-1245.

Newell, S.A., Girgis, A., Sanson-Fisher, R.W. & Savolainen, N.J. 1999, "The accuracy of

self-reported health behaviors and risk factors relating to cancer and cardiovascular

disease in the general population: a critical review.", American journal of preventive

medicine, vol. 17, pp. 211-229.

Queensland Government 2015, 12 May, 2015-last update, Queensland preventive health

surveys. Available: https://www.health.qld.gov.au/epidemiology/publications/phs-

qld.asp [2015, 07/22].

Rimm, E.B., Giovannucci, E.L., Stampfer, M.J., Colditz, G.A., Litin, L.B. & Willett, W.C.

1992, "Reproducibility and validity of an expanded self-administered semiquantitative

food frequency questionnaire among male health professionals.", American Journal of

Epidemiology, vol. 135, pp. 1114-1126.

Shea, B.J., Grimshaw, J.M., Wells, G.A., Boers, M., Andersson, N., Hamel, C., Porter, A.C.,

Tugwell, P., Moher, D. & Bouter, L.M. 2007, "Development of AMSTAR: a

measurement tool to assess the methodological quality of systematic reviews", BMC

Medical Research Methodology, vol. 7, no. 10.

8

Sheridan, C.L., Mulhern, M. & Martin, D. 1998, "Validation of a self-report measure of

somatic health.", Psychol Rep, , pp. 679-687.

Siegal, M.D., Martin, B. & Kuthy, R.A. 1988, "Usefulness of a local oral health survey in

program development.", J Public Health Dent, vol. 48, pp. 121-124.

the Scottish Government 2015, Wednesday, March 04, 2015-last update, Scottish Health

Survey. Available: http://www.gov.scot/Topics/Statistics/Browse/Health/scottish-health-

survey [2015, 07/22].

Tormo, M.J., Navarro, C., Chirlaque, M.D. & Barber, X. 2000, "Validation of self diagnosis

of high blood pressure in a sample of the Spanish EPIC cohort: overall agreement and

predictive values. EPIC Group of Spain.", J Epidemiol Community Health, vol. 54, pp.

221-226.

Tourangeau, R. & Yan, T. 2007, "Sensitive questions in surveys.", Psychological Bulletin,

vol. 133, no. 5, pp. 859.

Willett, W. 1990, Nutritional epidemiology. Oxford University Press, New York.

Wolf, A.M., Hunter, D.J., Coblitz, G.A., Manson, J.E., Stampfer, M.J., Corsano, K.A. & et al.

1994, "Reproducibility and validity of a self-administered physical activity

questionnaire.", International journal of epidemiology, vol. 23, pp. 991-999.

Wright, F.V., Law, M., Crombie, V., Goldsmith, C.H. & Dent, P. 1994, "Development of a

self-report functional status index for juvenile rheumatoid arthritis.", Journal of

Rheumatology, vol. 21, pp. 536-544.

9

Appendix 1 Map of searching terms in MEDLINE

AND

All the groups results will be combined with

AND

AND Only Group I and II will be combined with AND for more results (some studies do not use the validity

terms in their titles)

Group I terms regarding

periodontal disease

Group II terms regarding

self-report

Group III terms regarding

validity

10

Appendix 2 Flow diagram of study selection process

Records that will be identified through

database searching (n=?)

Scr

een

ing

Incl

ud

ed

Eli

gib

ilit

y

Iden

tifi

cati

on

Additional records that will be identified through hand-searching

journals (n = ?)

Records screening (n = ?)

Records after screening (n = ?)

Excluding records due to language or validity (n

= ?)

Full-text articles assessment for eligibility

(n = ?)

Full-text articles excluded (n = ?)

- No PD - No self-reported - Neither PD nor self-

reported - Not looking at

validity of self-reporting

Studies that will be included in qualitative synthesis

(n =?)

Removal of Duplication (n = ?)

11

Appendix3 Data extraction form

ID: First author’s sure name:

Title: Country of study:

Year of publication

Type of study

Participants Number: Male Female Age range:

Data extraction

Self-reported

Question

Clinical Gold

Standard

Validity test DOR

(Diagnistic odds retio)

12

Appendix 4 Quality assessment tool (AMSTAR)

1. Was an ‘a priori’ design provided?

The research question and inclusion criteria should be established before

the conduct of the review.

Yes

No

Can’t answer

Not applicable

2. Was there duplicate study selection and data extraction?

There should be at least two independent data extractors and a

consensus procedure for disagreements should be in place.

Yes

No

Can’t answer

Not applicable

3. Was a comprehensive literature search performed?

At least two electronic sources should be searched. The report must

include years and databases used (e.g. Central, EMBASE, and

MEDLINE). Key words and/or MESH terms must be stated and where

feasible the search strategy should be provided. All searches should be

supplemented by consulting current contents, reviews, textbooks,

specialized registers, or experts in the particular field of study, and by

reviewing the references in the studies found.

Yes

No

Can’t answer

Not applicable

4. Was the status of publication (i.e. grey literature) used as an

inclusion criterion?

The authors should state that they searched for reports regardless of

their publication type. The authors should state whether or not they

excluded any reports (from the systematic review), based on their

publication status, language etc.

Yes

No

Can’t answer

Not applicable

5. Was a list of studies (included and excluded) provided?

A list of included and excluded studies should be provided.

Yes

No

Can’t answer

Not applicable

6. Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies

should be provided on the participants, interventions and outcomes. The

ranges of characteristics in all the studies analyzed e.g. age, race, sex,

relevant socioeconomic data, disease status, duration, severity, or other

diseases should be reported.

Yes

No

Can’t answer

Not applicable

7. Was the scientific quality of the included studies assessed and

documented?

‘A priori’ methods of assessment should be provided (e.g., for

effectiveness studies if the author(s) chose to include only randomized,

double-blind, placebo controlled studies, or allocation concealment as

inclusion criteria); for other types of studies alternative items will be

relevant.

Yes

No

Can’t answer

Not applicable

13

8. Was the scientific quality of the included studies used

appropriately in formulating conclusions?

The results of the methodological rigor and scientific quality should be

considered in the analysis and the conclusions of the review, and

explicitly stated in formulating recommendations.

Yes

No

Can’t answer

Not applicable

9. Were the methods used to combine the findings of studies

appropriate?

For the pooled results, a test should be done to ensure the studies were

combinable, to assess their homogeneity (i.e. Chi-squared test for

homogeneity, I²). If heterogeneity exists a random effects model should

be used and/or the clinical appropriateness of combining should be

taken into consideration (i.e. is it sensible to combine?).

Yes

No

Can’t answer

Not

applicable

10. Was the likelihood of publication bias assessed?

An assessment of publication bias should include a combination of

graphical aids (e.g., funnel plot, other available tests) and/or statistical

tests (e.g., Egger regression test).

Yes

No

Can’t answer

Not applicable

11. Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both the

systematic review and the included studies.

Yes

No

Can’t answer

Not applicable

14

Appendix 5 Methodology Checklist 5: Studies of Diagnostic Accuracy

S I G N

Methodology Checklist 5: Studies of Diagnostic Accuracy

This checklist is based on the work of the QUADAS2 team at Bristol Univeristy (http://www.bris.ac.uk/quadas/).

Study identification (Include author, title, reference, year of publication)

Guideline topic: Key Question No:

Before completing this checklist, consider:

1. Is the paper really a study of diagnostic accuracy? It should be comparing a specific diagnostic test against another, and not a general paper or comment on diagnosis.

2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist..

Reason for rejection: Reason for rejection: 1. Paper not relevant to key question □ 2. Other reason □ (please specify):

Checklist completed by:

All the questions in the following sections have associated footnotes providing short explanations behind each of the questions. Users who want more detailed explanations should consult the QUADAS-2: Background Document.

DOMAIN 1 – PATIENT SELECTION

Risk of bias

In a well conducted diagnostic study… Is that true in this study?

1.1 A consecutive sequence or random selection of patients is enrolled.

i

Yes

No

Can’t say

1.2 Case – control methods are not used.ii Yes

No

Can’t say

1.3 Inappropriate exclusions are avoided.iii Yes

No

Can’t say

Applicability

1.4 The included patients and settings match the key question.

iv

Yes

No

Can’t say

DOMAIN 2 – INDEX TEST

Risk of bias

In a well conducted diagnostic study… Is that true in this study?

2.1 The index test results interpreted without knowledge of the results of the reference standard.

v

Yes

No

Can’t say

2.2 If a threshold is used, it is pre-specified.vi Yes

No

Can’t say

15

Applicability

2.3 The index test, its conduct, and its interpretation is similar to that used in practice with the target population of the guideline.

vii

Yes

No

Can’t say

DOMAIN 3 – REFERENCE STANDARD

Risk of bias

In a well conducted diagnostic study… Is that true in this study?

3.1 The reference standard is likely to correctly identify the target condition.

viii

Yes

No

Can’t say

3.2 Reference standard results are interpreted without knowledge of the results of the index test.

ix

Yes

No

Can’t say

Applicability

3.3 The target condition as defined by the reference standard matches that found in the target population of the guideline.

x

Yes

No

Can’t say

DOMAIN 4 – FLOW AND TIMING

Risk of bias

In a well conducted diagnostic study… Is that true in this study?

4.1 There is an appropriate interval between the index test and reference standard.

xi

Yes

No

Can’t say

4.2 All patients receive the same reference standard.xii

Yes

No

Can’t say

4.3 All patients recruited into the study are included in the analysis.

xiii

Yes

No

Can’t say

SECTION 5: OVERALL ASSESSMENT OF THE STUDY

5.1 How well was the study done to minimise bias?

Code as follows:xiv

High quality (++)

Acceptable (+)

Low quality (-)□

Unacceptable – reject 0

5.2 What is your assessment of the applicability of this study to our target population?

Directly applicable

Some indirectness (Please explain in the following section for Notes)

5.2 Notes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question.

16

i Studies should enrol either all eligible patients suspected of having the target condition during a specified period, or a random sample of those patients. The essential point is that investigators should have no freedom of choice as to which individual patients are or are not included. ii There is evidence that studies comparing patients with known disease with a control group without

the condition tend to exaggerate diagnostic accuracy. iii Inappropriate exclusions may result in either overestimates (eg by excluding ‘difficult to diagnose’

patients) or underestimates (eg by excluding patients with ‘red flags’ suggesting presence of disease) of the degree of diagnostic accuracy. iv Patients included in the study should match the target population of the guideline in terms of severity

of the target condition, demographic features, presence of differential diagnosis or co-morbidity, setting of the study and previous testing protocols. v This is similar to the question of ‘blinding’ in intervention studies. The index test should always been

done first, or by a separate investigator with no knowledge of the outcome of the reference test. vi Bias can be introduced if a threshold level is set after data has been collected. Any minimum

threshold should be specified at the start of the trial. vii

Variations in test technology, execution, or interpretation (eg use of a higher ultrasound transducer frequency) may affect estimates of diagnostic accuracy. viii

Estimates of test accuracy are based on the assumption that the reference standard is 100% sensitive (=accurately diagnoses the target condition). ix This is the similar to question 2.1, but in this case relates to making sure the reference standard is

applied without any prior knowledge of the outcome of previous tests. x The definition of the target condition used when testing the reference standard may differ from that

used by the NHS in Scotland. eg threshold levels used in laboratory cultures may differ. xi The index test and reference standard should be performed as close together in time as possible,

otherwise changes in the patients condition is likely to invalidate the results. xii

In some cases the choice of reference standard may be influenced by the outcome of the index test or the urgency of the need for diagnosis. Use of different reference standards is likely to lead to overestimates of both sensitivity and specificity. xiii

Not including all patients in the analysis may lead to bias as there may be some systematic difference between those lost to follow-up and those analysed. xiv

Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (-): Either most criteria not met, or significant flaws relating to key aspects of study design. Conclusions likely to change in the light of further studies.