Eindverslag Stage wetenschap

The Control of Allergic Rhinitis and Asthma

Test: validation of the Dutch version

Sander van der Leeuw

s1428098

Facultair begeleider: Prof. T. van der Molen

Dagelijks begeleider: Dr. B.M.J. Flokstra

Rijksuniversiteit Groningen, UMCG, Afdeling Huisartsgeneeskunde

1

The Control of Allergic Rhinitis and Asthma Test: validation

of the Dutch version

S van der Leeuw

Rijksuniversiteit Groningen, Universitair Medisch Centrum Groningen, Afdeling Huisartsgeneeskunde en GRIAC research instituut, Groningen

Background: Simultaneous treatment of allergic rhinitis and asthma has been

recommended by the Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines.

The Portuguese Control of Allergic Rhinitis and Asthma Test (CARAT) was therefore

developed to monitor the control of both asthma and allergic rhinitis in primary and

secondary care. This questionnaire was translated to Dutch.

Objectives: The primary objectives of this study were to evaluate the internal

consistency, construct validity and longitudinal validity of the Dutch CARAT.

Furthermore, test-retest reliability and Minimal Clinically Important Difference (MCID)

were investigated.

Methods: The Dutch CARAT study is a longitudinal study consisting of three

measurements with one month intervals. 176 patients diagnosed with asthma and/or

rhinitis from three primary care practices and three secondary care centers were

approached. Cronbach’s alpha was used to evaluate internal consistency. CARAT

scores (ranging from 0-30) were compared to the Asthma Control Questionnaire

(ACQ) and VAS scores concerning airway symptoms. Spearman’s correlation

coefficients were used to determine construct and longitudinal validity. For the

longitudinal analysis delta scores (t2-t1 and t3-t2) of the CARAT, ACQ and VAS were

used. Test-retest reliability and Minimal Clinically Important Difference (MCID) were

evaluated using Global Rating of Change (GRC) scores to define categories

according to change in symptoms over time.

Results: A total of 93 patients were included. The mean CARAT score for all patients

was 19.4 (SD 6.80). Cronbach’s alpha for the CARAT was 0.82. Correlation

coefficients between CARAT and the ACQ and VAS questions ranged from

0.64(p<0.01) to 0.76(p<0.01). Longitudinally, correlation coefficients between delta

CARAT scores and delta ACQ and VAS ranged from 0.41 to 0.67 (p<0.01) for both

one month intervals. Calculations for test-retest reliability showed correlation

coefficients of 0.74(p<0.01) and 0.78(p<0.01) between CARAT scores of patients in

the ‘no change’-category according to GRC. The MCID for the CARAT was 3.50.

Conclusion: The Dutch CARAT has high internal consistency and good construct

validity and is therefore comparable to findings according to the Portuguese CARAT.

The Dutch CARAT also proves to be longitudinally valid, with good test-retest

reliability. The determination of MCID is highly beneficial for interpreting CARAT

scores. Therefore, the CARAT is a valid and reliable tool for use in clinical research

and practice to monitor asthma and allergic rhinitis symptoms simultaneously.

2

Table of contents

Background………………………………. ..3

Methods…………………………………... ..5

Results……………………………………. 10

Discussion……………………………….. 16

Conclusion………...……………………... 18

References……..………………………… 19

Appendix……….………………………… 21

3

Background

Asthma and allergic rhinitis are very common diseases that represent a global health

problem. Epidemiologic studies consistently show that asthma and allergic rhinitis

frequently co-exist (1). This has led to the concept that asthma and allergic rhinitis

are strongly related disorders with symptoms occurring in the upper airways (allergic

rhinitis) or lower airways (asthma). A number of studies have shown that 70-90% of

patients with asthma also have allergic rhinitis and 40-50% of patients with allergic

rhinitis also have asthma. Furthermore, there is a probable association between the

severity of asthma and allergic rhinitis (2-5).

The quality of asthma and allergic rhinitis control is of great influence on patients’

quality of life (6). Asthma and allergic rhinitis symptoms have shown to have a

negative influence on social life, school performance, and work productivity (1).

Patients who suffer from severe forms of disease are notably affected (7). Also,

allergic rhinitis symptoms have a subversive effect on academic performance. The

expenses caused by subjects with allergic rhinitis are substantial (8).

Most patients with asthma and/or allergic rhinitis are treated by general practitioners,

who are confronted with a variety of treatment options. Given the variation in patients,

symptoms and clinical presentation, primary care clinicians are faced with uncertainty

about the considerations regarding asthma and allergic rhinitis management (9,10).

Although 70-90% of patients with asthma also suffer from allergic rhinitis, symptoms

of asthma are generally only measured by questionnaires specific for asthma and not

for allergic rhinitis (11-14). General practitioners are therefore often not aware of

allergic rhinitis symptoms in their patients although their allergic rhinitis symptoms

might have a large impact on asthma control. This results in a large number of

patients who do not receive appropriate care despite the availability of effective

treatment options.

Clinical practice guidelines for the management of allergic rhinitis have been

developed over the past 15 years and have improved the care of this group of

patients (9). The first in the field of these evidence-based guidelines were the Allergic

Rhinitis and its Impact on Asthma (ARIA) guidelines. These guidelines were

developed in collaboration with specialists in allergy, primary care physicians, and

patient representatives from the European Federation of Allergy and Airways

Diseases Patients Associations (EFA)(15).

The ARIA guidelines recommend the optimal control of both asthma and allergic

rhinitis airway disease as the primary goal of their treatment (1,16). A combined

approach to the management of upper and lower airway disease has been

extensively proposed (1,15-17). To assess the effects on control of treatment,

validated questionnaires have been proposed as key instruments (18-20).

4

The Control of Allergic Rhinitis and Asthma Test (CARAT) questionnaire was created

to evaluate the effect of treatment for both asthma and allergic rhinitis (21). It is a

self-administered questionnaire that consists of 10 items monitoring upper and lower

airway symptoms and the use of medication. The CARAT questionnaire has been

validated to assess disease control of both asthma and allergic rhinitis in the

Portuguese population. Also, statistically redundant questions were removed,

shortening the CARAT questionnaire from 17 to 10 items, thus reducing the

completion time (6,21). The CARAT questionnaire translation is currently in different

steps in several languages including English, French, Greek, Spanish, Italian,

Turkish, Spanish, Finnish, Swedish, Norwegian, and Danish. Translation and cultural

adaptation of the CARAT questionnaire in the Netherlands was finished in November

2011. The Dutch CARAT study is part of the research focus “COPD, asthma and

allergy in general practice”, at the UMCG department of general practice. There is no

experience with the CARAT questionnaire in the Netherlands.

The objective of the current study is to evaluate the internal consistency and cross-

sectional validity for the Dutch version of the CARAT, as is done for the Portuguese

version of the CARAT. In addition, longitudinal validity, test-retest reliability and

Minimal Clinically important difference of the translated CARAT are investigated.

Another goal is to evaluate the sensitivity of the CARAT for detecting the influence of

pollen counts on asthma and allergic rhinitis symptoms.

Specific activities during internship

During my internship concerning the CARAT study I was responsible for distribution

of the questionnaire packages and the data entry. Logistic monitoring of the

questionnaires (using an Excel database) and the distribution of reminders were

further activities. Other responsibilities concerned the selection and inclusion of

patients, gathering patient characteristics from the centers in question, and creating

an SPSS database for the CARAT study. I was also involved in study design,

presentations concerning the CARAT study, performing statistical analyses and

writing a scientific paper.

5

Methods

Study design

The (CARAT study) is a longitudinal study consisting of three measurement points

with one month intervals (T1, T2, T3). The study was conducted between February

2012 and December 2012.

Participants

Patients were approached from the outpatient clinics from the department of allergy

in the UMCG (Dr. J.N.G Oude Elberink), the department of pulmonary diseases of

the Radboud University (Prof. R. Dekhuizen), and the allergy department of the

Erasmus University (Prof. R. Gerth van Wijk). Patients from primary care practices of

Dr. F.R. van Gemert in Harlingen, Dr. J.W.H. Kocks in Groningen and Dr. R.A.

Riemersma in Appingedam were also approached. From these six centers a total of

176 patients with a physician diagnosis of asthma and/or allergic rhinitis were asked

to participate. Patients from 18 to 70 years of age were included. Patients with

insufficient command of the Dutch language and patients diagnosed with dementia

were excluded from the study. Informed consent was obtained from all patients in

writing before the acquisition of baseline characteristics. House addresses of all

participants were acquired from each center for logistic purposes.

Sample size could not be calculated by power analysis, since there is no expected

difference in the process of validation. There are no general criteria for the required

sample size in a validation study of a translated questionnaire. A sample size of at

least 50-100 participants is generally recommended. After one round of

questionnaires and reminders the number of participants to be included in the study

was 92.

Baseline characteristics were obtained from the patient files in each center. A special

form was created for this purpose, including the following characteristics: age, sex,

age of onset asthma and/or allergic rhinitis, history of smoking, allergens to which

patient is allergic (clinical reactivity), specific IgE and / or skin prick test including tree

pollen, grass pollen, and other relevant allergens, lung function (FEV1%pred,

reversibility) and respiratory co-morbidity.

Materials

The CARAT questionnaire was developed in Portugal to measure the quality of

control of both asthma and allergic rhinitis (21). It was validated for the Portuguese

population in 2010. Cronbach’s alpha was 0.86 for the Portuguese CARAT. The

6

CARAT scores showed a good correlation with valid questionnaires for asthma and

allergic rhinitis symptoms. (6).Translation to Dutch was finished in February 2012.

The Dutch CARAT questionnaire* consists of 10 multiple choice questions which

refer to the four weeks prior to the date of completion. Seven questions relate to the

frequency of airway symptoms, four of which are upper airway symptoms and three

are lower airway symptoms. The other three questions concern sleep impairment,

activities limitation, and need for higher doses of medication respectively.

The first 9 questions are scored on a four point Likert scale ranging from 0 points

(almost every day) to 3 points (never), a higher score reflecting better asthma and

allergic rhinitis control. The last question concerning additional medication uses the

same scale, but it is scored in a different way: 3 points for the answers “I take no

medication” and “never”, 2 points for “less than seven days” and 0 points for “more

than seven days”. The total score is calculated by summing up the scores off all ten

questions, resulting in a range of 0-30 points. The total CARAT score and the two

separate factors of the questionnaire were distinguished in statistic analyses. Factor

1 consists of CARAT questions 1-4 and represents the questions about allergic

rhinitis symptoms. Factor 2 consists of questions 5-10 and represents the questions

about asthma symptoms (6).

The asthma control questionnaire (ACQ)* consists of six questions which are scored

on a 7 point Likert scale. These questions are based on the participants’ situation of

the last seven days. The total ACQ score is the mean score of all questions (ranging

from 0 to 6), a lower score representing better asthma control. The ACQ has been

shown to be reliable (ICC = 0.90, p<0.0001) and have strong evaluative properties

for the measurement of asthma control (18). It has also shown to have good

discriminative qualities which enable the ACQ to distinguish between patients who

have well controlled and not well controlled asthma. (22)

Three visual analog scales (VAS)* were used to assess general airway symptoms

and symptoms specific for asthma and allergic rhinitis. VAS question 1 covered all

airway symptoms, question 2 concerned lower airway symptoms and question 3

concerned higher airway symptoms. Participants were asked to mark the location on

a 10 centimeter line corresponding to the amount of symptoms they experienced in

the preceding week. Visual analog scales have shown to be highly responsive to

change during treatment when used for a global evaluation of allergic rhinitis

symptoms (23).

For the second and third measurement (T2 and T3) a global rating of change (GRC)*

question with a 15 point scale was used to monitor the participants’ subjective

experience of symptoms of asthma and allergic rhinitis, compared to the previous

measurement. The score range of this question was from -7 (no symptoms) to 7

(extreme symptoms).

* Questionnaire package T2 included in appendix

7

In addition, daily pollen-specific counts were provided by the

LUMC, measured by a pollen sampler on the roof of the 6th floor.

The filter is replaced weekly and analyzed in a laboratory, to

determine the pollen counts per m3 for every 24 hours. These daily

counts discriminate 41 different pollen species. Eight of these are

classified as allergenic, in three different categories: mildly

allergenic (four species), very allergenic (two species) and

extremely allergenic (two species). Only the eight allergenic pollen

species were used for statistical analysis.

Procedure

The CARAT study was approved by the Medical Ethical Commission (METc) in

February 2012, who deemed that permission from the commission was not required.

Questionnaire packages were distributed by mail to 176 patients with an

accompanying information letter from both clinician and researcher and an informed

consent form. Participants were requested to return the completed questionnaires

and informed consent form in a post-paid envelope. The first questionnaire

packages* were sent to the patients in May 2012.

All participants were asked to fill out a questionnaire package three times with a time

interval of a month (T1, T2, T3). The questionnaire packages included: the CARAT,

the ACQ5, the VAS on pulmonary symptoms, and questions on present medication

use and demographics. The GRC question was added to the package for T2 and T3.

Participants were requested to state the date of completion of the first questionnaire.

This date was used for administrative purposes to time the one month follow-up

questionnaires accordingly. In this way the date could be determined for each

participant individually. The same method was used for the third questionnaire.

Reminders were sent once if a participant had not returned the questionnaire within

one week after the aimed date. The number of reminders was counted to use for

descriptive purposes.

The acquisition of completed questionnaires covered a period of five months, and

lasted from May through September 2012, which covers the allergic rhinitis season.

Pollen counts were entered in a matrix in Microsoft Excel to make them available for

statistical analysis. Entering and analyzing data from the six participating centers

were centrally coordinated by the student at the department of General Practice at

the UMCG.

* Questionnaire package T2 included in appendix

8

Statistical Analyses

Statistical analyses were performed using SPSS 19.

All variables were tested for normal distribution using QQ-plots.

Internal consistency reliability of the CARAT was evaluated by calculating Cronbach’s

alpha. A Cronbach’s alpha of at least 0.70 was expected to consider the CARAT

suitable for the comparison of groups of patients. Further priori expectations were

based on the Portuguese version of the CARAT which has a Cronbach’s alpha of

0.86.

The cross-sectional validity of the CARAT was evaluated by calculating the

correlation between CARAT scores and ACQ5 as well as VAS scores. For this, we

used total CARAT scores and the scores of the separate Factors 1 (for allergic

rhinitis) and 2 (for asthma). Factor 1 of the CARAT and VAS3 represent higher

airway symptoms whereas Factor 2 of the CARAT and VAS2 represent lower airway

symptoms. VAS1 represents airway symptoms in general. VAS2+3 is obtained by

summing VAS2 and VAS3 and using it as a combined score for general airway

symptoms. A priori expectations were based on the Portuguese version of the

CARAT which showed correlation coefficients ranging from 0.6-0.8 with the ACQ and

VAS scores. Factor 1 was expected to correlate best with the VAS score concerning

higher airway symptoms, Factor 2 was expected to correlate best with the VAS score

concerning lower airway symptoms.

Longitudinal validity was evaluated by calculating the correlation between delta

scores of the CARAT and delta scores of the ACQ and VAS scores. These

correlation coefficients were calculated for intervals T1-T2 and T2-T3, subtracting the

former score from the latter. Separate factors of the CARAT were also compared to

ACQ and VAS scores.

The minimal clinically important difference was calculated using the absolute GRC

scores |GRC| measured on T2, this measurement representing the difference in

symptoms between T1 and T2. Patients were divided into four categories based on

|GRC|: no difference (0-1), minimal difference (2-3), moderate difference (4-5) and

large difference (6-7). For each of these categories the mean difference in CARAT

score between T1 and T2 was calculated. The mean difference for the ‘minimal

difference’-group was listed as MCID (24).

Test retest reliability was evaluated using the GRC scores measured on T2 and T3,

using only the patients that had scored between -1 and 1 (no difference).

Subsequently, the correlation between the CARAT scores in consecutive

measurements was calculated for T1-T2 and T2-T3 to determine test retest reliability

for both intervals. For this, Spearman correlation coefficients were used.

The discriminative properties (ability to distinguish patients at a single time point) of

the CARAT were investigated by using ACQ scores. The sample on T1 was

separated into two groups: one group containing patients with ACQ score < 1.5 (well

9

controlled), the other containing patients with ACQ score > 1.5 (not well controlled).

These scores were used because 1.5 is the cut-off point for the ACQ to distinguish

well controlled patients from patients that are not well controlled (22). The mean

CARAT score for T1 was calculated for both groups and compared using an

independent samples t-test. The a priori expectation was to find a significantly lower

mean CARAT score for the ‘ACQ not well controlled’ group, when compared to the

‘ACQ well controlled’ group.

Finally, the correlation between pollen counts during the course of the study and the

CARAT scores were investigated. The mean pollen counts in 4 weeks prior to the

date of completing the questionnaire package were calculated for each patient, for

each measuring moment (T1, T2, T3). This resulted in one number for each date,

representing the mean pollen counts for the 28 days prior to that particular date. In

this way, a mean pollen count could be used for every individual patient, depending

on the date of completing the questionnaires. To evaluate the sensitivity of the

CARAT questionnaire to changes in pollen counts, the correlation between the

differences in mean pollen counts (individually for each patient) and the change in

their total CARAT scores was calculated. This was executed for different samples of

patients depending on their diagnoses and clinical reactivity. The expectation was to

find a consistent longitudinal correlation between CARAT scores and pollen counts

during the period represented by the questionnaires.

10

Results

Of the 176 patients that where approached for the CARAT study 92 patients completed T1. Of these 92 patients, 82 patients completed T2 and of these 82 patients, 72 patients completed T3 (Table 1). A number of these patients filled out the questionnaires after receiving a reminder and these numbers are also shown in Table 1. The patient characteristics of this sample are described in Table 2. The data concerning IgE, skin prick tests and lung function were not used in statistical analysis due to inconsistent data from different centers.

Table 1. Response rates

Table 2. Patient characteristics

Table 2. Respiratory co-morbidity in patients consisted exclusively of COPD. Clinical reactivity categories were based on all available data from patient files. All patients allergic to pollen were put in the ‘Pollen’ category, irrespective of other allergies. The category ‘Other’ consisted of patients that were clinically reactive to any other allergens than pollen. Data for skin prick tests and specific IgE could not be described due to different outcome measures in each center. The percentages represent the number of patients who underwent the investigation in question.

Measurements Completed/approached Reminders sent/ completed

T1, n (%) 92/176 (53%) 17/92 (18%)

T2, n (%) 82/92 (89%) 25/82 (30%)

T3, n (%) 72/82 (88%) 38/72 (53%)

Characteristics Primary care Secondary care

Patients , n 46 46

Age in years, mean (SD) 46.6 (127) 41.3 (142)

Sex - Male (%) - Female (%)

41.3 58.7

23.4 76.6

Age of onset in years, mean (SD) - Asthma - Rhinitis

37.0 (15.1) 32.7 (11.1)

19.8 (14.5) 19.4 (10.5)

Respiratory co-morbidity, n (%) 8.7 23.4

Clinical reactivity - None (%) - Pollen(%) - Other(%) - Unknown (%)

26.1 39.1 17.4 17.4

2.1

85.1 10.6 2.1

Skin prick test, n (%) 1 (2.2%) 30 (65.2%)

Specific IgE, n (%) 2 (4.4%) 60 (32.6%)

Lung function, n (%) 41 (89.1%) 33 (71.7%)

11

The mean score of the CARAT among all patients for T1 was 19,4 with a standard deviation of 6,8. There was a minimum score of 3 (n = 1) and a maximum score of 30 (n = 4). There were differences in mean CARAT scores among gender and age categories, listed in Table 3. Table 3. Mean CARAT scores

CARAT scores Mean (SD) n

CARAT T1 19.36 (6.80) (n=92)

CARAT T1 ♀ CARAT T1 ♂

18.63 (6.59) 20.93 (7.11)

(n=63) (n=29)

CARAT T1 age percentiles

<33

34-43

44-53

>54

17.87 (7.07) 17.38 (7.64) 19.38 (7.04) 22.24 (4.88)

(n=23) (n=23) (n=23) (n=23)

CARAT T2 19.99 (7.13) (n=82)

CARAT T3 21.67 (6.51) (n=72)

CARAT T1 not well controlled (ACQ) CARAT T1 well controlled (ACQ)

12.77 (5.32) 21.43 (5.85)

(n=22) (n=70)

Internal consistency

In internal consistency analysis, Cronbach’s alpha for the total CARAT questionnaire was 0.82 on T1, with an alpha of 0.81 for Factor 1 (allergic rhinitis) and 0.77 for Factor 2 (asthma). Cronbach’s alpha for the total CARAT questionnaire was 0.86 on T2 and 0,83 on T3. Construct validity

Spearman correlation coefficients between CARAT scores and its separate factors and ACQ and VAS scores are listed in Table 4. All correlations proved to be statistically significant (p < 0.01). The most relevant correlations according to symptomatology categories are shown bold. These four correlations and the correlation of the total CARAT score with the ACQ score are visualized in scatter plots 1-4. Note that correlation coefficients are remarkably lower when comparing factors and questions that cover different symptoms (lower versus higher airway symptoms). Table 4. Spearman correlations construct validity

T1 ACQ VAS1 VAS2 VAS3 VAS2+3 CARAT total score -0.66 -0.69 -0.62 -0.64 -0.76

CARAT factor 1 (allergic rhinitis) -0.41 -0.47 -0.41 -0.70 -0.66

CARAT factor 2 (asthma) -0.70 -0.68 -0.64 -0.44 -0.71

Table 4. All correlations were statistically significant (p < 0.01).

12

Scatter plots 1-4. Correlations construct validity

Longitudinal validity

Spearman correlation coefficients between delta scores for the CARAT and delta scores for the ACQ and VAS scores are listed in Table 5. Delta 1 represents the four week period between T1 and T2, delta 2 represents the four week period between T2 and T3. All correlations proved to be statistically significant (p < 0.01). Longitudinal correlations for total CARAT scores are visualized in scatter plots 5-8.

13

Table 5. Spearman correlations longitudinal validity

Delta 1 (T1-T2) Delta 2 (T2-T3)

CARAT total scores

CARAT - ACQ 0.45 0.40

CARAT - VAS1 0.61 0.45

CARAT - VAS2+3 0.67 0.40

CARAT factors

Factor 1 (allergic rhinitis) - VAS3 0.55 0.36

Factor 2 (asthma) - VAS2 0.45 0.29

Factor 2 (asthma)- ACQ 0.41 0.41

Table 5. All correlations proved to be statistically significant (p < 0.01).

Scatter plots 5-8. Correlations longitudinal validity

14

Minimal clinically important difference

The minimal clinically important difference (MCID) was calculated using the GRC scores that were measured on T2, thus representing the four week period between T1 and T2. The different categories are listed in Table 6. The MCID for total CARAT scores, derived from the ‘minimal difference’ group, is 3.50 shown bold in the table. The maximum CARAT score being 30 points, the MCID is 11.6% of the total CARAT score.

Table 6. MCID

Difference category Global rating of change | GRC |

Difference CARAT score Mean (SD)

None (n=44) 0-1 3.00 (2.79)

Minimal (n=16) 2-3 3.50 (2.78)

Moderate (n=15) 4-5 7.07 (4.08)

Large (n=7) 6-7 5.57 (6.40)

Test-retest reliability Test-retest reliability was calculated for a sample of patients that were in the ‘No difference’ category based on their absolute GRC scores for T1-T2 and T2-T3. Comparing the CARAT score for these two groups there was a Spearman’s correlation coefficient of 0.74 for T1-T2 (n=44) and 0.78 for T2-T3 (n=31). Both correlations were statistically significant (p < 0.01). Discriminative properties On T1, the group with ‘well controlled’ disease (n=70) according to the ACQ had a mean CARAT score of 21.43 (SD 5.85), whereas the ‘not well controlled’ group according to the ACQ (n=22) had a mean CARAT score of 12.77 (SD 5.32). The difference in means between both groups was significant (p < 0.001). Group means are listed in Table 3.

15

Pollen counts Delta CARAT scores for the four week period prior to T1 and T2 showed correlation coefficients ranging from 0.14 to 0.32, when compared to delta pollen counts in the same period. For T2-T3, delta CARAT scores and delta pollen counts showed correlation coefficients ranging from 0.08 to 0.17. Correlation coefficients between delta CARAT scores and delta pollen counts are listed in Table 7. Different samples were used according to diagnosis and clinical reactivity. Delta CARAT scores and delta pollen counts for patients with clinical reactivity to at least one pollen species were normally distributed and therefore analyzed using Pearson correlation coefficients. All other results are Spearman correlation coefficients.

Table 7. Correlation of delta CARAT with delta pollen counts

Delta pollen counts T1-T2 Delta pollen counts T2-T3

Delta CARAT score 0.20 (p < 0.01) 0.13 (p < 0.01)

Delta CARAT Factor 1(allergic rhinitis) 0.14 (p < 0.01) 0.08 (p < 0.01)

Diagnose allergic rhinitis (n= 49) Delta CARAT score

0.23 (p < 0.01)

0.17 (p < 0.01)

Clinical reactivity to pollen (n= 41) Delta CARAT score

0.32 (p < 0.05)

0.10 (p < 0.05)

Table 7. The ‘clinical reactivity to pollen’ group was compared to delta pollen counts using

Pearson correlation coefficients. Other listed results are Spearman correlation coefficients.

16

Discussion

The CARAT is the first questionnaire to monitor the control of allergic rhinitis and

asthma simultaneously. This article describes the validation and evaluation of the

Dutch version of the CARAT. This study found that the Dutch CARAT is a valid tool

with good consistency and discriminative properties. Accordingly, the CARAT is

useful in the Netherlands to facilitate optimal control of both asthma and allergic

rhinitis simultaneously, which has been extensively proposed (1,16).

The Dutch CARAT is shown to have high internal consistency. The Cronbach’s alpha

for all of the three measurements is comparable to the alpha that was found in the

Portuguese CARAT study (6). Also, both Factor 1 (allergic rhinitis) and Factor 2

(asthma) individually show high internal consistency, suggesting both asthma and

allergic rhinitis questions are comparably consistent.

Due to the absence of a gold standard, the ACQ and VAS scores concerning lower

and higher airway symptoms were used to evaluate construct validity of the CARAT.

Correlations between both total scores and scores for different domains of the

questionnaires were calculated. All a priori expectations were met. The total CARAT

score has good construct validity when compared to the total score of the ACQ and

the VAS questions that cover both lower and higher airway symptoms. Comparing

only Factor 2 of the CARAT to the ACQ shows better correlation than the total

CARAT score. This result was expected, given the fact ACQ covers asthma

symptoms exclusively, as does CARAT Factor 2. The same effect was seen in the

correlation between CARAT Factor 2 and VAS question 2, which is exclusively about

higher airway symptoms. CARAT Factor 1 showed especially good correlation with

VAS question 3, which covers higher airway symptoms. Comparing lower and higher

airway domains vice versa showed lower correlation coefficients throughout. Results

were similar to findings for the Portuguese version of the questionnaire (6),

underlining good construct validity of the CARAT.

The CARAT study is shown to have satisfactory longitudinal validity. Correlation

coefficients between Delta scores of the CARAT and Delta scores of the ACQ and

VAS questions were reasonably high. Comparing total scores showed higher

correlation coefficients than the individual sub domains of the questionnaires. Also,

Delta 1 (T1-T2) showed better correlation throughout when compared to Delta 2 (T2-

T3). This may be explained by the fact that the amount of reminders that had to be

sent to participants was substantially bigger for T3 compared to T2 and T1, which

was probably due to the summer holiday (this is shown in Table 1). This resulted in

more individual variability in the period between T2 and T3 (Delta 2) Another reason

for Delta 2 correlating less convincingly than Delta 1 might be the smaller number of

participants in T3, resulting in less power. Overall longitudinal validity was as

expected. Correlation coefficients that were calculated between all Delta scores are

17

satisfactorily when evaluating quality of life scales, because scores can be easily

impacted by patient and external influences.

Minimal clinically important difference for the CARAT was evaluated. The ‘minimal

difference’ group shows a mean CARAT score difference of 3.50. This number

represents the difference in CARAT score for one patient over time that the patient

perceives as relevant and is therefore a clinically relevant finding. The minimal

clinically important difference will aid clinicians in interpreting the importance of

change in asthma and allergic rhinitis patients. The unexpected mean difference in

CARAT score for the ‘large difference’ group is assumed to be due to the small

sample size.

Test-retest reliability for the Dutch CARAT is shown to be high. CARAT scores on

different measuring moments were compared, only using the ‘no difference’ group

(based on GRC scores) in analysis. For these participants with no change or hardly

any change in symptoms, CARAT scores correlate very well individually.

The Dutch CARAT is shown to be able to distinguish patients based on ACQ scores.

Patients in the lower scoring category showed a mean CARAT score that is

significantly lower that the mean CARAT scores of the patients in the higher scoring

category. Therefore, discriminative properties of the Dutch CARAT are good when it

comes to distinguishing “well controlled” patients from “not well controlled” patients

based on ACQ scores (22).

The sensitivity of the CARAT questionnaire for detecting environmental influences on

asthma and allergic rhinitis was evaluated. Correlations between pollen counts during

the course of the study and CARAT scores did not show the expected consistency.

Both periods (T1-T2, T2-T3) show differing correlations, when looking at total CARAT

scores and scores of Factor 1, the allergic rhinitis subdomain. Correlation is higher

when using a sample of patients who are diagnosed with allergic rhinitis. Ultimately a

correlation coefficient of 0.32 was found for the first four week period by using a

sample of patients who were clinically allergic to at least one species of pollen. The

correlation coefficient for the second period was markedly lower. This is expected to

be due to the same factors that were discussed for longitudinal validity. The

correlation coefficient for the first period is not persuasively high, though it certainly

suggests a relation between pollen counts and changes in CARAT score. High

correlations cannot be expected given the fact that pollen counts were measured in

Leiden while participants were located elsewhere. Also, all allergenic pollen species

were treated equally in statistical analysis, not accounting the different degrees of

allergenicity. For more concrete results, further analysis of the measured pollen

counts is required to further explore relations between pollen counts and changes in

CARAT scores. An important step for this would be to weigh pollen counts differently,

in order to give more allergenic pollen species more impact in all calculations. Also,

mean pollen counts for a shorter period (several days) before each measurement

could be used for analyses, based on the hypothesis that pollen counts and therefore

18

symptoms of the last few days have more impact on the way participants fill out the

questionnaire. It could also be useful to let participants fill out questionnaires on the

same date, so intervals between measurements are actual periods rather than virtual

periods. This could result in larger and more consistent differences in pollen counts if

the first measurement is made before, and the second during the peak of allergic

rhinitis season.

Patient characteristics show a female predominance in the sample, with lower mean

CARAT scores than male participants. These results underline findings in the

Portuguese CARAT study. Results of skin prick tests, specific IgE and lung function

were not used for statistical analysis. This was due to the low rates of available data

concerning additional clinical research (especially in primary care patients) and the

different ways these results were presented in each center. Also, medication use was

not included in statistical analyses because of inconsistent patient data.

The sample size for each measurement (T1, T2, T3) is sufficient, given the fact that a

number of 50-100 patients is recommended for validation studies. For each

measuring moment the number of patients reduces with 10, resulting in slightly

reducing power in the course of the study. For this reason, the main focus in the

interpretation of the results lies with T1 and Delta 1 (T1-T2). It is remarkable how the

percentage of sent reminders increases during the course of the study. This is

expected to be due to the summer holiday period surrounding T3.

A limitation of the study may be the incomplete descriptive baseline characteristics

concerning lung function, specific IgE and skin prick tests, especially for primary care

patients. However, these are not standard clinical investigations in primary care. This

particular data would have been included if it were available. This particular data

could be included in further research if these clinical investigations were performed

during the course of the study

Conclusion

The Dutch version of the CARAT has high internal consistency and good construct

validity and is therefore comparable to findings according to the Portuguese CARAT.

Additionally, the Dutch CARAT proves to be longitudinally valid, with good test-retest

reliability. The determination of MCID is highly beneficial for interpreting CARAT

scores. Further research is required to find consistent correlations between pollen

counts and CARAT scores. However, the CARAT is a valid and reliable tool for use in

clinical research and practice to monitor asthma and allergic rhinitis symptoms

simultaneously.

19

References

(1) Bousquet J, Khaltaev N, Cruz AA, Denburg J, Fokkens W, Togias A, et al. ARIA update 2008: allergic rhinitis and its effect on asthma. Allergologie 2009 AUG;32(8):306-319.

(2) Bresciani M, Paradis L, Des Roches A, Vernhet H, Vachier I, Godard P, et al. Rhinosinusitis in severe asthma. J Allergy Clin Immunol 2001 JAN;107(1):73-80.

(3) Sole D, Camelo-Nunes I, Wandalsen G, Melo K, Naspitz C. Is rhinitis alone or associated with atopic eczema a risk factor for severe asthma in children? Pediatr Allergy Immunol 2005 MAR;16(2):121-125.

(4) Bousquet J, Gaugris S, Kocevar V, Zhang Q, Yin D, Polos P, et al. Increased risk of asthma attacks and emergency visits among asthma patients with allergic rhinitis: a subgroup analysis of the improving asthma control trial. Clin Exp Allergy 2005 JUN;35(6):723-727.

(5) Terreehorst I, Oosting A, Tempels-Pavlica Z, de Monchy J, Bruijnzeel-Koomen C, Hak E, et al. Prevalence and severity of allergic rhinitis in house dust mite-allergic patients with bronchial asthma or atopic dermatitis. Clin Exp Allergy 2002 AUG;32(8):1160-1165.

(6) Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M, et al. Validation of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy 2010 AUG;65(8):1042-1048.

(7) Bousquet J, Neukirch F, Bousquet P, Gehano P, Klossek J, Le Gal M, et al. Severity and impairment of allergic rhinitis in patients consulting in primary care. J Allergy Clin Immunol 2006 JAN;117(1):158-162.

(8) Walker S, Khan-Wasti S, Fletcher M, Cullinan P, Harris J, Sheikh A. Seasonal allergic rhinitis is associated with a detrimental effect on examination performance in United Kingdom teenagers: Case-control study. J Allergy Clin Immunol 2007 AUG;120(2):381-387.

(9) Bousquet J, Bodez T, Gehano P, Klossek JM, Liard F, Neukirch F, et al. Implementation of Guidelines for Allergic Rhinitis in Specialist Practices Randomized Pragmatic Controlled Trial. Int Arch Allergy Immunol 2009;150(1):75-82.

(10) Church MK, Maurer M, Simons FER, Bindslev-Jensen C, van Cauwenberge P, Bousquet J, et al. Risk of first-generation H-1-antihistamines: a GA2LEN position paper. Allergy 2010 APR;65(4):459-466.

(11) Baiardini I, Pasquali M, Giardini A, Specchia C, Passalacqua G, Venturi S, et al. Rhinasthma: a new specific QoL questionnaire for patients with rhinitis and asthma. Allergy 2003 APR;58(4):289-294.

20

(12) Kilpelainen M, Terho E, Helenius H, Koskenvuo M. Validation of a new questionnaire on asthma, allergic rhinitis, and conjunctivitis in young adults. Allergy 2001 MAY;56(5):377-384.

(13) Fischer PE, Grabbe Y, Nolting H-. Development and validation of a screening questionnaire for allergy airway diseases (ASF Screening Questionnaire). Allergologie 2006 OCT;29(10):393-402.

(14) Wasserfallen J, Gold K, Schulman K, Baraniuk J. Development and validation of a rhinoconjunctivitis and asthma symptom score for use as an outcome measure in clinical trials. J Allergy Clin Immunol 1997 JUL;100(1):16-22.

(15) Bousquet J, Bousquet P, Godard P, Daures J. The public health implications of asthma. Bull World Health Organ 2005 JUL;83(7):548-554.

(16) Braunstahl G, Fokkens W. Nasal involvement in allergic asthma. Allergy 2003 DEC;58(12):1235-1243.

(17) Bateman ED, Hurd SS, Barnes PJ, Bousquet J, Drazen JM, FitzGerald M, et al. Global strategy for asthma management and prevention: GINA executive summary. Eur Resp J 2008 JAN;31(1):143-178.

(18) Juniper E, O'Byrne P, Guyatt G, Ferrie P, King D. Development and validation of a questionnaire to measure asthma control. Eur Resp J 1999 OCT;14(4):902-907.

(19) Nathan R, Sorkness C, Kosinski M, Schatz M, Li J, Marcus P, et al. Development of the asthma control test: A survey for assessing asthma control. J Allergy Clin Immunol 2004 JAN;113(1):59-65.

(20) Vollmer W, Markson L, O'Connor E, Sanocki L, Fitterman L, Berger M, et al. Association of asthma control with health care utilization and quality of life. Am J Respir Crit Care Med 1999 NOV;160(5):1647-1652.

(21) Nogueira-Silva L, Martins SV, Cruz-Correia R, Azevedo LF, Morais-Almeida M, Bugalho-Almeida A, et al. Control of allergic rhinitis and asthma test - a formal approach to the development of a measuring tool. Respir Res 2009 JUN 17;10:52.

(22) Juniper E, Bousquet J, Abetz L, Bateman E, GOAL Comm. Identifying 'well-controlled' and 'not well-controlled' asthma using the Asthma Control Questionnaire. Respir Med 2006 APR;100(4):616-621.

(23) Bousquet P, Combescure C, Klossek J, Daures J, Bousquet J. Change in visual analog scale score in a pragmatic randomized cluster trial of allergic rhinitis. J Allergy Clin Immunol 2009 JUN;123(6):1349-1354.

(24) Jaeschke R, Singer J, Guyatt G. Measurement of Health-Status - Ascertaining the Minimal Clinically Important Difference. Control Clin Trials 1989 DEC;10(4):407-415.

21

Appendix

22

Allergie en Astma Vragenlijst voor volwassenen

De ingevulde vragenlijst graag terugsturen in de

bijgeleverde envelop. Een postzegel is niet nodig!

23

1. Hoe vaak hebt u, vanwege uw allergische neusklachten en/of astma, de laatste 4 weken

gemiddeld last gehad van:

Nooit 1 of 2

dagen per

week

Meer dan 2 dagen

per week

(Bijna) iedere

dag

1. Verstopte neus?

2. Niezen?

3. Jeuk aan de neus?

4. Loopneus?

5. Benauwdheid / Kortademigheid?

6. Piepende ademhaling?

7. Beklemd gevoel op de borst bij lichamelijke inspanning?

8. Vermoeidheid en/of beperkingen bij het uitvoeren van dagelijkse bezigheden?

9. ’s-Nachts wakker worden?

Hoe vaak moest u de laatste 4 weken, vanwege allergische neusklachten en/of astma:

Ik gebruik geen

medicijnen

Nooit Minder dan 7

dagen

7 dagen of

meer

10. Uw medicijnen extra

gebruiken?

CARAT

24

2. Zet een cirkeltje om het getal dat het beste beschrijft hoe u zich de afgelopen week heeft

gevoeld (slechts één antwoord per vraag). LET OP! Bij deze vragen gaat het om de afgelopen week

(en niet de afgelopen 4 weken).

1. Hoe vaak bent u per nacht gemiddeld wakker

geworden door uw astma in de afgelopen

week?

0. Nooit 1. Bijna nooit 2. Een paar keer 3. Verscheidene keren 4. Vaak 5. Heel vaak 6. Kon niet slapen vanwege astma

2. Hoe ernstig waren uw astma klachten bij het 's

morgens wakker worden gemiddeld in de

afgelopen week?

0. Geen klachten 1. heel lichte klachten 2. lichten klachten 3. matige klachten 4. vrij ernstige klachten 5. ernstige klachten 6. heel ernstige klachten

3. In welke mate werd u over het algemeen in de

afgelopen week door uw astma beperkt bij uw

activiteiten?

0. helemaal niet 1. nauwelijks 2. een beetje 3. tamelijk 4. erg 5. heel erg 6. volledig

4. In welke mate heeft u zich over het algemeen

kortademig gevoeld in de afgelopen week ten

gevolge van uw astma?

0. helemaal niet 1. nauwelijks 2. een beetje 3. middelmatig 4. vrij ernstig 5. ernstig 6. heel ernstig

5. Hoe vaak had u in de afgelopen week over het

algemeen een piepende ademhaling?

0. nooit 1. zelden 2. af en toe 3. geregeld 4. vaak 5. meestal 6. altijd

6. Hoe veel pufs/inhalaties van een kortwerkend

luchtwegverwijdend middel (bijv. ventolin of

bricanyl) heeft u op de meeste dagen

genomen in de afgelopen week?

0. geen 1. 1-2 pufs/inhalaties 2. 3-4 pufs/inhalaties 3. 5-8 pufs/inhalaties 4. 9-12 pufs/inhalaties 5. 13-16 pufs/inhalaties 6. meer dan 16 pufs/inhalaties

ACQ

Juniper EF, O'Byrne PM, Guyatt GH, Ferrie PJ, King DR. Development and validation of a questionnaire to measure asthma

control. Eur Respir J. 1999;14(4):902–7.

Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M, Fernandes L, Bousquet J. Validation

of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy. 2010 Aug;65(8):1042-8.

25

3. Hieronder ziet u een lijn die loopt van ‘geen klachten’ tot ‘extreem veel klachten’. Wilt u per

vraag op de lijn een kruis zetten op de plek die het beste weergeeft hoeveel klachten u de

afgelopen 4 weken heeft gehad? Bijvoorbeeld;

Geen klachten I------------------------------------------X----------------------------------I Extreem veel

1. Hoeveel klachten van uw luchtwegen in het algemeen had u de afgelopen 4 weken?

Geen klachten I-------------------------------------------------------------------------------I Extreem veel

2. Hoeveel klachten van uw longen (astma) had u de afgelopen 4 weken?

Geen klachten I-------------------------------------------------------------------------------I Extreem veel

3. Hoeveel klachten van uw neus (hooikoorts) had u de afgelopen 4 weken?

Geen klachten I-------------------------------------------------------------------------------I Extreem veel

0 1 2 3 4 5 6 7 8 9 10 klachten

0 1 2 3 4 5 6 7 8 9 10 klachten

0 1 2 3 4 5 6 7 8 9 10 klachten

0 1 2 3 4 5 6 7 8 9 10 klachten

VAS

26

4. Is er enige verandering in uw allergische neusklachten en/of astma sinds de vorige keer dat u

deze vragenlijsten heeft ingevuld (ongeveer een maand geleden)? Zet een cirkeltje om het getal dat

het beste weergeeft hoeveel slechter of beter het nu gaat. Slechts één antwoord mogelijk.

De allergische neusklachten en/of astma is:

-7. extreem veel slechter

-6. erg veel slechter

-5. veel slechter

-4. redelijk veel slechter

-3. enigszins slechter

-2. een beetje slechter

-1. ongeveer hetzelfde, nauwelijks slechter

0. hetzelfde

1. ongeveer hetzelfde, nauwelijks beter

2. een beetje beter

3. enigszins beter

4. redelijk veel beter

5. veel beter

6. erg veel beter

7. extreem veel beter

GRC

27

5. Heeft u de afgelopen 4 weken medicijnen gebruikt voor uw neusklachten en/of astma?

Nee Ja Vul hieronder naam, soort en gebruik in

Naam Medicijn Toedieningsvorm * Sterkte Hoe vaak gebruikt u deze?

T Z P I

T Z P I

T Z P I

T Z P I

T Z P I

* Tablet = T, Zalf of crème = Z, Pufje = P, Injectie = I

Tenslotte volgen hier nog een aantal algemene vragen. Graag volledig invullen. Deze gegevens zijn

van belang voor het onderzoek en worden gebruikt om de vragenlijst te coderen.

6. Datum van invullen …. - …. - …….. (dd-mm-jjjj)

7. Geboorte datum …. - …. - …….. (dd-mm-jjjj)

8. Geslacht Man Vrouw

Algemeen

Dit is het einde van de vragenlijst.

Zou u de vragenlijst zo spoedig mogelijk willen terugsturen in de bijgevoegde enveloppe?

Een postzegel is niet nodig!

Hartelijk dank voor het invullen!