Technical Report Puffing Topography Inter-lab Study · 2017. 2. 21. · 3 ISO 5725-1:1994 Accuracy...

Smoking Behaviour Sub-Group

Technical Report

Puffing Topography Inter-lab Study

January 2017

Study Co-ordinators:

Krishna Prasad and Sandy Slayford

British American Tobacco, United Kingdom

Statistical Support:

Thomas Verron and Rémi Julien

SEITA – Imperial Tobacco, France

Table of Contents

1. INTRODUCTION ............................................................................................................... 3

2. EXPERIMENTAL DESIGN ............................................................................................... 3

2.1 Part 1: Intra and inter-lab-device variability ....................................................................... 4

2.2 Part 2: Time effect ................................................................................................................ 5

3. STATISTICAL EVALUATION ......................................................................................... 6

3.1. Uncertainty evaluation ......................................................................................................... 6

3.2. Accuracy profile ................................................................................................................... 8

3.3. Limitations ............................................................................................................................ 9

4. RESULTS .......................................................................................................................... 10

4.1. Outlier results ...................................................................................................................... 10

4.1.1. Puff duration ................................................................................................. 11

4.1.2. Puff volume .................................................................................................. 13

4.2. Intra and inter-lab-device variability estimations ............................................................. 15

4.2.1 Puff duration ................................................................................................. 15

4.2.2. Puff volume .................................................................................................. 16

4.3. Intra and inter-lab-device variability modeling ................................................................ 17

4.3.1. Puff duration ................................................................................................. 17

4.3.2. Puff volume .................................................................................................. 18

4.4. Desirability indices ............................................................................................................. 19

4.4.1. Puff duration ................................................................................................. 20

4.4.2. Puff volume .................................................................................................. 21

5. COMMENTS AND CONCLUSIONS .............................................................................. 22

5.1. General comments .............................................................................................................. 22

5.2. Puff duration ....................................................................................................................... 22

5.3. Puff volume......................................................................................................................... 22

APPENDIX A – List of participating laboratories and device manufacturers............................ 23

APPENDIX B – Part 2 – Intra and inter-lab-device variability estimations ............................... 24

APPENDIX C – Data representation .......................................................................................... 25

APPENDIX D – Accuracy profile results ................................................................................... 36

APPENDIX E – Experimental protocol ...................................................................................... 38

APPENDIX F – Departures from experimental protocol ........................................................... 44

APPENDIX G – Definitions ....................................................................................................... 45

TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 3/45

1. INTRODUCTION

In 2012, the CORESTA Sub-Group, Smoking Behaviour (SG-TSB), agreed to conduct a

study to determine if various human puffing topography devices deliver comparable results

for puff volume and puff duration. In order to assess this, the uncertainty of both puffing

topography devices and laboratories has been evaluated based on the experimental design

described below.

2. EXPERIMENTAL DESIGN

Four laboratories took part in this study and four puffing topography devices (see Figure 1)

were evaluated by four different SG-TSB member organizations. Appendix A lists the

laboratories and device manufacturers involved in the study. Note that the listings of

laboratories and device manufacturers are in alphabetical order, while the results are presented

by code (1-4 for laboratories; A-D for devices). Labs and devices were both randomized for

data presentation, so nothing can (or should) be inferred about the identity of any lab or

device based on the order of presentation in this report.

CME–P4

CReSSmicro

SA7

SPA-M

Figure 1: Presentation of the four evaluated devices

The study was designed to evaluate the intra-lab-device variability (r*) and the inter-lab-

device variability (R*) of the puffing topography devices (Part 1) and to evaluate the time

effect (Part 2).


2.1 Part 1: Intra and inter-lab-device variability

In this part, three different units of each type of device were tested. The same device set was

circulated to each of the four participating laboratories over the course of two years. Each

participating laboratory was asked to test the set of 12 puffing topography devices (four

brands and three units of each) on two samples of king size cigarettes: Camel Blue (CB) and

West White (WW). These corresponded to two different open pressure drops: 45 mmWG and

130.5 mmWG, respectively. Two different cigarette modes were considered as well: lit and

unlit. Whichever mode, a total of 10 puffs per sample was taken at each smoking regime (see

Table 1).

The data structure with the key figures is illustrated in Figure 2. A total of 9600 lines of data

was generated for each of the three parameters (puff volume, puff duration and flow rate).

Figure 2: Data structure of Part 1

Table 1 gives the details of the five smoking regimes considered in Part1. All puff profiles

were square with inter-puff intervals of 30 s.

Table 1: Set of five smoking regimes

Smoking Regime Puff Volume (ml) Puff Duration (s) Average Flow (ml/s)

1 35 3.5 10

2 80 4.0 20

3 90 3.0 30

4 80 2.0 40

5 50 1.0 50

Puffing Topography Study

Lab 1

A

B

C

D

Rep. 1

Rep. 2

….

Rep. 10

Terms

Lab 4 Lab 2 Participating laboratories (4)

Evaluated devices (4)

Puffing regimes (5)

Replicates (10)

Individual results (9600)

Lab 3

Evaluated units (3)

Tested configurations (4) CB Lit CB Unlit WW Lit WW Unlit Unlit

#1 #2 #3

SR1 SR2 SR3 SR4 SR5

Equipment

Protocol


The objective was to determine the maximum difference expected between given puff

volumes or durations measured: 1) using a given device in a given lab and 2) using different

devices in different labs.

2.2 Part 2: Time effect

In order to evaluate the time effect, each participating laboratory was asked to test only one

unit per device, with only one puffing regime (3) on five separate days (see Figure 3). A total

of 3200 lines of data was generated for each of the three parameters (puff volume, puff

duration and flow rate).

Figure 3: Data structure Part 2

Puffing Topography Study

Lab 1

A

B

C

D

SR3

Rep. 1

Rep. 2

….

Rep. 10

Terms

Lab 4 Lab 2 Participating laboratories (4)

Evaluated devices (4)

Puffing regime (1)

Replicates (10)

Individual results (3200)

Lab 3

Evaluated unit (1)

Tested configurations (4)

#1

Testing days (5) D1 D2 D3 D4 D5

CB Lit CB Unlit WW Lit WW Unlit Unlit

Equipment

Protocol


3. STATISTICAL EVALUATION

3.1. Uncertainty evaluation

The statistical evaluation of data for this collaborative study followed the methods provided

by ISO 5725-21. For outlier testing, the Grubbs and Cochran methods were used.

ISO 5725-2 Tests Consistency

Cochran’s test Within-laboratory variability: Suitable for detecting whether or not the highest value in a set of laboratory standard deviations is an outlier (test statistic greater than 1% critical value) or a straggler (test statistic greater than 5% and less than 1% critical value).

Grubbs’ test Between-laboratory variability: Suitable for detecting whether or not the highest (or lowest) laboratory’s average is an outlier (test statistic greater than 1% critical value) or a straggler (test statistic greater than 5% and less than 1% critical value).

Both statistical tests allowed the identification of stragglers and outliers, however only the

outlier values were discarded in the statistical evaluation.

The protocol “Harmonized Statistical Procedure” defined by IUPAC (International Union of

Pure and Applied Chemistry) was applied. This involved sequential applications of the

Cochran and Grubbs tests until no further outliers were detected or until the data dropped

amounted to 22.2% (for this study = 2/9) of the total data collected (see Figure 4).

The Cochran outlier test was applied first, and if an outlying laboratory was identified, then a

single-value Grubbs test was performed on the individual values of this outlying laboratory

(Individual Grubbs). If no individual value was identified as an outlier, the outlying

laboratory is removed. If an individual value was identified as an outlier, only that value was

removed. This was followed by the application of single-value Grubbs test and outlying

laboratories were removed. If no laboratory was identified as an outlier, the pair-value test

was applied (two values at the same end). Any laboratory(ies) flagged by these tests were

removed until the data dropped amounted to 22.2% (for this study = 2/9) of the total data

collected.

1 “ISO 5725-2:1994: Accuracy (trueness and precision) of measurement methods and results – Part 2: Basic

method for the determination of repeatability and reproducibility of a standard measurement method.


Figure 4: Process of outlier detection

At the end of the loop of outlier detection, if any laboratory(ies) had been removed, then the

process was stopped. On the remaining data set, a nested ANOVA was performed in order to

evaluate the intra-lab-device variability (r*) and the inter-lab-device variability (R*).


3.2. Accuracy profile

In addition for Part 1, the accuracy profile approach was used to test the ‘fitness for purpose’

of the devices when used by one laboratory. The accuracy profile is a graphical decision-

making tool (see Figure 5).

A Tolerance Interval is computed considering the puff measurements (10 replicates) and

intermediate precision (three units) and Acceptability Interval (λ) defined by the

collaborative study are determined for each device type. These intervals give information

about the expected proportion of future measurements that will lie at a certain level of risk (1

– β) in the specified range and whether or not the device results are acceptable.

Figure 5: Accuracy profile as a graphical decision-making tool

The ‘fitness-for-purpose’ of the device is represented by the Desirability Index (ID), which is

computed as the cube root of the product of the three indices: the Range Index (IR), the

Precision Index (IP) and the Trueness Index (IT). All indices vary from 0 to 1. For further

details on computation of the Desirability Index (ID), refer to E. Rozet et al2 .

2 E. Rozet, V. Wascotte, N. Lecouturier, V. Préat, W. Dewé, B. Boulanger, P. Hubert, Improvement of

the decision efficiency of the accuracy profile by means of a desirability function for analytical

methods validation: Application to a diacetyl-monoxime colorimetric assay used for the determination

of urea in transdermal iontophoretic extracts, Analytica Chimica Acta, Volume 591, Issue 2, 22 May

2007, Pages 239-247.


Finally, the Desirability Index gives the ‘fitness for purpose’ of the device within the limits of

inter-lab-device variability (estimated in the collaborative study) when it is used within a

given laboratory. Figure 6 shows some examples of the range of information provided by the

accuracy profiles at different levels of the Desirability Index.

Figure 6: Examples of desirability index

3.3. Limitations

Despite having a well-balanced design, there are some limitations to this collaborative study.

First of all, the small number of participating laboratories (only four) cannot be representative

of inter-laboratories variability, and can impact the interpretation of the statistical tests of

outlier detection. Although the four participating laboratories were experienced in the use of

puffing topography devices, it would have been better to have at least eight laboratories as

recommended by ISO 5725-13 and ASTM E2480

4.

Secondly, the small number of participating laboratories makes evaluation per device less

robust. Consequently, we had to combine the inter-laboratories variability with the inter-

3 ISO 5725-1:1994 Accuracy (trueness and precision) of measurement methods and results -- Part 1: General

principles and definitions.

4 ASTM E2480 - 12 Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a

Test Method with Multi-Valued Measurands.


devices variability, to calculate global lab-device performance evaluation. With such

evaluation, the inter-laboratories variability and inter-devices variability are confounded. For

this reason, we decided to designate the intra-lab-device variability and the inter-lab-device

variability as r* and R*, respectively, in order to avoid any confusion with the usual notations

r and R for repeatability and reproducibility respectively.

Thirdly, although global indices allow a comparison of devices, it is important to note that a

bad index is not necessarily the consequence of a bad device; it can also reflect a bad use of

the device, or a cigarette sampling issue.

4. RESULTS

This section relates only to the results obtained within Part 1 of the study. Appendix B

provides the intra-lab-device variability and the inter-lab-device variability estimations for

Part 2 for information purposes only. Indeed, the evaluations conducted within this study lead

to the conclusion that the estimations of the intra-lab-device variability and the inter-lab-

device variability are similar between both parts. This means that the major part of the study

variability has been captured in Part 1.

4.1. Outlier results

In order to accurately estimate the intra-lab-device and the inter-lab-device variabilities, the

outlier detection procedure, as described previously, was applied to 160 datasets

corresponding to five smoking regimes, two lighting modes and 16 combinations of

laboratories-devices (four laboratories and four devices). Each dataset is a set of 60 individual

results (three units, two samples and 10 puffs).


4.1.1. Puff duration

Table 2 provides the summary of outliers and stragglers detected throughout the study for

each combination for puff duration. Tables 3 and 4 then link this information to the related

smoking regime and lighting mode.

Table 2: Summary of outliers and stragglers for puff duration

Single test Double test Single test Double test

Dev.A_Lab.1 10 600 2 (20%)

Dev.A_Lab.2 10 600 6 (60%)

Dev.A_Lab.3 10 600 5 (50%)

Dev.A_Lab.4 10 600 6 (60%)

Dev.B_Lab.1 10 600 4 (40%)

Dev.B_Lab.2 10 600

Dev.B_Lab.3 10 600

Dev.B_Lab.4 10 600 1 (10%)

Dev.C_Lab.1 10 600 1 (10%)

Dev.C_Lab.2 10 600 2 (20%)

Dev.C_Lab.3 10 600

Dev.C_Lab.4 10 600

Dev.D_Lab.1 10 600 2 (20%)

Dev.D_Lab.2 10 600 4 (40%)

Dev.D_Lab.3 10 600

Dev.D_Lab.4 10 600 5 (50%)

Total 160 9600 1 (0.6%) 37 (23%)

Combination

Labs-Devices

Number of

datasets

analysed

STRAGGLER DATASET OUTLIER DATASET

GRUBBSCOCHRAN

GRUBBSCOCHRAN

Total number

of datapoints

analysed


Expressing the number of outliers from Table 2 as a percentage of the total number of 160

datasets, almost 1 out 4 (23%) datasets was detected as an outlier, as related to the within-

device laboratory variability (Cochran’s test). Details on how outliers are distributed across

the different smoking regimes and lighting modes are given in the two following tables.

Table 3: Distribution of stragglers for puff duration

Table 4: Distribution of outliers for puff duration

C: Cochran’s test, G: Grubbs’ simple test (_h:high and _l:low),

G2: Grubbs’ double test (_h:high and _l:low), Gi: Grubbs’ test on individual values.

In some cases, where the “Gi” label occurs in the table, it means that a few of the individual

results were removed because they were identified as outliers. As the rest of the dataset was

still considered in the estimation of the intra-lab-device variability and the inter-lab-device

variability, the “Gi” status is not mentioned in the summary table (Table 2).

LIT_SR1 LIT_SR2 LIT_SR3 LIT_SR4 LIT_SR5 UNLIT_SR1 UNLIT_SR2 UNLIT_SR3 UNLIT_SR4 UNLIT_SR5

Dev.A_Lab.1

Dev.A_Lab.2

Dev.A_Lab.3

Dev.A_Lab.4

Dev.B_Lab.1

Dev.B_Lab.2

Dev.B_Lab.3

Dev.B_Lab.4 C

Dev.C_Lab.1

Dev.C_Lab.2

Dev.C_Lab.3

Dev.C_Lab.4

Dev.D_Lab.1

Dev.D_Lab.2

Dev.D_Lab.3

Dev.D_Lab.4

Combination

Labs-Devices

PUFF DURATION - STRAGGLER


Dev.A_Lab.1 Gi Gi Gi C Gi C Gi Gi

Dev.A_Lab.2 C Gi Gi Gi C C C Gi C C

Dev.A_Lab.3 C C C Gi C C

Dev.A_Lab.4 C C Gi C C Gi C C

Dev.B_Lab.1 C C C C

Dev.B_Lab.2

Dev.B_Lab.3

Dev.B_Lab.4 Gi Gi Gi Gi Gi

Dev.C_Lab.1 C

Dev.C_Lab.2 C C

Dev.C_Lab.3

Dev.C_Lab.4

Dev.D_Lab.1 C C

Dev.D_Lab.2 C C Gi C C

Dev.D_Lab.3 Gi

Dev.D_Lab.4 C C C C C

Combination

Labs-Devices

PUFF DURATION - OUTLIER


4.1.2. Puff volume

Table 5 provides the summary of outliers and stragglers detected throughout the study for

each combination for the puff volume. Tables 6 and 7 then link this information to the related

smoking regime and lighting mode.

Table 5: Summary of outliers and stragglers for puff volume

Single test Double test Single test Double test

Dev.A_Lab.1 10 600

Dev.A_Lab.2 10 600

Dev.A_Lab.3 10 600

Dev.A_Lab.4 10 600 2 (20%)

Dev.B_Lab.1 10 600 10 (100%)

Dev.B_Lab.2 10 600 1 (10%) 3 (30%) 5 (50%)

Dev.B_Lab.3 10 600 5 (50%)

Dev.B_Lab.4 10 600 1 (10%) 3 (30%) 6 (60%)

Dev.C_Lab.1 10 600 2 (20%)

Dev.C_Lab.2 10 600

Dev.C_Lab.3 10 600

Dev.C_Lab.4 10 600

Dev.D_Lab.1 10 600 7 (70%)

Dev.D_Lab.2 10 600

Dev.D_Lab.3 10 600 5 (50%)

Dev.D_Lab.4 10 600 1 (10%)

Total 160 9600 41 (26%)

Combination

Labs-Devices

Number of

datasets

analysed

STRAGGLER DATASET OUTLIER DATASET

GRUBBSCOCHRAN

GRUBBSCOCHRAN

Number of

datapoints

analysed


Expressing the number of outliers from Table 5 as a percentage of the total number of 160

datasets, more than 1 out 4 (26%) datasets was detected as an outlier related to the within-

device laboratory variability (Cochran’s test). Details on how outliers are distributed across

the different smoking regimes and lighting modes are given in the two following tables.

Table 6: Distribution of stragglers for puff volume

Table 7: Distribution of outliers for puff volume

C: Cochran’s test, G: Grubbs’ simple test (_h:high and _l:low),

G2: Grubbs’ double test (_h:high and _l:low), Gi: Grubbs’ test on individual values.

In some cases, where the “Gi” label occurs in the table, it means that a few of the individual

results were removed because they were identified as outliers. As the rest of the dataset was

still considered in the estimation of the intra-lab-device variability and the inter-lab-device

variability, the “Gi” status is not mentioned in the summary table (Table 2).


Dev.A_Lab.1

Dev.A_Lab.2

Dev.A_Lab.3

Dev.A_Lab.4 G2_h G2_h

Dev.B_Lab.1

Dev.B_Lab.2 G2_h G2_h G_h, G2_h

Dev.B_Lab.3

Dev.B_Lab.4 G_h, G2_h G2_h G2_h

Dev.C_Lab.1

Dev.C_Lab.2

Dev.C_Lab.3

Dev.C_Lab.4

Dev.D_Lab.1

Dev.D_Lab.2

Dev.D_Lab.3

Dev.D_Lab.4

Combination

Labs-Devices

PUFF VOLUME - STRAGGLER


Dev.A_Lab.1

Dev.A_Lab.2

Dev.A_Lab.3

Dev.A_Lab.4

Dev.B_Lab.1 C C C C C C C C C C

Dev.B_Lab.2 C C C C C

Dev.B_Lab.3 C C C C C

Dev.B_Lab.4 C C C C C C

Dev.C_Lab.1 C C

Dev.C_Lab.2

Dev.C_Lab.3

Dev.C_Lab.4

Dev.D_Lab.1 C C C C C C C

Dev.D_Lab.2

Dev.D_Lab.3 C C C C C

Dev.D_Lab.4 C

Combination

Labs-Devices

PUFF VOLUME - OUTLIER


4.2. Intra and inter-lab-device variability estimations

The intra-lab-device variability limits (r*) and the inter-lab-device variability limits (R*) are

calculated for each smoking regime, for each mode (lit and unlit) and for each parameter (puff

duration and puff volume).

4.2.1 Puff duration

Estimations for puff duration are summarized in Table 8 by keeping the outlier dataset

(combination laboratory-device) and in Table 9 by removing the outlier dataset. The

corresponding graphs named “individual values” are shown in Appendix C of this report.

Table 8: r* and R* estimations for puff duration with outlier dataset

Table 9: r* and R* estimations for puff duration without outlier dataset

Both tables demonstrate the benefit of the outlier detection, since the range of CVr* changes

from [17.2% ; 41.8%] to [3.9% ; 12.3%] and the range of CVR* changes from [23.9% ;

57.4%] to [13.6% ; 32.3%].

Considering the range of the inter-lab-device variability limits computed after outlier removal,

the Acceptability Interval () was set at a conservative 30% to build the accuracy profile later.

Mode Regime Number

combination

labs-devices

Mean r* R* CVr* CVR*

LIT SR1 16 3.19 0.549 0.760 17.2% 23.9%

LIT SR2 16 3.76 1.361 1.531 36.2% 40.7%

LIT SR3 16 2.93 0.521 0.691 17.8% 23.6%

LIT SR4 16 2.09 0.688 0.855 33.0% 41.0%

LIT SR5 16 1.07 0.318 0.499 29.6% 46.5%

UNLIT SR1 16 3.18 0.695 0.869 21.9% 27.3%

UNLIT SR2 16 3.74 1.035 1.171 27.7% 31.3%

UNLIT SR3 16 2.92 0.905 1.035 31.0% 35.5%

UNLIT SR4 16 2.04 0.436 0.601 21.3% 29.5%

UNLIT SR5 16 1.08 0.453 0.621 41.8% 57.4%

Puff duration [s]

Mode Regime Number

combination

labs-devices


LIT SR1 12 3.15 0.126 0.553 4.0% 17.6%

LIT SR2 12 3.74 0.298 0.565 8.0% 15.1%

LIT SR3 12 2.90 0.342 0.529 11.8% 18.2%

LIT SR4 15 2.04 0.245 0.420 12.0% 20.6%

LIT SR5 12 1.01 0.124 0.267 12.3% 26.4%

UNLIT SR1 12 3.17 0.123 0.599 3.9% 18.9%

UNLIT SR2 12 3.72 0.179 0.507 4.8% 13.6%

UNLIT SR3 12 2.91 0.215 0.500 7.4% 17.2%

UNLIT SR4 12 2.01 0.124 0.363 6.2% 18.1%

UNLIT SR5 12 1.03 0.108 0.332 10.5% 32.3%

Puff duration [s]


4.2.2. Puff volume

Estimations for puff volume are summarized in Table 10 by keeping the outlier dataset

(combination laboratory-device) and in Table 11 by removing the outlier dataset. The

corresponding graphs named “individual values” are shown in Appendix C of this report.

Table 10: r*and R* estimations for puff volume with outlier dataset

Table 11: r* and R* estimations for puff volume without outlier dataset

Both tables demonstrate the benefit of the outlier detection, since the range of CVr* changes

from [11.0% ; 23.9%] to [4.3% ; 12.7%] and the range of CVR* changes from [21.0% ;

32.8%] to [9.7% ; 18.2%].

Considering the range of the inter-lab-device variability limits computed after outlier removal,

the Acceptability Interval () was set at 20% to build the accuracy profile later.

Mode Regime Number

combination

labs-devices


LIT SR1 16 36.9 4.80 10.23 13.0% 27.7%

LIT SR2 16 82.9 18.61 23.38 22.5% 28.2%

LIT SR3 16 93.9 14.07 19.74 15.0% 21.0%

LIT SR4 16 82.2 14.22 18.20 17.3% 22.1%

LIT SR5 16 51.5 11.63 16.86 22.6% 32.8%

UNLIT SR1 16 36.3 4.02 9.05 11.0% 24.9%

UNLIT SR2 16 80.7 15.78 20.74 19.6% 25.7%

UNLIT SR3 16 89.7 18.92 23.53 21.1% 26.2%

UNLIT SR4 16 79.9 19.06 24.08 23.9% 30.1%

UNLIT SR5 16 50.9 9.45 14.25 18.6% 28.0%

Puff volume [ml]

Mode Regime Number

combination

labs-devices


LIT SR1 12 35.3 2.01 4.56 5.7% 12.9%

LIT SR2 12 82.3 9.34 15.00 11.4% 18.2%

LIT SR3 12 91.6 11.63 14.53 12.7% 15.9%

LIT SR4 12 82.7 8.35 11.89 10.1% 14.4%

LIT SR5 12 51.6 2.68 7.41 5.2% 14.4%

UNLIT SR1 12 34.9 2.09 4.26 6.0% 12.2%

UNLIT SR2 12 81.6 3.63 11.25 4.4% 13.8%

UNLIT SR3 12 91.1 5.62 12.60 6.2% 13.8%

UNLIT SR4 12 80.9 4.10 9.36 5.1% 11.6%

UNLIT SR5 12 52.1 2.24 5.04 4.3% 9.7%

Puff volume [ml]


4.3. Intra and inter-lab-device variability modeling

We have investigated whether there is a relationship between the average for results and the

precision (r* and R*).


Figure 7 shows the power relationships established between the average puff duration and the

associated standard deviations of precision (Sr* and SR*).

Figure 7: Power relationships for puff duration and the associated Sr* and SR*

Sr* and SR* increase more or less linearly when puff duration increases. However, the

relationships between Sr* and puff duration are not good enough (R² lower than 0.5).

Improvements of model quality are shown in Figure 8, where non-linear relationships are

established between Sr* or SR* and the average flow.

Puff duration average (s)

Sr

or

SR

(s)

1.0 1.5 2.0 2.5 3.0 3.5 4.0

0.00

0.05

0.10

0.15

0.20

0.25 Sr LIT 0.0483x0.497 R² 0.294 Sr UNLIT 0.0369x0.3843 R² 0.465

SR LIT 0.0951x0.6071

R² 0.986 SR UNLIT 0.1109x0.4218

R² 0.766


Figure 8: Non-linear relationships for average flow and Sr* and SR* of puff duration

Considering results shown in Figure 8, the intra and inter-lab-device variabilities are non-

linearly dependent on average flow rate.

4.3.2. Puff volume

Figure 9 shows the power relationships established between the average puff volume and the

associated standard deviations of precision (Sr* and SR*).

Figure 9: Power relationships for puff volume and the associated Sr* and SR*

Flow rate average (ml/s)

Sr

or

SR

(s)

10 20 30 40 50 60

0.00

0.05

0.10

0.15

0.20

0.25 Sr LIT -0.0538 + 0.011x -2e-04x2 R² 0.964 Sr UNLIT 0.0112 + 0.0039x -1e-04x2 R² 0.708

SR LIT 0.1663 + 0.0038x -1e-04x2

R² 0.998 SR UNLIT 0.2372 -0.0023x -2e-06x2

R² 0.932

3.5s 4s 3s 2s 1s

Puff volume average (ml)

Sr

or

SR

(m

l)

30 40 50 60 70 80 90 100

0

1

2

3

4

5

6 Sr LIT 6e-04x1.9184

R² 0.953 Sr UNLIT 0.0223x0.9509

R² 0.844

SR LIT 0.0199x1.2356

R² 0.971 SR UNLIT 0.0214x1.1666

R² 0.928


Sr* and SR* increase more or less linearly when puff volume increases. Variability obtained

on LIT cigarettes is higher than that obtained on UNLIT cigarettes. Improvements of model

quality are shown in Figure 10, where non-linear relationships are established between Sr*

and SR* and the average flow.

Figure 10: Non-linear relationships for average flow and Sr* and SR* of puff volume

Considering results shown in the Figure 10, the intra and inter-lab-device variabilities are

non-linearly dependent on average flow rate.

4.4. Desirability indices

As described previously, the Desirability Index gives the device ‘fitness for purpose’ within

the limits of inter-lab-device variability when it is used within a given laboratory.

Consequently, for each of the two assessed parameters, 64 accuracy profiles were computed

for the 64 tested configurations (four devices * four laboratories * two modes * two samples).

Figures 11 and 12 summarize the computed Desirability Indices, distributed on a scale from

0% to 100%. Indices are categorized into three classes: 1) “Unsatisfactory” for indices below

60%, 2) “Questionable” for indices between 60% and 80%, and 3) “Fit for purpose” for

indices greater than 80%.

Additionally, the right margin provides the device code information (A-B-C-D) as well as the

laboratory code information (1-2-3-4). The left margin provides the number of defective

profiles.

Appendix D provides the complete Table of Indices.

Flow rate average (ml/s)

Sr

or

SR

(m

l)

10 20 30 40 50 60

0

1

2

3

4

5

6 Sr LIT -3.7528 + 0.4915x -0.0079x2

R² 0.985 Sr UNLIT -0.7841 + 0.1608x -0.0026x2

R² 0.881

SR LIT -2.777 + 0.5101x -0.008x2

R² 0.902 SR UNLIT -2.2137 + 0.4241x -0.0069x2

R² 0.964

35ml 80ml 90ml 80ml 50ml



Considering the range of inter-lab-device variability limits computed after outlier removal, the

Acceptability Interval () was set at 30% for puff duration. Desirability Indices are

summarized in Figure 11.

Figure 11: Desirability indices for puff duration

Device A:

Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.

Indeed, indices vary from 66% to 89% and two datasets were defective (from laboratories 1

and 4).

Device B:

Desirability Indices are systematically spread in the “Fit for purpose” area. Indeed, indices

vary from 86% to 97% and four datasets were defective (from laboratory 1).

Device C:


Indeed, indices vary from 69% to 95% and one dataset was defective (from laboratory 1).

Device D:


vary from 85% to 98%.

Unsatisfactory Questionable Fit for purpose

A

B

C

D

1

1

1

1

2

2

2

2

3

3

3

3

4

4

4

4

Nb=7 Nb=0 Nb=0 Nb=10 Nb=47

# D

efe

ctive

pro

file

#1

#1

#4

#1

0% 20% 40% 60% 80% 100%


4.4.2. Puff volume

Considering the range of inter-lab-device variability limits computed after outlier removal, the

Acceptability Interval () was set at 20% for puff volume. Desirability Indices are

summarized in Figure 12.

Figure 12: Desirability indices for puff volume

Device A:


vary from 83% to 95%.

Device B:


Indeed, indices vary from 69% to 90% and eight datasets were defective (from laboratories 1,

2, 3 and 4).

Device C:


Indeed, indices vary from 76% to 96% and one dataset was defective (from laboratory 1).

Device D:


Indeed, indices vary from 75% to 97% and four datasets were defective (from laboratory 1).

Unsatisfactory Questionable Fit for purpose

A

B

C

D

1

1

1

1

2

2

2

2

3

3

3

3

4

4

4

4

Nb=13 Nb=0 Nb=0 Nb=8 Nb=43

# D

efe

ctive

pro

file

#4#1#1#2

#1

#4

0% 20% 40% 60% 80% 100%


5. COMMENTS AND CONCLUSIONS

This study resulted in estimations of the intra-lab-device variability limits (r*) and the inter-

lab-device variability limits (R*) for the two parameters of puff duration and puff volume.

The study has been completed by using an accuracy profile per device and laboratory.

5.1. General comments

For puff duration and volume, non-linear relationships were observed between the flow rate

and the intra- and inter-lab-device variabilities.

Laboratory 1 reported some data containing odd values, leading to outlier removal and to

defective accuracy profiles.

It should be noted that the collaborative study took much longer than expected to complete,

starting in April 2013 and finishing in September 2015. As a reminder, during this period the

same device set was circulated to each of the four participating laboratories. None of the

devices or associated equipment were serviced for the duration of the study. This may have

affected the performance of some devices for the later laboratories that participated.

5.2. Puff duration

Coefficients of variation of the intra-lab-device variability CVr* (r* expressed as a percentage

of the overall mean) of puff duration ranged from 4% to 12% regardless of the cigarette

sample (West White or Camel Blue) and the mode (Lit or Unlit). However, the coefficients of

variation of the inter-lab-device variability ranged from 15% to 32%.

The Desirability Indices of non-defective accuracy profiles (57 of 64) ranged from 66% to

98%. Two devices, B and D, provided better Desirability Indices (from 85% to 98%) than A

and C (from 66% to 95%).

Most of the laboratories showed significantly higher puff duration variabilities for device A

than for the other devices.

Laboratory 1 reported data containing odd values, leading to outlier removal and to defective

accuracy profiles.

5.3. Puff volume

Coefficients of variation of the intra-lab-device variability CVr* ranged from 4% to 13%,

which is similar to those obtained for puff duration. However, the coefficients of variation of

the inter-lab-device variability for puff volume ranged from 10% to 18%, which is

significantly lower than the range obtained for puff duration.

The Desirability Indices of non-defective accuracy profiles (51 of 64) ranged from 69% to

97%. Device A provided the best Desirability Indices (from 83% to 95%), followed by C and

D (from 75% to 97%), and then device B (from 69% to 90%, but with a high percentage

[50%] of defective accuracy profiles).

Most of the laboratories showed significantly higher puff volume variabilities for device B

than for the other devices.

The high variabilities reported for device D seemed to be the result of odd data obtained with

one of the three units used.


APPENDIX A – List of participating laboratories and device

manufacturers

Laboratories and devices are listed below in alphabetic order. The codifications used in this

report were randomly assigned to laboratories and devices.

Laboratories Country

BAT United Kingdom

ITG France

RJRT USA

SODIM France

Device Manufacturer

CME-P4 RJRT

Cressmicro BORGWALDT

SA7 BAT

SPA-M SODIM


APPENDIX B – Part 2 - Intra and inter-lab-device variability

estimations

B.1 Puff duration

Estimations for puff duration are summarized in Table B.1.1 by keeping outlier dataset

(combination laboratory-device) and in Table B.1.2 by removing outlier dataset.

Table B.1.1: r* and R* estimations for puff duration with outlier dataset

Table B.1.2: r* and R* estimations for puff duration without outlier dataset

B.2 Puff Volume

Estimations for puff volume are summarized in Table B.2.1 by keeping outlier dataset

(combination laboratory-device) and in Table B.2.2 by removing outlier dataset.

Table B.2.1: r* and R* estimations for puff volume with outlier dataset

Table B.2.2: r* and R* estimations for puff volume without outlier dataset

Mode Regime Nb combination

labo-device


LIT SR3 16 2.96 1.452 1.645 49.0% 55.5%

UNLIT SR3 16 2.94 1.585 1.720 53.9% 58.5%

Puff duration [s]


labo-device


LIT SR3 12 2.85 0.273 0.445 9.6% 15.6%

UNLIT SR3 12 2.83 0.186 0.456 6.6% 16.1%

Puff duration [s]


labo-device


LIT SR3 16 92.66 21.627 28.035 23.3% 30.3%

UNLIT SR3 16 89.38 18.840 23.860 21.1% 26.7%

Puff volume [ml]


labo-device


LIT SR3 12 92.72 11.039 16.647 11.9% 18.0%

UNLIT SR3 12 90.47 5.508 11.597 6.1% 12.8%

Puff volume [ml]


APPENDIX C – Data representation

The following figures show the raw data for the five smoking regimes, the two lighting modes

and the two reported parameters. The plots indicate mean values with 95% confidence

interval (CI) highlighted in green (for valid laboratories) or in red (for laboratories detected as

an outlier). Laboratories excluded by outlier statistics are denoted by the name of the test

rejecting the laboratory (G for Grubbs, G2 for double Grubbs or C for Cochran). The

confidence limits at 95% and 99% for the difference between the reference value (average of

all “valid” laboratories) and the average of one laboratory are plotted in green and red,

respectively.

Codes for participating laboratories that did not provide data are marked with an asterisk.


C.1 Puff duration

Smoking regime 1

Figure 13: Individual observations of Puff Duration in LIT mode

Figure 14: Individual observations of Puff Duration in UNLIT mode


Smoking regime 2




Smoking regime 3




Smoking regime 4




Smoking regime 5




C.2 Puff Volume

Smoking regime 1

Figure 23: Individual observations of Puff Volume in LIT mode

Figure 24: Individual observations of Puff Volume in UNLIT mode


Smoking regime 2




Smoking regime 3




Smoking regime 4




Smoking regime 5




APPENDIX D – Accuracy profile results

Puff Duration: complete results of the computed accuracy profiles.

Device.code Labo.code Lighting Sample Beta Lambda Accuracy_Index Range_Index Precision_Index Trueness_Index

A 1 LIT CB 90 30 0 0 0 0

A 1 LIT WW 90 30 76.5 73.26 64.07 95.36

A 1 UNLIT CB 90 30 74.31 69.3 61.36 96.48

A 1 UNLIT WW 90 30 81.61 84.33 67.93 94.9

A 2 LIT CB 90 30 79.79 78.79 68.34 94.33

A 2 LIT WW 90 30 66.35 39.66 77.57 94.92

A 2 UNLIT CB 90 30 86.85 87.17 77.6 96.84

A 2 UNLIT WW 90 30 69.18 46.4 75.56 94.45

A 3 LIT CB 90 30 74.66 61.86 68.67 97.98

A 3 LIT WW 90 30 84.6 83.48 77.24 93.91

A 3 UNLIT CB 90 30 80.82 72.1 78.95 92.74

A 3 UNLIT WW 90 30 82.52 75.62 79.02 94.02

A 4 LIT CB 90 30 0 0 0 0

A 4 LIT WW 90 30 86.38 84.56 79.96 95.31

A 4 UNLIT CB 90 30 87.84 94.85 73.43 97.31

A 4 UNLIT WW 90 30 89.09 90.43 81.44 96

B 1 LIT CB 90 30 0 0 0 0

B 1 LIT WW 90 30 0 0 0 0

B 1 UNLIT CB 90 30 0 0 0 0

B 1 UNLIT WW 90 30 0 0 0 0

B 2 LIT CB 90 30 86.51 100 71.68 90.34

B 2 LIT WW 90 30 91.92 100 91.12 85.23

B 2 UNLIT CB 90 30 86.47 100 73.26 88.27

B 2 UNLIT WW 90 30 91.91 100 91.35 84.99

B 3 LIT CB 90 30 97.05 100 95.32 95.91

B 3 LIT WW 90 30 95.72 100 94.51 92.81

B 3 UNLIT CB 90 30 96.77 100 94.97 95.41

B 3 UNLIT WW 90 30 96.16 100 95.92 92.7

B 4 LIT CB 90 30 93.7 100 90.92 90.48

B 4 LIT WW 90 30 93.9 100 96.12 86.13

B 4 UNLIT CB 90 30 93.7 100 92.43 89.02

B 4 UNLIT WW 90 30 94.12 100 97.34 85.66

C 1 LIT CB 90 30 79.68 100 52.32 96.68

C 1 LIT WW 90 30 92.98 100 84.15 95.51

C 1 UNLIT CB 90 30 0 0 0 0

C 1 UNLIT WW 90 30 92.87 100 84.18 95.15

C 2 LIT CB 90 30 69.39 100 36.42 91.74

C 2 LIT WW 90 30 89.39 100 82.21 86.89

C 2 UNLIT CB 90 30 74.65 100 46.24 89.95

C 2 UNLIT WW 90 30 87.42 100 77.16 86.59

C 3 LIT CB 90 30 91.33 100 78.98 96.44

C 3 LIT WW 90 30 95.07 100 89.62 95.88

C 3 UNLIT CB 90 30 93.11 100 84.02 96.09

C 3 UNLIT WW 90 30 94.71 100 88.8 95.67

C 4 LIT CB 90 30 75.19 100 46.91 90.62

C 4 LIT WW 90 30 87.84 100 78.12 86.75

C 4 UNLIT CB 90 30 82.66 100 63.31 89.2

C 4 UNLIT WW 90 30 88.57 100 80.12 86.73

D 1 LIT CB 90 30 88.21 100 71.24 96.35

D 1 LIT WW 90 30 97.44 100 93.23 99.24

D 1 UNLIT CB 90 30 96.45 100 91.02 98.58

D 1 UNLIT WW 90 30 97.67 100 93.86 99.26

D 2 LIT CB 90 30 85.8 100 72.27 87.41

D 2 LIT WW 90 30 93.86 100 91.94 89.92

D 2 UNLIT CB 90 30 92.43 100 88.49 89.23

D 2 UNLIT WW 90 30 91.43 100 84.8 90.14

D 3 LIT CB 90 30 89.03 100 72.51 97.32

D 3 LIT WW 90 30 98.23 100 96.51 98.21

D 3 UNLIT CB 90 30 97.36 100 94.07 98.1

D 3 UNLIT WW 90 30 98.41 100 96.89 98.38

D 4 LIT CB 90 30 85.39 100 73.43 84.79

D 4 LIT WW 90 30 92.46 100 88.24 89.58

D 4 UNLIT CB 90 30 93.95 100 93.84 88.38

D 4 UNLIT WW 90 30 92.79 100 89.05 89.71


Puff Volume: complete results of the computed accuracy profiles.

Device.code Labo.code Lighting Sample Beta Lambda Accuracy_Index Range_Index Precision_Index Trueness_Index

A 1 LIT CB 90 20 83.09 100 66.36 86.45

A 1 LIT WW 90 20 86.7 100 69.22 94.15

A 1 UNLIT CB 90 20 86.15 98.52 68.78 94.37

A 1 UNLIT WW 90 20 86.63 100 68.05 95.53

A 2 LIT CB 90 20 84.08 100 72.1 82.43

A 2 LIT WW 90 20 93.41 100 86.36 94.38

A 2 UNLIT CB 90 20 92.84 100 82.28 97.24

A 2 UNLIT WW 90 20 91.89 100 80.85 95.98

A 3 LIT CB 90 20 91 100 77.22 97.57

A 3 LIT WW 90 20 90.86 100 77.31 97.02

A 3 UNLIT CB 90 20 94.6 100 89.62 94.47

A 3 UNLIT WW 90 20 91.52 100 79.38 96.57

A 4 LIT CB 90 20 84.25 100 75.54 79.17

A 4 LIT WW 90 20 91.26 100 81.78 92.94

A 4 UNLIT CB 90 20 93.62 100 85.2 96.32

A 4 UNLIT WW 90 20 90.78 100 79.34 94.31

B 1 LIT CB 90 20 0 0 0 0

B 1 LIT WW 90 20 0 0 0 0

B 1 UNLIT CB 90 20 0 0 0 0

B 1 UNLIT WW 90 20 0 0 0 0

B 2 LIT CB 90 20 0 0 0 0

B 2 LIT WW 90 20 90.35 100 89.84 82.09

B 2 UNLIT CB 90 20 74.15 86.95 86.8 54.01

B 2 UNLIT WW 90 20 89.02 100 84.31 83.68

B 3 LIT CB 90 20 0 0 0 0

B 3 LIT WW 90 20 87.27 100 74.34 89.4

B 3 UNLIT CB 90 20 69.15 84.95 69.14 56.29

B 3 UNLIT WW 90 20 85.87 100 70.9 89.3

B 4 LIT CB 90 20 0 0 0 0

B 4 LIT WW 90 20 0 0 0 0

B 4 UNLIT CB 90 20 71.63 83.99 83.71 52.27

B 4 UNLIT WW 90 20 87.01 100 76.66 85.92

C 1 LIT CB 90 20 76.48 98.58 47.37 95.79

C 1 LIT WW 90 20 79.99 100 52.56 97.36

C 1 UNLIT CB 90 20 86.69 100 66.35 98.19

C 1 UNLIT WW 90 20 0 100 0 97.36

C 2 LIT CB 90 20 88.92 100 73.99 95.03

C 2 LIT WW 90 20 94.74 100 86.21 98.64

C 2 UNLIT CB 90 20 92.03 100 79.41 98.15

C 2 UNLIT WW 90 20 92.11 100 79.67 98.08

C 3 LIT CB 90 20 90.73 100 74.81 99.86

C 3 LIT WW 90 20 94.23 100 84.25 99.31

C 3 UNLIT CB 90 20 89.6 100 72.08 99.79

C 3 UNLIT WW 90 20 94.77 100 85.61 99.41

C 4 LIT CB 90 20 93.61 100 85.38 96.07

C 4 LIT WW 90 20 95.54 100 88.54 98.51

C 4 UNLIT CB 90 20 95.2 100 87.61 98.47

C 4 UNLIT WW 90 20 94.86 100 86.95 98.18

D 1 LIT CB 90 20 0 0 0 0

D 1 LIT WW 90 20 0 0 0 0

D 1 UNLIT CB 90 20 0 0 0 0

D 1 UNLIT WW 90 20 0 0 0 0

D 2 LIT CB 90 20 90.08 100 75.86 96.35

D 2 LIT WW 90 20 97.31 100 92.26 99.87

D 2 UNLIT CB 90 20 93.15 100 82.55 97.93

D 2 UNLIT WW 90 20 97.04 100 91.85 99.48

D 3 LIT CB 90 20 87.29 97.5 75.7 90.1

D 3 LIT WW 90 20 97.41 100 95.82 96.45

D 3 UNLIT CB 90 20 77.51 84.84 69.12 79.42

D 3 UNLIT WW 90 20 96.77 100 95.35 95.05

D 4 LIT CB 90 20 75.09 100 43.34 97.69

D 4 LIT WW 90 20 91.88 100 77.87 99.6

D 4 UNLIT CB 90 20 78.57 100 49.82 97.35

D 4 UNLIT WW 90 20 91.3 100 76.76 99.15


APPENDIX E – Experimental protocol

Study title

CORESTA collaborative study to evaluate the accuracy and precision of puffing topography

devices.

Background/Rationale

Commercial and proprietary puffing topography equipment have been available and used for

many years. However, there is limited information regarding validation of instruments used

for puffing topography. The FDA’s recent Draft Guidance for Applications for Premarket

Review of New Tobacco Products (PMTA) recommends studies to evaluate topography data

from adult smokers. This study has the advantage of evaluating several topography devices

under the same conditions, in different laboratories.

Objectives

This study will measure puff volume and puff duration with four different types of human

topography devices, using two commercial cigarettes that represent the low and higher end

pressure drops (PD) of commercial products. The cigarettes will be tested over various puff

volumes and durations in both unlit and lit machine smokings. The data collected will be used

to determine:

Accuracy: How close a measured value is to the true value (calibrated puff volume).

This will be done by taking 10 puffs on unlit cigarettes and 10 puffs on lit cigarettes

(unless a set butt length is reached) at each of five puffing regimes (Table 1).

Precision: How close the measured values of each unit are to each other (measured puff

volume). This is measured in terms of Repeatability and Reproducibility.

Repeatability: within device and within laboratory will be determined.

Reproducibility: within and between device type and within and between laboratories

will be determined.

Design

This study will assess four topography devices over a matrix of puff volumes and durations

that cover the range of these parameters found in typical cigarette smoking behaviour. It will

include measuring two different commercial cigarette brands with different pressure drops and

the cigarettes will be measured both when unlit and lit.

Each participating laboratory will be provided with five units of each of the four different types

of topography devices, although only three units of each type will be used in the study. Serial

numbers will be recorded to ensure that each laboratory uses the same subset of units.

Each device will be calibrated prior to use following provided instructions. A smoking

machine or puff generator (Borgwaldt A14 syringe driver) capable of drawing constant flow

rates will be used to generate the five puffing regimes (Table 1). The smoking machine will be

set up to produce the constant flow rates and duration conditions shown in Table 1. A bubble

meter will be used to confirm the puff volumes after changing smoking machine settings and

daily prior to testing. When confirming the puff volume, a standard pressure drop of

approximately 100mmWG will be used. Bubble meter readings will be recorded.


Table 1: Puff Volume, Duration Matrix

Regime Puff

Profile

Puff Volume (mL)

Duration (s)

Inter-Puff Interval(s)

Flow Rate (mLs

-1)

1 Square 35 3.5 30 10

2 Square 80 4.0 30 20

3 Square 90 3.0 30 30

4 Square 80 2.0 30 40

5 Square 50 1.0 30 50

Each puff in the matrix above will be drawn 10 times (10 total puffs) through each of the three

units of the four different topography devices with the cigarette initially unlit. Following each

set of 10 unlit puffs, the cigarette will be lit using an electric lighter and 10 puffs will be drawn,

or if drawing 10 puffs would burn into the filter, puffs will be drawn through the cigarette until

the burning coal reaches a pre-determined butt length (overwrap +3 mm). This process will be

repeated for both commercial cigarette brands.

In addition, the 30 mLs-1

flow rate for 3.0 seconds duration, puff volume 90 mL smoking

regime will be repeated (x 10 lit & unlit puffs) with one of each of the four units over five

separate days, smoking both the West King Size and Camel King Size cigarettes. The same

unit should be used for the additional smoking on each day. The original day of smoking on

each device will be counted as the first of the five days.

Settings

This analysis will be performed in a laboratory at each of the participating companies.

Laboratory conditions will be controlled according to ISO 3308:

Temperature: 22°C +/- 2°C;

Humidity: 60%RH + / - 5%RH

The cigarettes will be conditioned prior to testing, according to ISO 4387:

Temperature: 22°C +/- 1°C;

Humidity: 60%RH + / - 3%RH

Cigarettes will be conditioned for at least 48 hours but no more than 10 days.

There is no requirement to record the airflow above the cigarettes.

Study equipment and supplies

Topography devices under investigation include:

1. CReSSmicro (Borgwaldt)

2. SPA-M (Sodim)

3. SA7 (BAT)

4. CME5-P4 (RJRT)

Five units of each of the above devices will be shipped to each laboratory taking part in the

study.

5 Carolina Medical Electronics, Inc.


Cigarettes:

A sufficient quantity of each of the two commercial cigarettes will be supplied along with the

devices. The two commercial cigarettes are:

West White King Size Box (ITG)

Camel Blue King Size Box (RJRT)

Physical Data for:

West White King Size Box

o Open pressure drop (draft) - 45 mm of H20 (SD = 1.4)

o Closed pressure drop (draft) - 151 mm of H20 (SD = 11.0)

o Filter pressure drop (draft) - NA

Camel Blue King Size Box

o Open pressure drop (draft) - 130.5 mm of H20 (SD=7.5)

o Closed pressure drop (draft) - 157.1 mm of H20 (SD=9.3)

o Filter pressure drop (draft) - 98.0 mm of H20 (SD=6.5)

Smoking Machine :

A Borgwaldt A14 syringe driver (single-port smoking machine) or a linear, multi-port smoking

machine, with the capability of generating square puff profiles, will be used in each laboratory.

The Borgwaldt A14 syringe driver will be provided by BAT. The Cambridge filter pad will be

changed after smoking each six cigarettes in the lit condition.

Assessments

Ten measurements of

Puff volume (mL)

Puff duration (s)

Peak flow (mLs-1, where possible)

Resistance to draw (mLs-1-mmH2O, where possible)

Each puff in the matrix will be drawn through each of the three units of all four different

topography devices and recorded with the cigarette unlit first, followed by lighting the cigarette

with an electric lighter and puffing either 10 puffs or to a pre-determined butt length

(overwrap+3mm) if 10 puffs would result in burning into the filter.

This will be repeated sequentially using the second commercial cigarette.

In addition, the 30 mLs-1

flow rate for 3.0 seconds duration, puff volume 90 mL smoking

regime will be repeated (x 10 lit puffs only) with one of each of the four devices over five

separate days (same unit each day).

Operation

The laboratory operators will be trained on the usage of each type of topography device. A

user guide will be available for each device type.

Data collation

A standard template (Excel spreadsheet) will be provided to report the measurements.


Data analysis

The evaluation of instruments will be based on a comparison of accuracy profiles.

Figure 1 gives an illustration of an accuracy profile with the key definitions.

Figure. 1: Illustration of an accuracy profile.

In this study, for each laboratory and each device, two accuracy profiles will be developed by

estimating the precision and the “trueness” 1) for each flow rate and 2) for each puff duration.

A specific device will meet the objective if the accuracy profile is within the acceptance limits.

The acceptance limits, for each studied parameter, will be defined by the CORESTA subgroup

members after a preliminary analysis of results.

An example of accuracy profiles is shown in Figure 2 (accuracy profiles based on the flow rate

to compare two instruments).

Figure. 2: Accuracy profiles based on the flow rate of devices 1 and 2.

To compare the accuracy profile of different devices, an indicator score will be calculated.

This score is the sum of the combination of three quality criteria:

Range Index

Precision Index

Trueness Index

bias (%)

parameter

+

-

C1 C2 C3 C4

RLinfRLsup

RANGE

mean relative bias

accep

tance

limits

0

bias limits of confidence

"Profil Exactitude : Modèle linéaire"

Concentrations

Re

co

uvre

me

nt

(%)

10 20 30 40 50

70

80

90

100

110

120

130

Device 1

Flow rate

Recovery

(%

)


Concentrations

Re

co

uvre

me

nt

(%)

10 20 30 40 50

70

80

90

100

110

120

130

Device 1

Flow rate

Recovery

(%

)


Concentrations

Re

co

uvre

me

nt

(%)

10 20 30 40 50

60

80

100

120

Device 2

Flow rate

Re

co

ve

ry (

%)


Concentrations

Re

co

uvre

me

nt

(%)

10 20 30 40 50

60

80

100

120

Device 2

Flow rate

Re

co

ve

ry (

%)

Device 2

Flow rate

Re

co

ve

ry (

%)


A score closer to 100 indicates that the device better met the objective. For example, in Table

2, Device 2 (Accuracy_Index = 78.67) is slightly better for this comparison than Device 1

(Accuracy_Index = 77.95).

Table 2: Accuracy index for devices 1 and 2

This statistical evaluation is proposed for the unlit cigarettes by considering a puff as a

replicate to estimate the repeatability (10 puffs), and an instrument unit as a replicate to

estimate the intermediate precision (3 units). The same evaluation will be made with the lit

cigarettes, with the exception that a variable number of puffs may be used (by smoking instead

to a set butt length), as required. See Appendix 2 for an explanation of terms used in the Data

Analysis section.

Appendix 1

Borgwaldt A14 Syringe driver entry guidelines

Setting Entry / Format Entry Example

Keyboard Control 2 2

Smoking 2 2

Puff Parameters 2 2

Bell profile? No No

Profile no. 1 1 = square puff

Correct? Yes Yes

Puff vol. (ml) ### 080

Correct? Yes Yes

Puff dur. (s) #,# 4,0 = 4.0

Correct? Yes Yes

Puff period (s) ## 30

Correct? Yes Yes

Puff vol. test? Yes Yes

Tester o.k.? Connect bubble meter prior to

selecting Yes

Yes

Puff volume test

Volume o.k.? Yes Yes

Accuracy_Index Range_Index Precision_Index Trueness_Index RL_Inf RL_Sup

Device 1 77.95 99.26 48.68 98.01 10.3 50.0

Device 2 78.67 93.1 54.23 96.46 12.8 50.0


Appendix 2

The relevance of method performance is defined by indicators6

Accuracy: closeness of agreement between a test result or measurement result and the true

value. Accuracy refers to a combination of trueness (common systematic error) and precision

(random error).

Trueness: closeness of agreement between the expected test result or measured result

and a true value.

Precision: closeness of agreement between independent test/measurement results

obtained under stipulated conditions.

-repeatability: precision under repeatability conditions.

-repeatability conditions: observation conditions where independent

test/measurement results are obtained with the same method on identical

test/measurement items in the same test or measurement facility by the same

operator using the same equipment within a short interval of time.

-intermediate precision: precision under intermediate precision conditions.

-intermediate precision conditions: conditions where test results or

measurement results are obtained with the same method, on identical

test/measurement items in the same test or measurement facility, under some

different operating condition.

6 ISO 15725-1

TruenessTrueness

Accuracy

=

Trueness + precision


APPENDIX F – Departures from experimental protocol

No departures from the experimental protocol were reported.


APPENDIX G – Definitions

Repeatability r:

The difference between two single results found on matched cigarette samples by one operator

using the same apparatus within the shortest feasible time interval will exceed the repeatability

value ( r ) on average not more than once in 20 cases in the normal and correct operation of the

method.

ISO 10362-1:1991(E), page 3

Reproducibility R:

Single results on matched cigarette samples reported by two laboratories will differ by more

than the reproducibility value ( R ) on average not more than once in 20 cases in the normal

and correct operation of the method.

ISO 10362-1:1991(E), page 3

1. INTRODUCTION2. EXPERIMENTAL DESIGN2.1 Part 1: Intra and inter-lab-device variability2.2 Part 2: Time effect

3. STATISTICAL EVALUATION3.1. Uncertainty evaluation3.2. Accuracy profile3.3. Limitations

4. RESULTS4.1. Outlier results4.1.1. Puff duration4.1.2. Puff volume

4.2. Intra and inter-lab-device variability estimations4.2.1 Puff duration4.2.2. Puff volume

4.3. Intra and inter-lab-device variability modeling4.3.1. Puff duration4.3.2. Puff volume

4.4. Desirability indices4.4.1. Puff duration4.4.2. Puff volume

5. COMMENTS AND CONCLUSIONS5.1. General comments5.2. Puff duration5.3. Puff volume

APPENDIX A – List of participating laboratories and device manufacturersAPPENDIX B – Part 2 - Intra and inter-lab-device variability estimationsAPPENDIX C – Data representationAPPENDIX D – Accuracy profile resultsAPPENDIX E – Experimental protocolAPPENDIX F – Departures from experimental protocolAPPENDIX G – Definitions

Technical Report Puffing Topography Inter-lab Study · 2017. 2. 21. · 3 ISO 5725-1:1994 Accuracy...

Documents

Transcript of Technical Report Puffing Topography Inter-lab Study · 2017. 2. 21. · 3 ISO 5725-1:1994 Accuracy...