Smoking Behaviour Sub-Group
Technical Report
Puffing Topography Inter-lab Study
January 2017
Study Co-ordinators:
Krishna Prasad and Sandy Slayford
British American Tobacco, United Kingdom
Statistical Support:
Thomas Verron and Rémi Julien
SEITA – Imperial Tobacco, France
Table of Contents
1. INTRODUCTION ............................................................................................................... 3
2. EXPERIMENTAL DESIGN ............................................................................................... 3
2.1 Part 1: Intra and inter-lab-device variability ....................................................................... 4
2.2 Part 2: Time effect ................................................................................................................ 5
3. STATISTICAL EVALUATION ......................................................................................... 6
3.1. Uncertainty evaluation ......................................................................................................... 6
3.2. Accuracy profile ................................................................................................................... 8
3.3. Limitations ............................................................................................................................ 9
4. RESULTS .......................................................................................................................... 10
4.1. Outlier results ...................................................................................................................... 10
4.1.1. Puff duration ................................................................................................. 11
4.1.2. Puff volume .................................................................................................. 13
4.2. Intra and inter-lab-device variability estimations ............................................................. 15
4.2.1 Puff duration ................................................................................................. 15
4.2.2. Puff volume .................................................................................................. 16
4.3. Intra and inter-lab-device variability modeling ................................................................ 17
4.3.1. Puff duration ................................................................................................. 17
4.3.2. Puff volume .................................................................................................. 18
4.4. Desirability indices ............................................................................................................. 19
4.4.1. Puff duration ................................................................................................. 20
4.4.2. Puff volume .................................................................................................. 21
5. COMMENTS AND CONCLUSIONS .............................................................................. 22
5.1. General comments .............................................................................................................. 22
5.2. Puff duration ....................................................................................................................... 22
5.3. Puff volume......................................................................................................................... 22
APPENDIX A – List of participating laboratories and device manufacturers............................ 23
APPENDIX B – Part 2 – Intra and inter-lab-device variability estimations ............................... 24
APPENDIX C – Data representation .......................................................................................... 25
APPENDIX D – Accuracy profile results ................................................................................... 36
APPENDIX E – Experimental protocol ...................................................................................... 38
APPENDIX F – Departures from experimental protocol ........................................................... 44
APPENDIX G – Definitions ....................................................................................................... 45
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 3/45
1. INTRODUCTION
In 2012, the CORESTA Sub-Group, Smoking Behaviour (SG-TSB), agreed to conduct a
study to determine if various human puffing topography devices deliver comparable results
for puff volume and puff duration. In order to assess this, the uncertainty of both puffing
topography devices and laboratories has been evaluated based on the experimental design
described below.
2. EXPERIMENTAL DESIGN
Four laboratories took part in this study and four puffing topography devices (see Figure 1)
were evaluated by four different SG-TSB member organizations. Appendix A lists the
laboratories and device manufacturers involved in the study. Note that the listings of
laboratories and device manufacturers are in alphabetical order, while the results are presented
by code (1-4 for laboratories; A-D for devices). Labs and devices were both randomized for
data presentation, so nothing can (or should) be inferred about the identity of any lab or
device based on the order of presentation in this report.
CME–P4
CReSSmicro
SA7
SPA-M
Figure 1: Presentation of the four evaluated devices
The study was designed to evaluate the intra-lab-device variability (r*) and the inter-lab-
device variability (R*) of the puffing topography devices (Part 1) and to evaluate the time
effect (Part 2).
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 4/45
2.1 Part 1: Intra and inter-lab-device variability
In this part, three different units of each type of device were tested. The same device set was
circulated to each of the four participating laboratories over the course of two years. Each
participating laboratory was asked to test the set of 12 puffing topography devices (four
brands and three units of each) on two samples of king size cigarettes: Camel Blue (CB) and
West White (WW). These corresponded to two different open pressure drops: 45 mmWG and
130.5 mmWG, respectively. Two different cigarette modes were considered as well: lit and
unlit. Whichever mode, a total of 10 puffs per sample was taken at each smoking regime (see
Table 1).
The data structure with the key figures is illustrated in Figure 2. A total of 9600 lines of data
was generated for each of the three parameters (puff volume, puff duration and flow rate).
Figure 2: Data structure of Part 1
Table 1 gives the details of the five smoking regimes considered in Part1. All puff profiles
were square with inter-puff intervals of 30 s.
Table 1: Set of five smoking regimes
Smoking Regime Puff Volume (ml) Puff Duration (s) Average Flow (ml/s)
1 35 3.5 10
2 80 4.0 20
3 90 3.0 30
4 80 2.0 40
5 50 1.0 50
Puffing Topography Study
Lab 1
A
B
C
D
Rep. 1
Rep. 2
….
Rep. 10
Terms
Lab 4 Lab 2 Participating laboratories (4)
Evaluated devices (4)
Puffing regimes (5)
Replicates (10)
Individual results (9600)
Lab 3
Evaluated units (3)
Tested configurations (4) CB Lit CB Unlit WW Lit WW Unlit Unlit
#1 #2 #3
SR1 SR2 SR3 SR4 SR5
Equipment
Protocol
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 5/45
The objective was to determine the maximum difference expected between given puff
volumes or durations measured: 1) using a given device in a given lab and 2) using different
devices in different labs.
2.2 Part 2: Time effect
In order to evaluate the time effect, each participating laboratory was asked to test only one
unit per device, with only one puffing regime (3) on five separate days (see Figure 3). A total
of 3200 lines of data was generated for each of the three parameters (puff volume, puff
duration and flow rate).
Figure 3: Data structure Part 2
Puffing Topography Study
Lab 1
A
B
C
D
SR3
Rep. 1
Rep. 2
….
Rep. 10
Terms
Lab 4 Lab 2 Participating laboratories (4)
Evaluated devices (4)
Puffing regime (1)
Replicates (10)
Individual results (3200)
Lab 3
Evaluated unit (1)
Tested configurations (4)
#1
Testing days (5) D1 D2 D3 D4 D5
CB Lit CB Unlit WW Lit WW Unlit Unlit
Equipment
Protocol
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 6/45
3. STATISTICAL EVALUATION
3.1. Uncertainty evaluation
The statistical evaluation of data for this collaborative study followed the methods provided
by ISO 5725-21. For outlier testing, the Grubbs and Cochran methods were used.
ISO 5725-2 Tests Consistency
Cochran’s test Within-laboratory variability: Suitable for detecting whether or not the highest value in a set of laboratory standard deviations is an outlier (test statistic greater than 1% critical value) or a straggler (test statistic greater than 5% and less than 1% critical value).
Grubbs’ test Between-laboratory variability: Suitable for detecting whether or not the highest (or lowest) laboratory’s average is an outlier (test statistic greater than 1% critical value) or a straggler (test statistic greater than 5% and less than 1% critical value).
Both statistical tests allowed the identification of stragglers and outliers, however only the
outlier values were discarded in the statistical evaluation.
The protocol “Harmonized Statistical Procedure” defined by IUPAC (International Union of
Pure and Applied Chemistry) was applied. This involved sequential applications of the
Cochran and Grubbs tests until no further outliers were detected or until the data dropped
amounted to 22.2% (for this study = 2/9) of the total data collected (see Figure 4).
The Cochran outlier test was applied first, and if an outlying laboratory was identified, then a
single-value Grubbs test was performed on the individual values of this outlying laboratory
(Individual Grubbs). If no individual value was identified as an outlier, the outlying
laboratory is removed. If an individual value was identified as an outlier, only that value was
removed. This was followed by the application of single-value Grubbs test and outlying
laboratories were removed. If no laboratory was identified as an outlier, the pair-value test
was applied (two values at the same end). Any laboratory(ies) flagged by these tests were
removed until the data dropped amounted to 22.2% (for this study = 2/9) of the total data
collected.
1 “ISO 5725-2:1994: Accuracy (trueness and precision) of measurement methods and results – Part 2: Basic
method for the determination of repeatability and reproducibility of a standard measurement method.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 7/45
Figure 4: Process of outlier detection
At the end of the loop of outlier detection, if any laboratory(ies) had been removed, then the
process was stopped. On the remaining data set, a nested ANOVA was performed in order to
evaluate the intra-lab-device variability (r*) and the inter-lab-device variability (R*).
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 8/45
3.2. Accuracy profile
In addition for Part 1, the accuracy profile approach was used to test the ‘fitness for purpose’
of the devices when used by one laboratory. The accuracy profile is a graphical decision-
making tool (see Figure 5).
A Tolerance Interval is computed considering the puff measurements (10 replicates) and
intermediate precision (three units) and Acceptability Interval (λ) defined by the
collaborative study are determined for each device type. These intervals give information
about the expected proportion of future measurements that will lie at a certain level of risk (1
– β) in the specified range and whether or not the device results are acceptable.
Figure 5: Accuracy profile as a graphical decision-making tool
The ‘fitness-for-purpose’ of the device is represented by the Desirability Index (ID), which is
computed as the cube root of the product of the three indices: the Range Index (IR), the
Precision Index (IP) and the Trueness Index (IT). All indices vary from 0 to 1. For further
details on computation of the Desirability Index (ID), refer to E. Rozet et al2 .
2 E. Rozet, V. Wascotte, N. Lecouturier, V. Préat, W. Dewé, B. Boulanger, P. Hubert, Improvement of
the decision efficiency of the accuracy profile by means of a desirability function for analytical
methods validation: Application to a diacetyl-monoxime colorimetric assay used for the determination
of urea in transdermal iontophoretic extracts, Analytica Chimica Acta, Volume 591, Issue 2, 22 May
2007, Pages 239-247.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 9/45
Finally, the Desirability Index gives the ‘fitness for purpose’ of the device within the limits of
inter-lab-device variability (estimated in the collaborative study) when it is used within a
given laboratory. Figure 6 shows some examples of the range of information provided by the
accuracy profiles at different levels of the Desirability Index.
Figure 6: Examples of desirability index
3.3. Limitations
Despite having a well-balanced design, there are some limitations to this collaborative study.
First of all, the small number of participating laboratories (only four) cannot be representative
of inter-laboratories variability, and can impact the interpretation of the statistical tests of
outlier detection. Although the four participating laboratories were experienced in the use of
puffing topography devices, it would have been better to have at least eight laboratories as
recommended by ISO 5725-13 and ASTM E2480
4.
Secondly, the small number of participating laboratories makes evaluation per device less
robust. Consequently, we had to combine the inter-laboratories variability with the inter-
3 ISO 5725-1:1994 Accuracy (trueness and precision) of measurement methods and results -- Part 1: General
principles and definitions.
4 ASTM E2480 - 12 Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a
Test Method with Multi-Valued Measurands.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 10/45
devices variability, to calculate global lab-device performance evaluation. With such
evaluation, the inter-laboratories variability and inter-devices variability are confounded. For
this reason, we decided to designate the intra-lab-device variability and the inter-lab-device
variability as r* and R*, respectively, in order to avoid any confusion with the usual notations
r and R for repeatability and reproducibility respectively.
Thirdly, although global indices allow a comparison of devices, it is important to note that a
bad index is not necessarily the consequence of a bad device; it can also reflect a bad use of
the device, or a cigarette sampling issue.
4. RESULTS
This section relates only to the results obtained within Part 1 of the study. Appendix B
provides the intra-lab-device variability and the inter-lab-device variability estimations for
Part 2 for information purposes only. Indeed, the evaluations conducted within this study lead
to the conclusion that the estimations of the intra-lab-device variability and the inter-lab-
device variability are similar between both parts. This means that the major part of the study
variability has been captured in Part 1.
4.1. Outlier results
In order to accurately estimate the intra-lab-device and the inter-lab-device variabilities, the
outlier detection procedure, as described previously, was applied to 160 datasets
corresponding to five smoking regimes, two lighting modes and 16 combinations of
laboratories-devices (four laboratories and four devices). Each dataset is a set of 60 individual
results (three units, two samples and 10 puffs).
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 11/45
4.1.1. Puff duration
Table 2 provides the summary of outliers and stragglers detected throughout the study for
each combination for puff duration. Tables 3 and 4 then link this information to the related
smoking regime and lighting mode.
Table 2: Summary of outliers and stragglers for puff duration
Single test Double test Single test Double test
Dev.A_Lab.1 10 600 2 (20%)
Dev.A_Lab.2 10 600 6 (60%)
Dev.A_Lab.3 10 600 5 (50%)
Dev.A_Lab.4 10 600 6 (60%)
Dev.B_Lab.1 10 600 4 (40%)
Dev.B_Lab.2 10 600
Dev.B_Lab.3 10 600
Dev.B_Lab.4 10 600 1 (10%)
Dev.C_Lab.1 10 600 1 (10%)
Dev.C_Lab.2 10 600 2 (20%)
Dev.C_Lab.3 10 600
Dev.C_Lab.4 10 600
Dev.D_Lab.1 10 600 2 (20%)
Dev.D_Lab.2 10 600 4 (40%)
Dev.D_Lab.3 10 600
Dev.D_Lab.4 10 600 5 (50%)
Total 160 9600 1 (0.6%) 37 (23%)
Combination
Labs-Devices
Number of
datasets
analysed
STRAGGLER DATASET OUTLIER DATASET
GRUBBSCOCHRAN
GRUBBSCOCHRAN
Total number
of datapoints
analysed
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 12/45
Expressing the number of outliers from Table 2 as a percentage of the total number of 160
datasets, almost 1 out 4 (23%) datasets was detected as an outlier, as related to the within-
device laboratory variability (Cochran’s test). Details on how outliers are distributed across
the different smoking regimes and lighting modes are given in the two following tables.
Table 3: Distribution of stragglers for puff duration
Table 4: Distribution of outliers for puff duration
C: Cochran’s test, G: Grubbs’ simple test (_h:high and _l:low),
G2: Grubbs’ double test (_h:high and _l:low), Gi: Grubbs’ test on individual values.
In some cases, where the “Gi” label occurs in the table, it means that a few of the individual
results were removed because they were identified as outliers. As the rest of the dataset was
still considered in the estimation of the intra-lab-device variability and the inter-lab-device
variability, the “Gi” status is not mentioned in the summary table (Table 2).
LIT_SR1 LIT_SR2 LIT_SR3 LIT_SR4 LIT_SR5 UNLIT_SR1 UNLIT_SR2 UNLIT_SR3 UNLIT_SR4 UNLIT_SR5
Dev.A_Lab.1
Dev.A_Lab.2
Dev.A_Lab.3
Dev.A_Lab.4
Dev.B_Lab.1
Dev.B_Lab.2
Dev.B_Lab.3
Dev.B_Lab.4 C
Dev.C_Lab.1
Dev.C_Lab.2
Dev.C_Lab.3
Dev.C_Lab.4
Dev.D_Lab.1
Dev.D_Lab.2
Dev.D_Lab.3
Dev.D_Lab.4
Combination
Labs-Devices
PUFF DURATION - STRAGGLER
LIT_SR1 LIT_SR2 LIT_SR3 LIT_SR4 LIT_SR5 UNLIT_SR1 UNLIT_SR2 UNLIT_SR3 UNLIT_SR4 UNLIT_SR5
Dev.A_Lab.1 Gi Gi Gi C Gi C Gi Gi
Dev.A_Lab.2 C Gi Gi Gi C C C Gi C C
Dev.A_Lab.3 C C C Gi C C
Dev.A_Lab.4 C C Gi C C Gi C C
Dev.B_Lab.1 C C C C
Dev.B_Lab.2
Dev.B_Lab.3
Dev.B_Lab.4 Gi Gi Gi Gi Gi
Dev.C_Lab.1 C
Dev.C_Lab.2 C C
Dev.C_Lab.3
Dev.C_Lab.4
Dev.D_Lab.1 C C
Dev.D_Lab.2 C C Gi C C
Dev.D_Lab.3 Gi
Dev.D_Lab.4 C C C C C
Combination
Labs-Devices
PUFF DURATION - OUTLIER
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 13/45
4.1.2. Puff volume
Table 5 provides the summary of outliers and stragglers detected throughout the study for
each combination for the puff volume. Tables 6 and 7 then link this information to the related
smoking regime and lighting mode.
Table 5: Summary of outliers and stragglers for puff volume
Single test Double test Single test Double test
Dev.A_Lab.1 10 600
Dev.A_Lab.2 10 600
Dev.A_Lab.3 10 600
Dev.A_Lab.4 10 600 2 (20%)
Dev.B_Lab.1 10 600 10 (100%)
Dev.B_Lab.2 10 600 1 (10%) 3 (30%) 5 (50%)
Dev.B_Lab.3 10 600 5 (50%)
Dev.B_Lab.4 10 600 1 (10%) 3 (30%) 6 (60%)
Dev.C_Lab.1 10 600 2 (20%)
Dev.C_Lab.2 10 600
Dev.C_Lab.3 10 600
Dev.C_Lab.4 10 600
Dev.D_Lab.1 10 600 7 (70%)
Dev.D_Lab.2 10 600
Dev.D_Lab.3 10 600 5 (50%)
Dev.D_Lab.4 10 600 1 (10%)
Total 160 9600 41 (26%)
Combination
Labs-Devices
Number of
datasets
analysed
STRAGGLER DATASET OUTLIER DATASET
GRUBBSCOCHRAN
GRUBBSCOCHRAN
Number of
datapoints
analysed
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 14/45
Expressing the number of outliers from Table 5 as a percentage of the total number of 160
datasets, more than 1 out 4 (26%) datasets was detected as an outlier related to the within-
device laboratory variability (Cochran’s test). Details on how outliers are distributed across
the different smoking regimes and lighting modes are given in the two following tables.
Table 6: Distribution of stragglers for puff volume
Table 7: Distribution of outliers for puff volume
C: Cochran’s test, G: Grubbs’ simple test (_h:high and _l:low),
G2: Grubbs’ double test (_h:high and _l:low), Gi: Grubbs’ test on individual values.
In some cases, where the “Gi” label occurs in the table, it means that a few of the individual
results were removed because they were identified as outliers. As the rest of the dataset was
still considered in the estimation of the intra-lab-device variability and the inter-lab-device
variability, the “Gi” status is not mentioned in the summary table (Table 2).
LIT_SR1 LIT_SR2 LIT_SR3 LIT_SR4 LIT_SR5 UNLIT_SR1 UNLIT_SR2 UNLIT_SR3 UNLIT_SR4 UNLIT_SR5
Dev.A_Lab.1
Dev.A_Lab.2
Dev.A_Lab.3
Dev.A_Lab.4 G2_h G2_h
Dev.B_Lab.1
Dev.B_Lab.2 G2_h G2_h G_h, G2_h
Dev.B_Lab.3
Dev.B_Lab.4 G_h, G2_h G2_h G2_h
Dev.C_Lab.1
Dev.C_Lab.2
Dev.C_Lab.3
Dev.C_Lab.4
Dev.D_Lab.1
Dev.D_Lab.2
Dev.D_Lab.3
Dev.D_Lab.4
Combination
Labs-Devices
PUFF VOLUME - STRAGGLER
LIT_SR1 LIT_SR2 LIT_SR3 LIT_SR4 LIT_SR5 UNLIT_SR1 UNLIT_SR2 UNLIT_SR3 UNLIT_SR4 UNLIT_SR5
Dev.A_Lab.1
Dev.A_Lab.2
Dev.A_Lab.3
Dev.A_Lab.4
Dev.B_Lab.1 C C C C C C C C C C
Dev.B_Lab.2 C C C C C
Dev.B_Lab.3 C C C C C
Dev.B_Lab.4 C C C C C C
Dev.C_Lab.1 C C
Dev.C_Lab.2
Dev.C_Lab.3
Dev.C_Lab.4
Dev.D_Lab.1 C C C C C C C
Dev.D_Lab.2
Dev.D_Lab.3 C C C C C
Dev.D_Lab.4 C
Combination
Labs-Devices
PUFF VOLUME - OUTLIER
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 15/45
4.2. Intra and inter-lab-device variability estimations
The intra-lab-device variability limits (r*) and the inter-lab-device variability limits (R*) are
calculated for each smoking regime, for each mode (lit and unlit) and for each parameter (puff
duration and puff volume).
4.2.1 Puff duration
Estimations for puff duration are summarized in Table 8 by keeping the outlier dataset
(combination laboratory-device) and in Table 9 by removing the outlier dataset. The
corresponding graphs named “individual values” are shown in Appendix C of this report.
Table 8: r* and R* estimations for puff duration with outlier dataset
Table 9: r* and R* estimations for puff duration without outlier dataset
Both tables demonstrate the benefit of the outlier detection, since the range of CVr* changes
from [17.2% ; 41.8%] to [3.9% ; 12.3%] and the range of CVR* changes from [23.9% ;
57.4%] to [13.6% ; 32.3%].
Considering the range of the inter-lab-device variability limits computed after outlier removal,
the Acceptability Interval () was set at a conservative 30% to build the accuracy profile later.
Mode Regime Number
combination
labs-devices
Mean r* R* CVr* CVR*
LIT SR1 16 3.19 0.549 0.760 17.2% 23.9%
LIT SR2 16 3.76 1.361 1.531 36.2% 40.7%
LIT SR3 16 2.93 0.521 0.691 17.8% 23.6%
LIT SR4 16 2.09 0.688 0.855 33.0% 41.0%
LIT SR5 16 1.07 0.318 0.499 29.6% 46.5%
UNLIT SR1 16 3.18 0.695 0.869 21.9% 27.3%
UNLIT SR2 16 3.74 1.035 1.171 27.7% 31.3%
UNLIT SR3 16 2.92 0.905 1.035 31.0% 35.5%
UNLIT SR4 16 2.04 0.436 0.601 21.3% 29.5%
UNLIT SR5 16 1.08 0.453 0.621 41.8% 57.4%
Puff duration [s]
Mode Regime Number
combination
labs-devices
Mean r* R* CVr* CVR*
LIT SR1 12 3.15 0.126 0.553 4.0% 17.6%
LIT SR2 12 3.74 0.298 0.565 8.0% 15.1%
LIT SR3 12 2.90 0.342 0.529 11.8% 18.2%
LIT SR4 15 2.04 0.245 0.420 12.0% 20.6%
LIT SR5 12 1.01 0.124 0.267 12.3% 26.4%
UNLIT SR1 12 3.17 0.123 0.599 3.9% 18.9%
UNLIT SR2 12 3.72 0.179 0.507 4.8% 13.6%
UNLIT SR3 12 2.91 0.215 0.500 7.4% 17.2%
UNLIT SR4 12 2.01 0.124 0.363 6.2% 18.1%
UNLIT SR5 12 1.03 0.108 0.332 10.5% 32.3%
Puff duration [s]
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 16/45
4.2.2. Puff volume
Estimations for puff volume are summarized in Table 10 by keeping the outlier dataset
(combination laboratory-device) and in Table 11 by removing the outlier dataset. The
corresponding graphs named “individual values” are shown in Appendix C of this report.
Table 10: r*and R* estimations for puff volume with outlier dataset
Table 11: r* and R* estimations for puff volume without outlier dataset
Both tables demonstrate the benefit of the outlier detection, since the range of CVr* changes
from [11.0% ; 23.9%] to [4.3% ; 12.7%] and the range of CVR* changes from [21.0% ;
32.8%] to [9.7% ; 18.2%].
Considering the range of the inter-lab-device variability limits computed after outlier removal,
the Acceptability Interval () was set at 20% to build the accuracy profile later.
Mode Regime Number
combination
labs-devices
Mean r* R* CVr* CVR*
LIT SR1 16 36.9 4.80 10.23 13.0% 27.7%
LIT SR2 16 82.9 18.61 23.38 22.5% 28.2%
LIT SR3 16 93.9 14.07 19.74 15.0% 21.0%
LIT SR4 16 82.2 14.22 18.20 17.3% 22.1%
LIT SR5 16 51.5 11.63 16.86 22.6% 32.8%
UNLIT SR1 16 36.3 4.02 9.05 11.0% 24.9%
UNLIT SR2 16 80.7 15.78 20.74 19.6% 25.7%
UNLIT SR3 16 89.7 18.92 23.53 21.1% 26.2%
UNLIT SR4 16 79.9 19.06 24.08 23.9% 30.1%
UNLIT SR5 16 50.9 9.45 14.25 18.6% 28.0%
Puff volume [ml]
Mode Regime Number
combination
labs-devices
Mean r* R* CVr* CVR*
LIT SR1 12 35.3 2.01 4.56 5.7% 12.9%
LIT SR2 12 82.3 9.34 15.00 11.4% 18.2%
LIT SR3 12 91.6 11.63 14.53 12.7% 15.9%
LIT SR4 12 82.7 8.35 11.89 10.1% 14.4%
LIT SR5 12 51.6 2.68 7.41 5.2% 14.4%
UNLIT SR1 12 34.9 2.09 4.26 6.0% 12.2%
UNLIT SR2 12 81.6 3.63 11.25 4.4% 13.8%
UNLIT SR3 12 91.1 5.62 12.60 6.2% 13.8%
UNLIT SR4 12 80.9 4.10 9.36 5.1% 11.6%
UNLIT SR5 12 52.1 2.24 5.04 4.3% 9.7%
Puff volume [ml]
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 17/45
4.3. Intra and inter-lab-device variability modeling
We have investigated whether there is a relationship between the average for results and the
precision (r* and R*).
4.3.1. Puff duration
Figure 7 shows the power relationships established between the average puff duration and the
associated standard deviations of precision (Sr* and SR*).
Figure 7: Power relationships for puff duration and the associated Sr* and SR*
Sr* and SR* increase more or less linearly when puff duration increases. However, the
relationships between Sr* and puff duration are not good enough (R² lower than 0.5).
Improvements of model quality are shown in Figure 8, where non-linear relationships are
established between Sr* or SR* and the average flow.
Puff duration average (s)
Sr
or
SR
(s)
1.0 1.5 2.0 2.5 3.0 3.5 4.0
0.00
0.05
0.10
0.15
0.20
0.25 Sr LIT 0.0483x0.497 R² 0.294 Sr UNLIT 0.0369x0.3843 R² 0.465
SR LIT 0.0951x0.6071
R² 0.986 SR UNLIT 0.1109x0.4218
R² 0.766
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 18/45
Figure 8: Non-linear relationships for average flow and Sr* and SR* of puff duration
Considering results shown in Figure 8, the intra and inter-lab-device variabilities are non-
linearly dependent on average flow rate.
4.3.2. Puff volume
Figure 9 shows the power relationships established between the average puff volume and the
associated standard deviations of precision (Sr* and SR*).
Figure 9: Power relationships for puff volume and the associated Sr* and SR*
Flow rate average (ml/s)
Sr
or
SR
(s)
10 20 30 40 50 60
0.00
0.05
0.10
0.15
0.20
0.25 Sr LIT -0.0538 + 0.011x -2e-04x2 R² 0.964 Sr UNLIT 0.0112 + 0.0039x -1e-04x2 R² 0.708
SR LIT 0.1663 + 0.0038x -1e-04x2
R² 0.998 SR UNLIT 0.2372 -0.0023x -2e-06x2
R² 0.932
3.5s 4s 3s 2s 1s
Puff volume average (ml)
Sr
or
SR
(m
l)
30 40 50 60 70 80 90 100
0
1
2
3
4
5
6 Sr LIT 6e-04x1.9184
R² 0.953 Sr UNLIT 0.0223x0.9509
R² 0.844
SR LIT 0.0199x1.2356
R² 0.971 SR UNLIT 0.0214x1.1666
R² 0.928
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 19/45
Sr* and SR* increase more or less linearly when puff volume increases. Variability obtained
on LIT cigarettes is higher than that obtained on UNLIT cigarettes. Improvements of model
quality are shown in Figure 10, where non-linear relationships are established between Sr*
and SR* and the average flow.
Figure 10: Non-linear relationships for average flow and Sr* and SR* of puff volume
Considering results shown in the Figure 10, the intra and inter-lab-device variabilities are
non-linearly dependent on average flow rate.
4.4. Desirability indices
As described previously, the Desirability Index gives the device ‘fitness for purpose’ within
the limits of inter-lab-device variability when it is used within a given laboratory.
Consequently, for each of the two assessed parameters, 64 accuracy profiles were computed
for the 64 tested configurations (four devices * four laboratories * two modes * two samples).
Figures 11 and 12 summarize the computed Desirability Indices, distributed on a scale from
0% to 100%. Indices are categorized into three classes: 1) “Unsatisfactory” for indices below
60%, 2) “Questionable” for indices between 60% and 80%, and 3) “Fit for purpose” for
indices greater than 80%.
Additionally, the right margin provides the device code information (A-B-C-D) as well as the
laboratory code information (1-2-3-4). The left margin provides the number of defective
profiles.
Appendix D provides the complete Table of Indices.
Flow rate average (ml/s)
Sr
or
SR
(m
l)
10 20 30 40 50 60
0
1
2
3
4
5
6 Sr LIT -3.7528 + 0.4915x -0.0079x2
R² 0.985 Sr UNLIT -0.7841 + 0.1608x -0.0026x2
R² 0.881
SR LIT -2.777 + 0.5101x -0.008x2
R² 0.902 SR UNLIT -2.2137 + 0.4241x -0.0069x2
R² 0.964
35ml 80ml 90ml 80ml 50ml
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 20/45
4.4.1. Puff duration
Considering the range of inter-lab-device variability limits computed after outlier removal, the
Acceptability Interval () was set at 30% for puff duration. Desirability Indices are
summarized in Figure 11.
Figure 11: Desirability indices for puff duration
Device A:
Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.
Indeed, indices vary from 66% to 89% and two datasets were defective (from laboratories 1
and 4).
Device B:
Desirability Indices are systematically spread in the “Fit for purpose” area. Indeed, indices
vary from 86% to 97% and four datasets were defective (from laboratory 1).
Device C:
Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.
Indeed, indices vary from 69% to 95% and one dataset was defective (from laboratory 1).
Device D:
Desirability Indices are systematically spread in the “Fit for purpose” area. Indeed, indices
vary from 85% to 98%.
Unsatisfactory Questionable Fit for purpose
A
B
C
D
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
Nb=7 Nb=0 Nb=0 Nb=10 Nb=47
# D
efe
ctive
pro
file
#1
#1
#4
#1
0% 20% 40% 60% 80% 100%
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 21/45
4.4.2. Puff volume
Considering the range of inter-lab-device variability limits computed after outlier removal, the
Acceptability Interval () was set at 20% for puff volume. Desirability Indices are
summarized in Figure 12.
Figure 12: Desirability indices for puff volume
Device A:
Desirability Indices are systematically spread in the “Fit for purpose” area. Indeed, indices
vary from 83% to 95%.
Device B:
Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.
Indeed, indices vary from 69% to 90% and eight datasets were defective (from laboratories 1,
2, 3 and 4).
Device C:
Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.
Indeed, indices vary from 76% to 96% and one dataset was defective (from laboratory 1).
Device D:
Desirability Indices are spread between the “Questionable” and “Fit for purpose” areas.
Indeed, indices vary from 75% to 97% and four datasets were defective (from laboratory 1).
Unsatisfactory Questionable Fit for purpose
A
B
C
D
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
Nb=13 Nb=0 Nb=0 Nb=8 Nb=43
# D
efe
ctive
pro
file
#4#1#1#2
#1
#4
0% 20% 40% 60% 80% 100%
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 22/45
5. COMMENTS AND CONCLUSIONS
This study resulted in estimations of the intra-lab-device variability limits (r*) and the inter-
lab-device variability limits (R*) for the two parameters of puff duration and puff volume.
The study has been completed by using an accuracy profile per device and laboratory.
5.1. General comments
For puff duration and volume, non-linear relationships were observed between the flow rate
and the intra- and inter-lab-device variabilities.
Laboratory 1 reported some data containing odd values, leading to outlier removal and to
defective accuracy profiles.
It should be noted that the collaborative study took much longer than expected to complete,
starting in April 2013 and finishing in September 2015. As a reminder, during this period the
same device set was circulated to each of the four participating laboratories. None of the
devices or associated equipment were serviced for the duration of the study. This may have
affected the performance of some devices for the later laboratories that participated.
5.2. Puff duration
Coefficients of variation of the intra-lab-device variability CVr* (r* expressed as a percentage
of the overall mean) of puff duration ranged from 4% to 12% regardless of the cigarette
sample (West White or Camel Blue) and the mode (Lit or Unlit). However, the coefficients of
variation of the inter-lab-device variability ranged from 15% to 32%.
The Desirability Indices of non-defective accuracy profiles (57 of 64) ranged from 66% to
98%. Two devices, B and D, provided better Desirability Indices (from 85% to 98%) than A
and C (from 66% to 95%).
Most of the laboratories showed significantly higher puff duration variabilities for device A
than for the other devices.
Laboratory 1 reported data containing odd values, leading to outlier removal and to defective
accuracy profiles.
5.3. Puff volume
Coefficients of variation of the intra-lab-device variability CVr* ranged from 4% to 13%,
which is similar to those obtained for puff duration. However, the coefficients of variation of
the inter-lab-device variability for puff volume ranged from 10% to 18%, which is
significantly lower than the range obtained for puff duration.
The Desirability Indices of non-defective accuracy profiles (51 of 64) ranged from 69% to
97%. Device A provided the best Desirability Indices (from 83% to 95%), followed by C and
D (from 75% to 97%), and then device B (from 69% to 90%, but with a high percentage
[50%] of defective accuracy profiles).
Most of the laboratories showed significantly higher puff volume variabilities for device B
than for the other devices.
The high variabilities reported for device D seemed to be the result of odd data obtained with
one of the three units used.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 23/45
APPENDIX A – List of participating laboratories and device
manufacturers
Laboratories and devices are listed below in alphabetic order. The codifications used in this
report were randomly assigned to laboratories and devices.
Laboratories Country
BAT United Kingdom
ITG France
RJRT USA
SODIM France
Device Manufacturer
CME-P4 RJRT
Cressmicro BORGWALDT
SA7 BAT
SPA-M SODIM
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 24/45
APPENDIX B – Part 2 - Intra and inter-lab-device variability
estimations
B.1 Puff duration
Estimations for puff duration are summarized in Table B.1.1 by keeping outlier dataset
(combination laboratory-device) and in Table B.1.2 by removing outlier dataset.
Table B.1.1: r* and R* estimations for puff duration with outlier dataset
Table B.1.2: r* and R* estimations for puff duration without outlier dataset
B.2 Puff Volume
Estimations for puff volume are summarized in Table B.2.1 by keeping outlier dataset
(combination laboratory-device) and in Table B.2.2 by removing outlier dataset.
Table B.2.1: r* and R* estimations for puff volume with outlier dataset
Table B.2.2: r* and R* estimations for puff volume without outlier dataset
Mode Regime Nb combination
labo-device
Mean r* R* CVr* CVR*
LIT SR3 16 2.96 1.452 1.645 49.0% 55.5%
UNLIT SR3 16 2.94 1.585 1.720 53.9% 58.5%
Puff duration [s]
Mode Regime Nb combination
labo-device
Mean r* R* CVr* CVR*
LIT SR3 12 2.85 0.273 0.445 9.6% 15.6%
UNLIT SR3 12 2.83 0.186 0.456 6.6% 16.1%
Puff duration [s]
Mode Regime Nb combination
labo-device
Mean r* R* CVr* CVR*
LIT SR3 16 92.66 21.627 28.035 23.3% 30.3%
UNLIT SR3 16 89.38 18.840 23.860 21.1% 26.7%
Puff volume [ml]
Mode Regime Nb combination
labo-device
Mean r* R* CVr* CVR*
LIT SR3 12 92.72 11.039 16.647 11.9% 18.0%
UNLIT SR3 12 90.47 5.508 11.597 6.1% 12.8%
Puff volume [ml]
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 25/45
APPENDIX C – Data representation
The following figures show the raw data for the five smoking regimes, the two lighting modes
and the two reported parameters. The plots indicate mean values with 95% confidence
interval (CI) highlighted in green (for valid laboratories) or in red (for laboratories detected as
an outlier). Laboratories excluded by outlier statistics are denoted by the name of the test
rejecting the laboratory (G for Grubbs, G2 for double Grubbs or C for Cochran). The
confidence limits at 95% and 99% for the difference between the reference value (average of
all “valid” laboratories) and the average of one laboratory are plotted in green and red,
respectively.
Codes for participating laboratories that did not provide data are marked with an asterisk.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 26/45
C.1 Puff duration
Smoking regime 1
Figure 13: Individual observations of Puff Duration in LIT mode
Figure 14: Individual observations of Puff Duration in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 27/45
Smoking regime 2
Figure 15: Individual observations of Puff Duration in LIT mode
Figure 16: Individual observations of Puff Duration in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 28/45
Smoking regime 3
Figure 17: Individual observations of Puff Duration in LIT mode
Figure 18: Individual observations of Puff Duration in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 29/45
Smoking regime 4
Figure 19: Individual observations of Puff Duration in LIT mode
Figure 20: Individual observations of Puff Duration in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 30/45
Smoking regime 5
Figure 21: Individual observations of Puff Duration in LIT mode
Figure 22: Individual observations of Puff Duration in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 31/45
C.2 Puff Volume
Smoking regime 1
Figure 23: Individual observations of Puff Volume in LIT mode
Figure 24: Individual observations of Puff Volume in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 32/45
Smoking regime 2
Figure 25: Individual observations of Puff Volume in LIT mode
Figure 26: Individual observations of Puff Volume in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 33/45
Smoking regime 3
Figure 27: Individual observations of Puff Volume in LIT mode
Figure 28: Individual observations of Puff Volume in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 34/45
Smoking regime 4
Figure 29: Individual observations of Puff Volume in LIT mode
Figure 30: Individual observations of Puff Volume in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 35/45
Smoking regime 5
Figure 31: Individual observations of Puff Volume in LIT mode
Figure 32: Individual observations of Puff Volume in UNLIT mode
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 36/45
APPENDIX D – Accuracy profile results
Puff Duration: complete results of the computed accuracy profiles.
Device.code Labo.code Lighting Sample Beta Lambda Accuracy_Index Range_Index Precision_Index Trueness_Index
A 1 LIT CB 90 30 0 0 0 0
A 1 LIT WW 90 30 76.5 73.26 64.07 95.36
A 1 UNLIT CB 90 30 74.31 69.3 61.36 96.48
A 1 UNLIT WW 90 30 81.61 84.33 67.93 94.9
A 2 LIT CB 90 30 79.79 78.79 68.34 94.33
A 2 LIT WW 90 30 66.35 39.66 77.57 94.92
A 2 UNLIT CB 90 30 86.85 87.17 77.6 96.84
A 2 UNLIT WW 90 30 69.18 46.4 75.56 94.45
A 3 LIT CB 90 30 74.66 61.86 68.67 97.98
A 3 LIT WW 90 30 84.6 83.48 77.24 93.91
A 3 UNLIT CB 90 30 80.82 72.1 78.95 92.74
A 3 UNLIT WW 90 30 82.52 75.62 79.02 94.02
A 4 LIT CB 90 30 0 0 0 0
A 4 LIT WW 90 30 86.38 84.56 79.96 95.31
A 4 UNLIT CB 90 30 87.84 94.85 73.43 97.31
A 4 UNLIT WW 90 30 89.09 90.43 81.44 96
B 1 LIT CB 90 30 0 0 0 0
B 1 LIT WW 90 30 0 0 0 0
B 1 UNLIT CB 90 30 0 0 0 0
B 1 UNLIT WW 90 30 0 0 0 0
B 2 LIT CB 90 30 86.51 100 71.68 90.34
B 2 LIT WW 90 30 91.92 100 91.12 85.23
B 2 UNLIT CB 90 30 86.47 100 73.26 88.27
B 2 UNLIT WW 90 30 91.91 100 91.35 84.99
B 3 LIT CB 90 30 97.05 100 95.32 95.91
B 3 LIT WW 90 30 95.72 100 94.51 92.81
B 3 UNLIT CB 90 30 96.77 100 94.97 95.41
B 3 UNLIT WW 90 30 96.16 100 95.92 92.7
B 4 LIT CB 90 30 93.7 100 90.92 90.48
B 4 LIT WW 90 30 93.9 100 96.12 86.13
B 4 UNLIT CB 90 30 93.7 100 92.43 89.02
B 4 UNLIT WW 90 30 94.12 100 97.34 85.66
C 1 LIT CB 90 30 79.68 100 52.32 96.68
C 1 LIT WW 90 30 92.98 100 84.15 95.51
C 1 UNLIT CB 90 30 0 0 0 0
C 1 UNLIT WW 90 30 92.87 100 84.18 95.15
C 2 LIT CB 90 30 69.39 100 36.42 91.74
C 2 LIT WW 90 30 89.39 100 82.21 86.89
C 2 UNLIT CB 90 30 74.65 100 46.24 89.95
C 2 UNLIT WW 90 30 87.42 100 77.16 86.59
C 3 LIT CB 90 30 91.33 100 78.98 96.44
C 3 LIT WW 90 30 95.07 100 89.62 95.88
C 3 UNLIT CB 90 30 93.11 100 84.02 96.09
C 3 UNLIT WW 90 30 94.71 100 88.8 95.67
C 4 LIT CB 90 30 75.19 100 46.91 90.62
C 4 LIT WW 90 30 87.84 100 78.12 86.75
C 4 UNLIT CB 90 30 82.66 100 63.31 89.2
C 4 UNLIT WW 90 30 88.57 100 80.12 86.73
D 1 LIT CB 90 30 88.21 100 71.24 96.35
D 1 LIT WW 90 30 97.44 100 93.23 99.24
D 1 UNLIT CB 90 30 96.45 100 91.02 98.58
D 1 UNLIT WW 90 30 97.67 100 93.86 99.26
D 2 LIT CB 90 30 85.8 100 72.27 87.41
D 2 LIT WW 90 30 93.86 100 91.94 89.92
D 2 UNLIT CB 90 30 92.43 100 88.49 89.23
D 2 UNLIT WW 90 30 91.43 100 84.8 90.14
D 3 LIT CB 90 30 89.03 100 72.51 97.32
D 3 LIT WW 90 30 98.23 100 96.51 98.21
D 3 UNLIT CB 90 30 97.36 100 94.07 98.1
D 3 UNLIT WW 90 30 98.41 100 96.89 98.38
D 4 LIT CB 90 30 85.39 100 73.43 84.79
D 4 LIT WW 90 30 92.46 100 88.24 89.58
D 4 UNLIT CB 90 30 93.95 100 93.84 88.38
D 4 UNLIT WW 90 30 92.79 100 89.05 89.71
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 37/45
Puff Volume: complete results of the computed accuracy profiles.
Device.code Labo.code Lighting Sample Beta Lambda Accuracy_Index Range_Index Precision_Index Trueness_Index
A 1 LIT CB 90 20 83.09 100 66.36 86.45
A 1 LIT WW 90 20 86.7 100 69.22 94.15
A 1 UNLIT CB 90 20 86.15 98.52 68.78 94.37
A 1 UNLIT WW 90 20 86.63 100 68.05 95.53
A 2 LIT CB 90 20 84.08 100 72.1 82.43
A 2 LIT WW 90 20 93.41 100 86.36 94.38
A 2 UNLIT CB 90 20 92.84 100 82.28 97.24
A 2 UNLIT WW 90 20 91.89 100 80.85 95.98
A 3 LIT CB 90 20 91 100 77.22 97.57
A 3 LIT WW 90 20 90.86 100 77.31 97.02
A 3 UNLIT CB 90 20 94.6 100 89.62 94.47
A 3 UNLIT WW 90 20 91.52 100 79.38 96.57
A 4 LIT CB 90 20 84.25 100 75.54 79.17
A 4 LIT WW 90 20 91.26 100 81.78 92.94
A 4 UNLIT CB 90 20 93.62 100 85.2 96.32
A 4 UNLIT WW 90 20 90.78 100 79.34 94.31
B 1 LIT CB 90 20 0 0 0 0
B 1 LIT WW 90 20 0 0 0 0
B 1 UNLIT CB 90 20 0 0 0 0
B 1 UNLIT WW 90 20 0 0 0 0
B 2 LIT CB 90 20 0 0 0 0
B 2 LIT WW 90 20 90.35 100 89.84 82.09
B 2 UNLIT CB 90 20 74.15 86.95 86.8 54.01
B 2 UNLIT WW 90 20 89.02 100 84.31 83.68
B 3 LIT CB 90 20 0 0 0 0
B 3 LIT WW 90 20 87.27 100 74.34 89.4
B 3 UNLIT CB 90 20 69.15 84.95 69.14 56.29
B 3 UNLIT WW 90 20 85.87 100 70.9 89.3
B 4 LIT CB 90 20 0 0 0 0
B 4 LIT WW 90 20 0 0 0 0
B 4 UNLIT CB 90 20 71.63 83.99 83.71 52.27
B 4 UNLIT WW 90 20 87.01 100 76.66 85.92
C 1 LIT CB 90 20 76.48 98.58 47.37 95.79
C 1 LIT WW 90 20 79.99 100 52.56 97.36
C 1 UNLIT CB 90 20 86.69 100 66.35 98.19
C 1 UNLIT WW 90 20 0 100 0 97.36
C 2 LIT CB 90 20 88.92 100 73.99 95.03
C 2 LIT WW 90 20 94.74 100 86.21 98.64
C 2 UNLIT CB 90 20 92.03 100 79.41 98.15
C 2 UNLIT WW 90 20 92.11 100 79.67 98.08
C 3 LIT CB 90 20 90.73 100 74.81 99.86
C 3 LIT WW 90 20 94.23 100 84.25 99.31
C 3 UNLIT CB 90 20 89.6 100 72.08 99.79
C 3 UNLIT WW 90 20 94.77 100 85.61 99.41
C 4 LIT CB 90 20 93.61 100 85.38 96.07
C 4 LIT WW 90 20 95.54 100 88.54 98.51
C 4 UNLIT CB 90 20 95.2 100 87.61 98.47
C 4 UNLIT WW 90 20 94.86 100 86.95 98.18
D 1 LIT CB 90 20 0 0 0 0
D 1 LIT WW 90 20 0 0 0 0
D 1 UNLIT CB 90 20 0 0 0 0
D 1 UNLIT WW 90 20 0 0 0 0
D 2 LIT CB 90 20 90.08 100 75.86 96.35
D 2 LIT WW 90 20 97.31 100 92.26 99.87
D 2 UNLIT CB 90 20 93.15 100 82.55 97.93
D 2 UNLIT WW 90 20 97.04 100 91.85 99.48
D 3 LIT CB 90 20 87.29 97.5 75.7 90.1
D 3 LIT WW 90 20 97.41 100 95.82 96.45
D 3 UNLIT CB 90 20 77.51 84.84 69.12 79.42
D 3 UNLIT WW 90 20 96.77 100 95.35 95.05
D 4 LIT CB 90 20 75.09 100 43.34 97.69
D 4 LIT WW 90 20 91.88 100 77.87 99.6
D 4 UNLIT CB 90 20 78.57 100 49.82 97.35
D 4 UNLIT WW 90 20 91.3 100 76.76 99.15
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 38/45
APPENDIX E – Experimental protocol
Study title
CORESTA collaborative study to evaluate the accuracy and precision of puffing topography
devices.
Background/Rationale
Commercial and proprietary puffing topography equipment have been available and used for
many years. However, there is limited information regarding validation of instruments used
for puffing topography. The FDA’s recent Draft Guidance for Applications for Premarket
Review of New Tobacco Products (PMTA) recommends studies to evaluate topography data
from adult smokers. This study has the advantage of evaluating several topography devices
under the same conditions, in different laboratories.
Objectives
This study will measure puff volume and puff duration with four different types of human
topography devices, using two commercial cigarettes that represent the low and higher end
pressure drops (PD) of commercial products. The cigarettes will be tested over various puff
volumes and durations in both unlit and lit machine smokings. The data collected will be used
to determine:
Accuracy: How close a measured value is to the true value (calibrated puff volume).
This will be done by taking 10 puffs on unlit cigarettes and 10 puffs on lit cigarettes
(unless a set butt length is reached) at each of five puffing regimes (Table 1).
Precision: How close the measured values of each unit are to each other (measured puff
volume). This is measured in terms of Repeatability and Reproducibility.
Repeatability: within device and within laboratory will be determined.
Reproducibility: within and between device type and within and between laboratories
will be determined.
Design
This study will assess four topography devices over a matrix of puff volumes and durations
that cover the range of these parameters found in typical cigarette smoking behaviour. It will
include measuring two different commercial cigarette brands with different pressure drops and
the cigarettes will be measured both when unlit and lit.
Each participating laboratory will be provided with five units of each of the four different types
of topography devices, although only three units of each type will be used in the study. Serial
numbers will be recorded to ensure that each laboratory uses the same subset of units.
Each device will be calibrated prior to use following provided instructions. A smoking
machine or puff generator (Borgwaldt A14 syringe driver) capable of drawing constant flow
rates will be used to generate the five puffing regimes (Table 1). The smoking machine will be
set up to produce the constant flow rates and duration conditions shown in Table 1. A bubble
meter will be used to confirm the puff volumes after changing smoking machine settings and
daily prior to testing. When confirming the puff volume, a standard pressure drop of
approximately 100mmWG will be used. Bubble meter readings will be recorded.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 39/45
Table 1: Puff Volume, Duration Matrix
Regime Puff
Profile
Puff Volume (mL)
Duration (s)
Inter-Puff Interval(s)
Flow Rate (mLs
-1)
1 Square 35 3.5 30 10
2 Square 80 4.0 30 20
3 Square 90 3.0 30 30
4 Square 80 2.0 30 40
5 Square 50 1.0 30 50
Each puff in the matrix above will be drawn 10 times (10 total puffs) through each of the three
units of the four different topography devices with the cigarette initially unlit. Following each
set of 10 unlit puffs, the cigarette will be lit using an electric lighter and 10 puffs will be drawn,
or if drawing 10 puffs would burn into the filter, puffs will be drawn through the cigarette until
the burning coal reaches a pre-determined butt length (overwrap +3 mm). This process will be
repeated for both commercial cigarette brands.
In addition, the 30 mLs-1
flow rate for 3.0 seconds duration, puff volume 90 mL smoking
regime will be repeated (x 10 lit & unlit puffs) with one of each of the four units over five
separate days, smoking both the West King Size and Camel King Size cigarettes. The same
unit should be used for the additional smoking on each day. The original day of smoking on
each device will be counted as the first of the five days.
Settings
This analysis will be performed in a laboratory at each of the participating companies.
Laboratory conditions will be controlled according to ISO 3308:
Temperature: 22°C +/- 2°C;
Humidity: 60%RH + / - 5%RH
The cigarettes will be conditioned prior to testing, according to ISO 4387:
Temperature: 22°C +/- 1°C;
Humidity: 60%RH + / - 3%RH
Cigarettes will be conditioned for at least 48 hours but no more than 10 days.
There is no requirement to record the airflow above the cigarettes.
Study equipment and supplies
Topography devices under investigation include:
1. CReSSmicro (Borgwaldt)
2. SPA-M (Sodim)
3. SA7 (BAT)
4. CME5-P4 (RJRT)
Five units of each of the above devices will be shipped to each laboratory taking part in the
study.
5 Carolina Medical Electronics, Inc.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 40/45
Cigarettes:
A sufficient quantity of each of the two commercial cigarettes will be supplied along with the
devices. The two commercial cigarettes are:
West White King Size Box (ITG)
Camel Blue King Size Box (RJRT)
Physical Data for:
West White King Size Box
o Open pressure drop (draft) - 45 mm of H20 (SD = 1.4)
o Closed pressure drop (draft) - 151 mm of H20 (SD = 11.0)
o Filter pressure drop (draft) - NA
Camel Blue King Size Box
o Open pressure drop (draft) - 130.5 mm of H20 (SD=7.5)
o Closed pressure drop (draft) - 157.1 mm of H20 (SD=9.3)
o Filter pressure drop (draft) - 98.0 mm of H20 (SD=6.5)
Smoking Machine :
A Borgwaldt A14 syringe driver (single-port smoking machine) or a linear, multi-port smoking
machine, with the capability of generating square puff profiles, will be used in each laboratory.
The Borgwaldt A14 syringe driver will be provided by BAT. The Cambridge filter pad will be
changed after smoking each six cigarettes in the lit condition.
Assessments
Ten measurements of
Puff volume (mL)
Puff duration (s)
Peak flow (mLs-1, where possible)
Resistance to draw (mLs-1-mmH2O, where possible)
Each puff in the matrix will be drawn through each of the three units of all four different
topography devices and recorded with the cigarette unlit first, followed by lighting the cigarette
with an electric lighter and puffing either 10 puffs or to a pre-determined butt length
(overwrap+3mm) if 10 puffs would result in burning into the filter.
This will be repeated sequentially using the second commercial cigarette.
In addition, the 30 mLs-1
flow rate for 3.0 seconds duration, puff volume 90 mL smoking
regime will be repeated (x 10 lit puffs only) with one of each of the four devices over five
separate days (same unit each day).
Operation
The laboratory operators will be trained on the usage of each type of topography device. A
user guide will be available for each device type.
Data collation
A standard template (Excel spreadsheet) will be provided to report the measurements.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 41/45
Data analysis
The evaluation of instruments will be based on a comparison of accuracy profiles.
Figure 1 gives an illustration of an accuracy profile with the key definitions.
Figure. 1: Illustration of an accuracy profile.
In this study, for each laboratory and each device, two accuracy profiles will be developed by
estimating the precision and the “trueness” 1) for each flow rate and 2) for each puff duration.
A specific device will meet the objective if the accuracy profile is within the acceptance limits.
The acceptance limits, for each studied parameter, will be defined by the CORESTA subgroup
members after a preliminary analysis of results.
An example of accuracy profiles is shown in Figure 2 (accuracy profiles based on the flow rate
to compare two instruments).
Figure. 2: Accuracy profiles based on the flow rate of devices 1 and 2.
To compare the accuracy profile of different devices, an indicator score will be calculated.
This score is the sum of the combination of three quality criteria:
Range Index
Precision Index
Trueness Index
bias (%)
parameter
+
-
C1 C2 C3 C4
RLinfRLsup
RANGE
mean relative bias
accep
tance
limits
0
bias limits of confidence
"Profil Exactitude : Modèle linéaire"
Concentrations
Re
co
uvre
me
nt
(%)
10 20 30 40 50
70
80
90
100
110
120
130
Device 1
Flow rate
Recovery
(%
)
"Profil Exactitude : Modèle linéaire"
Concentrations
Re
co
uvre
me
nt
(%)
10 20 30 40 50
70
80
90
100
110
120
130
Device 1
Flow rate
Recovery
(%
)
"Profil Exactitude : Modèle linéaire"
Concentrations
Re
co
uvre
me
nt
(%)
10 20 30 40 50
60
80
100
120
Device 2
Flow rate
Re
co
ve
ry (
%)
"Profil Exactitude : Modèle linéaire"
Concentrations
Re
co
uvre
me
nt
(%)
10 20 30 40 50
60
80
100
120
Device 2
Flow rate
Re
co
ve
ry (
%)
Device 2
Flow rate
Re
co
ve
ry (
%)
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 42/45
A score closer to 100 indicates that the device better met the objective. For example, in Table
2, Device 2 (Accuracy_Index = 78.67) is slightly better for this comparison than Device 1
(Accuracy_Index = 77.95).
Table 2: Accuracy index for devices 1 and 2
This statistical evaluation is proposed for the unlit cigarettes by considering a puff as a
replicate to estimate the repeatability (10 puffs), and an instrument unit as a replicate to
estimate the intermediate precision (3 units). The same evaluation will be made with the lit
cigarettes, with the exception that a variable number of puffs may be used (by smoking instead
to a set butt length), as required. See Appendix 2 for an explanation of terms used in the Data
Analysis section.
Appendix 1
Borgwaldt A14 Syringe driver entry guidelines
Setting Entry / Format Entry Example
Keyboard Control 2 2
Smoking 2 2
Puff Parameters 2 2
Bell profile? No No
Profile no. 1 1 = square puff
Correct? Yes Yes
Puff vol. (ml) ### 080
Correct? Yes Yes
Puff dur. (s) #,# 4,0 = 4.0
Correct? Yes Yes
Puff period (s) ## 30
Correct? Yes Yes
Puff vol. test? Yes Yes
Tester o.k.? Connect bubble meter prior to
selecting Yes
Yes
Puff volume test
Volume o.k.? Yes Yes
Accuracy_Index Range_Index Precision_Index Trueness_Index RL_Inf RL_Sup
Device 1 77.95 99.26 48.68 98.01 10.3 50.0
Device 2 78.67 93.1 54.23 96.46 12.8 50.0
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 43/45
Appendix 2
The relevance of method performance is defined by indicators6
Accuracy: closeness of agreement between a test result or measurement result and the true
value. Accuracy refers to a combination of trueness (common systematic error) and precision
(random error).
Trueness: closeness of agreement between the expected test result or measured result
and a true value.
Precision: closeness of agreement between independent test/measurement results
obtained under stipulated conditions.
-repeatability: precision under repeatability conditions.
-repeatability conditions: observation conditions where independent
test/measurement results are obtained with the same method on identical
test/measurement items in the same test or measurement facility by the same
operator using the same equipment within a short interval of time.
-intermediate precision: precision under intermediate precision conditions.
-intermediate precision conditions: conditions where test results or
measurement results are obtained with the same method, on identical
test/measurement items in the same test or measurement facility, under some
different operating condition.
6 ISO 15725-1
TruenessTrueness
Accuracy
=
Trueness + precision
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 44/45
APPENDIX F – Departures from experimental protocol
No departures from the experimental protocol were reported.
TSB-047-CTR Puffing Topography Inter-lab Study – January 2017 45/45
APPENDIX G – Definitions
Repeatability r:
The difference between two single results found on matched cigarette samples by one operator
using the same apparatus within the shortest feasible time interval will exceed the repeatability
value ( r ) on average not more than once in 20 cases in the normal and correct operation of the
method.
ISO 10362-1:1991(E), page 3
Reproducibility R:
Single results on matched cigarette samples reported by two laboratories will differ by more
than the reproducibility value ( R ) on average not more than once in 20 cases in the normal
and correct operation of the method.
ISO 10362-1:1991(E), page 3
1. INTRODUCTION2. EXPERIMENTAL DESIGN2.1 Part 1: Intra and inter-lab-device variability2.2 Part 2: Time effect
3. STATISTICAL EVALUATION3.1. Uncertainty evaluation3.2. Accuracy profile3.3. Limitations
4. RESULTS4.1. Outlier results4.1.1. Puff duration4.1.2. Puff volume
4.2. Intra and inter-lab-device variability estimations4.2.1 Puff duration4.2.2. Puff volume
4.3. Intra and inter-lab-device variability modeling4.3.1. Puff duration4.3.2. Puff volume
4.4. Desirability indices4.4.1. Puff duration4.4.2. Puff volume
5. COMMENTS AND CONCLUSIONS5.1. General comments5.2. Puff duration5.3. Puff volume
APPENDIX A – List of participating laboratories and device manufacturersAPPENDIX B – Part 2 - Intra and inter-lab-device variability estimationsAPPENDIX C – Data representationAPPENDIX D – Accuracy profile resultsAPPENDIX E – Experimental protocolAPPENDIX F – Departures from experimental protocolAPPENDIX G – Definitions
Top Related