Validation / citations
Validation
Expert review of model structure Expert review of basic code implementation Reproduce original inputs Correctly respond to changes in parameters Consistent with other analyses Consistent across versions
Documentation specifications
Validation requires adequate documentation to… Describe clinical model Describe programming algorithms Describe inputs (sources / values /
data quality) Describe validation strategy
Citations
Basic rationale and structure -- Matchar et al., Annals of Internal Medicine (1997)
Bootstrapped sensitivity analysis -- Parmigiani et al., Medical Decision Making (1997)
Application to acute stroke trial -- Samsa et al., Journal of Clinical Epidemiology (in press)
Outline
Rationale for modeling Stroke model described Applying the SPM to a
randomized trial (*) Extensions
Applying the SPM to an RCT
Basic idea: After the short follow-up period typical of an RCT, we are likely to observe that the intervention induces small shifts in the distribution of morality / disability (measured at the conclusion of follow-up). Modeling can be used to account for the
effects of these shifts on long-term outcomes.
Disability measure
Disability could be measured by the Rankin Scale (RS), Barthel Index (BI), or a number of other instruments.
For concreteness, we use the RS with 5 categories. The analysis easily extends to other scales and/or cut-points.
Risk (hazard) ratios for disability on subsequent outcomes*
RS Utility h(IS) h(MI) h(DT) Cost 0-1 .80 1.00 1.00 1.00 1.00 2 .65 1.05 1.07 1.11 1.27 3 .50 1.12 1.14 1.27 1.94 4 .35 1.24 1.26 1.71 3.98 5 .20 1.30 1.31 2.37 6.01
*As derived by an expert panel, supported by published and unpublished literature
Rationale The utilities for RS of 0-1 and 5 were obtained
from the PORT’s patient survey.
• Other values obtained by interpolation• Consistent with unpublished RCT data
Increased hazard of death results from sequelae of inactivity such as aspiration pneumonia, etc.
Daily costs greatly increase when disability level implies high likelihood of institutionalization.
SPM results by Rankin*
RS Survival QALY Cost 0-1 10.60 6.07 143,820 2 9.99 4.70 167,602 3 9.28 3.37 217,039 4 8.22 2.13 328,895 5 7.09 1.09 403,911 *Beginning 6 months after IS, for patients
with mean age 70 years
Thought experiment -- typical RCT results (6 month follow-up)
Intervention Placebo QALY 0.25 0.23 Cost 27,000 24,000
That is, little difference in QALY in absolute terms. For cost, we assume equivalent utilization plus the cost of the drug.
Typical RCT results (cont’d)
RS Intervention Placebo 0-131% 25% 221% 15% 311% 10% 46% 10% 511% 15% Died20% 25%
Typical results (cont’d)
The previous results describe a “break-through drug”, but the ICER based upon 6 months of follow-up is only (27,000-24,000) / (0.25-0.23) = 150,000 $/QALY.
Combining short- and long-term outcomes
For each patient, we assign long-term outcomes (cost and QALYs) based upon RS at 6 months. For example, total costs then become 6-month costs (observed) + expected long-term costs (simulated).
Long-term QALYs by group
Intervention: (.31)(6.07) + (.21)(4.70) + (.11)(3.37) + (.06)(2.13) + (.11)(1.09) + (.20)(0) = 3.49 QALY
Placebo: (.25)(6.07) + (.15)(4.70) + (.10)(3.37) + (.10)(2.13) + (.15)(1.09) + (.25)(0) = 2.94 QALY– Identical multipliers -- groups only differ in
proportion of patients in each RS category– Same weighted average calculations for cost
Comprehensive CEA
Intervention QALY = 0.25+3.49 = 3.74 Intervention cost = 27,000+167,818 =
194,818 Placebo QALY = 0.23+2.94 = 3.17 Placebo cost = 24,000 + 176,275 = 200,275 ICER = (194,818-200,275)/(3.74-3.17) = -
9,573 $/QALY
Conclusion
By considering its long-term effects, the intervention moves from “not cost effective” to “cost saving,” even if the price of the drug is increased substantially. This result holds across a wide range of sensitivity analyses.
Outline
Rationale for modeling Stroke model described Applying the SPM to a
randomized trial Extensions (*)
Extensions
International applications Planning trials
International applications
Basic idea: Use the same SPM but change the inputs.
In theory, every component of the input data (e.g., natural history, effect of covariates, effect of interventions, QOL, cost) could differ by nation.
In practice, the biggest differences will involve utilization patterns (and thus cost).
Comment
The size of the task depends upon the difficulty in estimating the relevant parameters. This, in turn, depends what we are willing to
assume versus what will require additional data collection.
Once new estimates have been derived, inserting them into the SPM is
straightforward.
International costs
Costs are affected by Different utilization patternsDifferent unit pricesDifferent payment mechanisms /
perspectives (affecting which components of cost to include in the calculation)
Recommendation
Begin with US data on utilization and unit prices. Then use expert judgement (supported by the
literature as available) to posit any changes in utilization patterns.
Attach country-specific unit prices. Limit the analysis to the cost categories and time
periods of interest to decision-makers in that country.
Planning trials
The SPM can help to determine the (minimum) clinically important difference in short-term disability.
This, in turn, would be the basis for the trial’s sample size calculations.
Observation
Many stroke trials have been under-powered.
That is, the size of the effect which is significant from the perspective of public health / CEA is much smaller than that suggested by clinical intuition (Samsa et al, Journal of Clinical Epidemiology, in press).
Observation
Many RCTs of acute stroke treatments have had other sources of statistical inefficiency -- for example, pertaining to the choice of target population, the choice of measure, and so forth.
Summation
Because most benefits of stroke treatments accrue in the long-term, modeling is necessary.
The SPM is a well-validated model which can be helpful in both the planning and analysis of trials.
Planned extensions include international practice patterns and user-friendly software.
Top Related