06-35 Cost-effectiveness of imaging in newborns with suspected occult spinal dysraphism

2

Click here to load reader

Transcript of 06-35 Cost-effectiveness of imaging in newborns with suspected occult spinal dysraphism

Page 1: 06-35 Cost-effectiveness of imaging in newborns with suspected occult spinal dysraphism

amounts of time and effort from readers. Our overall objecti:ee is to en- hance participation by providing feedback after each case during the ex- periment. As a first step, we propose a statistical model based on item response theory.

Materials and Methods: In a ROC study, 18 radiology residents were asked to identify the presence or absence of injury in a set of 50 cervical spine radiographs. The data were analyzed using a linear model drawn from item response theory, which allows each case to be modeled with a different degree of difficulty. The model was fit- ted using Bayesian analysis.

Results: Statisttcally significant differences were found between many cases in the set. These differences correlated with the qualitative diffi- culty of the cases and established the statistical model's face validity.

Conclusion: Item response theory can be successfully applied to ana- lyze reader performance experiments. Accounting for case difficulty will enable detection of improvement when feedback is given after each case. Feedback adds educational value for readers and should signifi- cantly enhance their participation in performance experiments.

Dr. Eng receives support as a GERRAF fellow.

06-32 Radiology Reporting Evaluat ions by Academic and Community Pediatr ic ians Richard B. Gunderman, MD, PhD, Indiana University, Indianapolis, IN Purpose: To compare the radiology reporting needs and preferences of two types of pediatricians, one based m an academic children's hospital and the other in community-based practices.

Materials and Methods: A two-page reporting questionnaire was sent to academic pediatricians and 217 to community pediatricians. Re- sponse rates for the two groups were 70.3% and 55.3%. respectively.

Results: Community pediatricians are significantly less likely than academic pediatricians both to view their patients" diagnostic images themselves and to learn examination findings through a route other than the written report. Community pediatricians rate the overall ac- curacy of radiology reporting, the clarity of radiology reports, the re- sponsiveness of reports to clinical questions proposed in the referral, and overall satisfaction with reporting significantly lower than do the academic pediatricians.

Conclusions The written radiology report is an especially important sou¢ce of information for community pediatricians, and special atten- tion should be paid to report quality in that arena. This presentation highlights key strategies for doing so.

06-33 Measuring the Va lue of Diagnostic Imaging Tests Daniel Heller, MD, MS, UCSF-Mount-Zion Medical Center, San Francisco, CA To ensure new technology is dissemmated into clinical practice, radi- ologists must show that the value of added diagnostic information is worth more than the cost of obtaining the test. In this talk, an evalua- tion of imaging options for patients with endometrial cancer illustrates alternative methods to measure the value of imaging examinations. ROC analysis measures value in terms of test efficacy, while Bayesian analysis measures value in the context of disease prevalence. Both de- fine value in the abstract for the statistically "average" patient. A method of cost-effectiveness analysis is demonstrated that defines value within the context of a patient's specific circumstances. This method makes it possible to develop imaging guidelines that are pa- tient specific rather than population based.

06-34 Does Real T ime Appl icat ion of Medical Management Tools Alter Physician Test Ordering Behavior: Early Results on Basel ine Data Ramin Khorasani, MD, Brigham and Women's Hospital, Boston, MA Purpose: To determine radiology utilization rates and the accuracy of claims data to measure utifization rates for primary care physicians (PCPs).

Materials and Methods: The study is being conducted at Partners HealthCare System, Inc., in Boston, Massachusetts. This is the control phase of an experiment to determine whether real-time implementation of medical management tools changes physician test-ordering behavior (GERRAF 1999). All imaging orders are entered directly into the com- puter by PCPs or their office staff. Three physicians have ordered for 12 months, one for 8 months, and the remaining three physicians have used the ordering process for 4 months. We compared physician profiles from the order entry pro~am to claims data available for a comparable period of time for a capitated program.

Results: Total patient population was 8,785, with an average patient panel size per physician (n = 7; 2 males, 5 females) of 1,255, ranging from 492 to 2,738. The average patient age was 44.4 (range, 19 to 103) with 6,952 females (79%) and 1,833 males (21%). The overall radiol- ogy utilization rate was 501 per 1,000 laves per year (289 mammogra- phy, 71 US, 20 CT, 10 MR, 111 other), differing significantly among physicians (range: total 178 to 1,597, mammogaphy 20 to 932, US 25 to 341, CT 7 to 91, MRI 2 to 40, other 61 to 256). Utilization rates were not significantly different among payers (501 total, 494 for capitated, 491 for commercial, 596 for Medicare). Comparison to claims data for a 6-month period for a capitated patient population showed that claims data consistently overestimated utilization attributed to PCPs (27/94 = 29% total, 16/20 = 80% mammography, 5/46 = 11% US, 0/6 = 0% CT, 1/5 = 20% MR[, 5/17 = 29% other).

Conclusion: Utilization rates varied near ten-fold among PCPs. Claims data markedly overestimated actual PCP utilization rates, likely due to specialist orders being attributed to PCPs.

06-35 Cost .ef fect iveness of Imaging in Newborns with Suspected Occult Spinal Dysraphism L. Santiago Medina, MD, MPH, Children's Hospital Medical Center, Cincinnati, OH Purpose: To assess the chnical and econormc consequences of differ- ent diagnostic strategies in newborns with suspected occult spinal dysraphism.

Materials and Methods: A decision-analytic Markov model was constructed to compare MR imaging, ultrasound (US), plain radio- graphs (PR), and no imaging in newborns with suspected occult spinal dysraphism. Morbidity and mortality from early versus late diagnosis of dysraphism, pro~ession rates of disease, and MR imaging, US, and PR sensitivity and specificity were obtained from the literature. Cost estimates were obtained from the hospital cost accounting database and from the Medicaid fee schedule.

Results: In newborns with a pilonidal sinus or newborns of diabetic mothers (pretest probability of 0.3 %), ultrasound was associated with the greatest quality-adjusted life expectancy (QALE), followed by no- imaging, PR, and MR imaging strategies; QALE values were 29.091, 29.088, 29.079, and 29.077, respectively. MR imaging resulted in the greatest lifetime treatment cost ($1,832), followed by PR ($1,568), US ($311), and no imaging ($150) strategies. The results were robust over a wide range of disease progression rates and procedure and treatment costs. Incremental cost-effectiveness ratios showed US being cost-ef- fective, but the choice between US and no imaging was dependent on

1045

Page 2: 06-35 Cost-effectiveness of imaging in newborns with suspected occult spinal dysraphism

the accuracy of ultrasound and quality adjustment of the life expect- ancy. In newborns with a lumbosacral dimple (pretest probability of 3.8%), US was cost-saving. Sensitivity analysis demonstrated that MR imaging was the preferred strategy if the specificity and sensitivity of US were below 50% and 77%, respectively. Conclusion: Ultrasound was cost-effective in studying newborns with low risk of occult spinal dysraphism. However, if only life ex- pectancy (without quality adjustment) is considered, no imaging is the preferred strategy in newborns with a pilonidal sinus or new- borns of diabetic mothers.

06-36 Physician Reliabil i ty in the Interpretat ion of Cranial Computed Tomography in Stroke Pat ients Eligible for Thrombolyt ic Therapy Katie D. Vo, MD, Mallinckrodt Institute of Radiology, Saint Louis, MO, Daniel K. Kido, MD, Chung Y. Hsu, MD, PhD, Benjamin Littenherg, MD

Background and Purpose: Reliability assessment of a diagnostic test is an important but seldom-examined component of diagnostic technology assessment. Observer variation can be substantial and can lead to misdi- agnosis even when a test is very accurate. In this study, we assess physi- cian reliabifity in the interpretation of cramal CT scans of patients with hyperacute stroke. The hyperacute CT scan is used as a test model be- cause it is a simple and widely available modality and is an important screening tool for the assessment of patients with acute neurologic defi- cits for possible thrombolytic therapy.

Materials and Methods: Twenty baseline CT scans from patients who presented with acute neurologic deficits within 3 hours of onset were se- lected from our institution's stroke log. These scans had a broad range of interpretive difficulty and pathology, including 7 clear-cut early signs of infarct, 5 subtle early signs of infarct, 3 dense MCA signs, 1 obvious hemorrhage, 2 subtle hemorrhages (identified by MRI), 1 tumor, and 6 normal scans. The scans were evaluated by 13 physicians, including 4 staff neuroradiologists, 6 neuroradiology fellows, and 3 stroke neurolo- gists. The findings were classified into five categories for the presence or absence of early signs of ischemia, hemorrhage, dense MCA sign, other nonischemic cause for patient' s deficits, and contraindications for tPA. For each CT finding, ~ statistics were used to assess proportion of agreement among the readers.

Results: Kappa values for all five categories ranged from 0.21-0.48 for neuroradiology fellows, 0.15-0.45 for neuroradiology staff, 0.27-0.56 for the stroke neurologists, and 0.23-0.44 for all readers as a group. With regard to the presence or absence of hemorrhage, the ~: values were 0.28 for the neuroradiology staff, 0.48 for neuroradiology fellows. 0.56 for stroke neurologists, and 0.43 for all readers. For early CT find- ings of ischemia, the 1¢ values were 0.27 for the neuroradiology staff, 0.438 for the neuroradiology fellows, 0.33 for the stroke neurologists, and 0.37 for all readers.

Conelnsion: When taking the base rate into consideratmn, there was excellent agreement in discerning hemorrhage and poor agreement in recognizing early CT signs of ischemia within each specialty group and between groups.

07-37 Is Training in Conscious Sedation Adequate for Radi- ology Residents? Charles M. Heaton, MD, Medical University of South Carolina, Charleston, SC, LeoNe Gordon, MD, Jeanne G. Hill, MD

Purpose: To evaluate the level of resident responsibility and training in conducting conscious sedation procedures and to quantify and catego- rize the average monthly studies requiring resident-directed conscious sedation, relative to both the university and private program settings.

Materials and Methods: A survey was sent to the program direc- tors of over 200 diagnostic radiology residency programs accredited by the Accreditation Council for Continuing Graduate Medical Edu- cation. Results were placed into a database and analyzed.

Results: Responses were received from 72 programs (46 university, 16 private, 10 both/other). Forty-four programs (32 university, 6 pri- vate, 6 both/other) require residents to provide sedation for proce- dures. Resident supervision ranged from direct faculty oversight to complete independence. Only thirteen required specific training in techniques. Training varied widely (e.g., formal 103-hour course, single one-hour videotape and quiz, "self-training"). The average num- ber of monthly studies requiring sedation was 142 for all programs re- sponding (8107 studies from 57 programs providing numbers).

Conclusion: Present radiology residency curriculum provides minimal training in conscious sedation despite it being a large, integral part of everyday practice, with increasing pubhc awareness of poor outcomes. Formal curriculmn changes to emphasize training and competency may be indicated.

07-38 The Case-Based Teaching Radiology Conference for Residents: Benef icial Effect of Previewing and Using Answer Sheets Felix S. Chew, MD, Wake Forest University School of Medicine, Winston-Salem, NC Purpose: Radiology residents often experience the case-based teach- ing conference as an inquisition in which the moderator painfully ex- tracts observations, conclusions, and facts from a discussant while other attendees passively observe. This is frequently disliked by all par- ticipants. We hypothesized that such conferences could be improved by previewing cases and using answer sheets.

Materials and Methods: A monthly one-hour case-based skeletal radi- ology teaching conference was modified so that residents previewed 20 single-image cases for 45 seconds each while completing answer sheets. Directed by a moderator, residents then took turns discussing their re- sponses. Attendees completed evaluation forms.

Results: Five conferences were evaluated, and a total of 81 evaluations were received. The average response rate per conference was 90%. The evaluations indicated that the content was appropriate (96%), the format helped learning (98%), the new format was preferred to the traditional format (98%), and more such conferences were desired (99%). Evalua- tions also suggested that the requirement to commit to a diagnosis was beneficial, that there were greater levels of participation and engage- ment by all the attendees, and a greater number of cases were discussed.

Conclusion: Modifying case-based radiology teaching conferences by having participants preview cases and use answer sheets has positive educational benefits and is well received.

07-39 "All Pancreas": An Integrated Learning Tool Justin Tholany, MD, University of Massachusetts Medical Center, Worcester, MA, Chun-Sham Yam, PhD, Thomas M. Cummings, MD, Chetan D. Rajadhyaksha, Jay M. Colby, MD, Ashley Davidoff, MD

Purpose: "All Pancreas" is an interactive computerized learning tool that aims at providing extensive and in-depth coverage of information relating to the pancreas. Unique software has been developed to inte- grate radiology with pathology, surgery, and clinical medicine.

Materials and Methods: We have developed an imaging library of pancreatic disease that currently consists of over 1000 images, inte- grated with information relating to the basic and clinical sciences. This allows the user to access in-depth information on the pancreas

1046