Post on 31-Dec-2015
description
Estimating the Economic Estimating the Economic Costs of a Finding of Costs of a Finding of
Research Misconduct in Research Misconduct in Faculty PI’s: Case Studies Faculty PI’s: Case Studies from ORI Annual Reportsfrom ORI Annual Reports
Elizabeth Gammon, Ph.D, CPA; Elizabeth Gammon, Ph.D, CPA; Luisa Franzini, PhD; Stephen Luisa Franzini, PhD; Stephen Linder, PhD; Jacquelyn Slomka, Linder, PhD; Jacquelyn Slomka, PhDPhDFleming CenterFleming CenterUniversity of Texas-School of University of Texas-School of Public HealthPublic Health
1
Why look at economic Why look at economic costs of research costs of research misconduct?misconduct?“ …“ …institutional leaders may wish to ignore institutional leaders may wish to ignore
or minimize allegations of possible research or minimize allegations of possible research misconduct to protect the revenue that the misconduct to protect the revenue that the researcher generates; some may avoid researcher generates; some may avoid investigations because they are costly in investigations because they are costly in terms of time and money.”terms of time and money.”
Titus, et al. Repairing research integrity. Nature. 453(7198):980-2, 2008 Jun 19. Titus, et al. Repairing research integrity. Nature. 453(7198):980-2, 2008 Jun 19.
2
Study AimsStudy Aims Survey Research Integrity Officers (RIO) Survey Research Integrity Officers (RIO)
responsible for handling findings of responsible for handling findings of research misconduct regarding cost research misconduct regarding cost issues;issues;
Develop a model to estimate the Develop a model to estimate the economic costs of research misconduct economic costs of research misconduct in publicly funded medical research from in publicly funded medical research from the perspective of NIH; andthe perspective of NIH; and
Use the model to estimate economic Use the model to estimate economic costs in known findings of research costs in known findings of research misconductmisconduct
3
Methods Methods Analytical Framework: Analytical Framework:
Transaction CostsTransaction Costs
All economic costs 1
Transaction costs - Costs of makingexchange or the indirect productionexpenses.
Production costs - Costs oftransforming inputs into outputsor the direct production expenses.
Motivation costs – Costs of motivatingspecialized agents to align their interests, e.g.:Cost of cheating or opportunistic behavior, Williamson [1975, 85].
Agency cost among owners, managers,and debt holders, Jensen and Meckling [1976].
Coordination costs – Costs of coordinatingactions between specialized agents, e.g.:Cost of obtaining information, Stigler [1961].Cost of coordinating input in production, Alchian and Demsetz [1972].Cost of measurement, Barzel [1982].
1http://www.encycogov.com/B11ResearchTraditions/TCE/Exhi_1DecomposeTC.asp
4
MethodsMethodsSequential Mixed Methods Sequential Mixed Methods
Study DesignStudy Design
5
QualitativeQualitative methodsmethods Quantitative Quantitative methodsmethods
MethodsMethodsData Sources Data Sources
Sample Selection and Qualitative MethodsSample Selection and Qualitative Methods
ORI Annual Reports, 2000-2005ORI Annual Reports, 2000-2005 Findings of MisconductFindings of Misconduct
Internet searchInternet search Institutional RIOInstitutional RIO
6
MethodsMethodsData Sources Data Sources
Quantitative MethodsQuantitative Methods Wage rates: American Association of Wage rates: American Association of
Medical College 2007 Report on Medical Medical College 2007 Report on Medical School Faculty Salaries School Faculty Salaries
Time Inputs: 42 CFR Parts 50 and 93 Time Inputs: 42 CFR Parts 50 and 93 Final RuleFinal Rule
NIH Awards: Computer Retrieval of NIH Awards: Computer Retrieval of Information on Scientific Projects (CRISP) Information on Scientific Projects (CRISP) and NIH Awards by State and Foreign and NIH Awards by State and Foreign SiteSite
7
ResultsResultsFindings of Faculty Research Misconduct Findings of Faculty Research Misconduct
2000-20052000-2005Rank Funding Agency Investigat
or(Case)
MD/PhD
Year
MisconductType
Professor(N= 4)
NIAMS,NIADNIA,NIDDK,NCRRNIAIDNINDS
KammerPoehlmanRadolfPrasad
MDPhDMDPhD
2005200520032002
Fabrication/FalsificationFabrication/FalsificationFabrication/FalsificationFabrication/Falsification
AssociateProfessor(N=5)
NHLBINHLBINIDCDNEINIDDK
GelbandYaoDreyerHartzerGanz
PhDMD/PhDMD/PhD
PhDMD
20032002200020002002
FalsificationFalsificationFabrication/FalsificationFalsificationFabrication/Falsification
AssistantProfessor(N=7)
GMMHNHLBINIAIDNIAIDNIDDKNIA
XiongRuggieroPandurangiSultanDuanArnoldPadgett
PhDPhDPhD
MD/PhDMDPhDPhD
2001200120012004200020012001
Fabrication/PlagiarismFabricationFalsification/PlagiarismFalsification/PlagiarismFalsificationFabrication/FalsificationPlagiarism
Instructor(N=1) NINDS
Jacoby MD/PhD 2001 Falsification/Plagiarism
8
ResultsResults
Key informant interviewsKey informant interviews
Economic costs are not measured nor Economic costs are not measured nor considered in research misconduct considered in research misconduct findingsfindings
Emphasis on proper completion of the Emphasis on proper completion of the inquiry and investigation within the inquiry and investigation within the mandated timelinemandated timeline
9
ResultsResults RIO interviews RIO interviews
Research Misconduct Questionnaire Research Misconduct Questionnaire ParticipationParticipation Total Sample
RIO Interview Participation (n) %
Total RIOs in sample 16 100
Consented to telephone interview 12 75 Case RIO 6 37.5 Assumed RIO position after case finding 6 37.5Declined telephone interview 2 12.5No response to interview request 2 12.5
10
ResultsResults
RIO experience in positionRIO experience in position Range = (< 6 months, >20 years)Range = (< 6 months, >20 years) All had additional research administration related All had additional research administration related
dutiesduties All RIOs believed measuring economic costs All RIOs believed measuring economic costs
was appropriatewas appropriate No institutions in the study sample measured No institutions in the study sample measured
economic costs of research misconducteconomic costs of research misconduct Faculty time was the single most costly Faculty time was the single most costly
elementelement
11
ResultsResults
Model - Economic cost components Model - Economic cost components of research misconductof research misconduct
Type 1 Type 2 Type 3 Type 4 TotalCosts
Investigative Costs
+ GrantAward(s)
+ Voluntary
Exclusion
Agreement
+ Retraction Cost
= TotalCost
12
ResultsResults
Model - Economic cost components Model - Economic cost components of research misconductof research misconduct
Type 1 Type 2 Type 3 Type 4 TotalCosts
Investigative Costs
17 cases
+ GrantAward(s)
13 cases
+ Voluntary
Exclusion
Agreement
3 cases
+ Retraction Cost
7 cases
= TotalCost
17 cases
13
ResultsResultsInvestigative Costs Investigative Costs
Key Assumptions for Investigative CostsKey Assumptions for Investigative Costs
100% RIO time = Estimated annual burden 100% RIO time = Estimated annual burden for ORI defined tasks + institutional tasks for ORI defined tasks + institutional tasks
RIO expends 50% of each day during an RIO expends 50% of each day during an inquiry and investigation overseeing the inquiry and investigation overseeing the investigation investigation
RIO hourly wage rate is $119 for PhD and RIO hourly wage rate is $119 for PhD and $144 for MD $144 for MD
Assumption for mixed faculty rank hourly Assumption for mixed faculty rank hourly wage rate is $144/hr.wage rate is $144/hr.
14
Type 1
Investigative Costsn=17
ResultsResults
1.1. Receipt of allegationReceipt of allegation
2.2. Preliminary assessmentPreliminary assessment
3.3. Conduct of inquiry and investigationConduct of inquiry and investigation
4.4. Institutional decision $102,115 - Institutional decision $102,115 - $141,090$141,090
5.5. ORI oversight reviewORI oversight review
6.6. PHS decisionPHS decision
7.7. Imposition of PHS administrative actions Imposition of PHS administrative actions
$ 14,110 - $ 14,110 - $ 24,495 $ 24,495
15
Type 1Type 1
Investigative Investigative CostsCostsn=17n=17
Results Results Type 1
Investigative Costsn=17
16
Time Input Cost Input
Federal Register Average Estimated Institutional PhD MD
Process Step RIO Process(Hours) Added Process (Hours) ($119/hour) ($169/hour)
Receipt of allegation
Inventory and sequester evidence 35 $ 4,165 $ 5,915
Supervise respondent access to research record 5 595 845
Subtotal 40 $ 4,760 6,760
ResultsResults
17
Type 1
Investigative Costsn=17
Time Input Cost Input
Federal Register Average Estimated Institutional PhD MD
Process Step RIO Process(Hours) Added Process (Hours) ($119/hour) ($169/hour)
Inquiry
RIO administrative oversight (50% effort)1 221 $ 26,299 37,349
Notify presumed respondent 1 119 169
Respondent review and comment on Inquiry report 1 119 169
Notice to respondent if investigation is warranted 0.5 60 85
Conduct of inquiry
Receipt and recording of allegation 1 119 169
Inquiry Panel assembly 1 119 169
Inquiry Panel time2
3 Full Professors @ 2 hrs x 3 mtgs 18 2,592 2,592
Inquiry Panel Preparation Time (3x.5x3 mtgs) 4.5 648 648
RIO support for Panel Inquiry 3 357 507
Legal support 2 238 238
Institutional notice to ORI of warranted investigation 16 1,904 2,704
Subtotal 18.5 250.5 $ 32,574 $ 44,799
Results Results
18
Time Input Cost Input
Federal Register Average Estimated Institutional PhD MD
Process Step RIO Process(Hours) Added Process (Hours) ($119/hour) ($169/hour)
Investigation
RIO administrative oversight (50% effort)1 394 $ 46,886 $ 66,586
Notify presumed respondent 1 119 169
Conduct of investigation
Selection of investigation panel 1 119 169
Charge to panel 1 119 169
Administration support for panel
Investigation Panel time2
3 Full Professors @ 2 hrs x 5 mtgs 30 4,320 4,320
Investigation Panel Preparation Time (3x.5x 5 mtgs) 7.5 1,080 1,080
RIO support for panel investigation 3 357 507
Legal support 4 476 476
Record and transcribe witness interviews 15 1,785 2,535
Submission of required documentation to ORI 80 9,520 13,520
Subtotal 96 440.5 $ 64,781 $ 89,531
Total institutional inputs per case 154.5 691 $ 102,115 $ 141,090
ResultsResults
19
Type 1 Type 2Type 2 Type 3 Type 4 TotalCosts
Investigative Costs
+ GrantGrantAwardAward
(s)(s)
+Voluntar
y Exclusio
nAgreem
ent
+ Retraction Cost
= TotalCost
Frequency of Grant Frequency of Grant Applications Cited in Applications Cited in
Misconduct Findings per Misconduct Findings per Investigator Investigator (n=54)(n=54)
02
46
8
Num
ber
of In
ve
stiga
tors
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Number of Grant Applications
20
ResultsResults
Grant Applications
Funding Status
Grant
ApplicationsNumber
% of Total(n=54) Economic Cost
No evidence of funding
31 57.4 -
PI guilty of research misconduct 19
35.2 $5,935,797
Investigator not PI 4 7.4 $5,644,062
Type 2Type 2
Grant Grant Award(s)Award(s)
n=13n=13
21
ResultsResults
22
Type 1 Type 2Type 3Type 3 Type 4 Total
Costs
Investigative Costs
+ GrantAward(
s)
+VoluntarVoluntar
y y ExclusioExclusio
nnAgreemAgreem
entent
+ Retraction Cost
= TotalCost
ResultsResultsType 3Type 3
Voluntary Voluntary Exclusion Exclusion
AgreementAgreementn=3n=3
Calculated VEA (n=2)
RIO reported VEA (n=1)
23
ResultsResultsType 3Type 3
Voluntary Voluntary Exclusion Exclusion
AgreementAgreement
RIO EstimatedAdministrative Hourly VEA
Case Action Time Input Total Hours Wage Rate Cost
Xiong1 General certification of research activities for 36 months 2 hours/ month for 36 months 72 119$ 8,538$ 2 Formal written apology 1 hour 1 119$ 119 3 Certify publications for 36 months(2002-2004 publication date) 28 publications @ 1 hour/publication 10 119$ 1,186 4 Certify grant applications for 36 months 33 grant aplications @ 1 hour/application 33 119$ 3,913 5 Review of formal essay of publication quality on plagiarism 1 hour 1 119$ 119
13,874$
Padgett1 General certification of research attribution for 36 months 2 hours/ month for 36 months 72 119$ 8,538$ 2 Certify grant reports and applications for 36 months
PI on two R29 2 grants @ 2 reports/ grant @ 1 hour/report 4 119$ 474 PHS funding credit on publication with 2003 publication date 5 grant reports @ 1 hour/report 5 119$ 593
9,605$
24
ResultsResults Type 3Type 3
VEAVEA
RIO reported VEA – Radolf case
60 month certification period2 full tenured professors4-8 hours per certification60 certifications
$74,880 VEA Cost 25
ResultsResults
26
Type 1 Type 2 Type 3Type 4Type 4 Total
Costs
Investigative Costs
+ GrantAward(s
)
+ Voluntary
Exclusion
Agreement
+RetractiRetraction Coston Cost
= TotalCost
ResultsResults
Calculated retraction costsCalculated retraction costs
RIO reported retraction costs RIO reported retraction costs (Poehlman Case)(Poehlman Case)
Type 4Type 4
Retraction Retraction CostsCostsn=7 n=7
27
ResultsResults Investigator # Articles
RetractedRIO Hourly
WageRetraction
Cost
Duan 2 $169 $ 678
Ruggiero 4 $119 $ 949
Arnold 1 $119 $ 237
Prasad 1 $119 $ 237
Yao 1 $119 $ 237
Gelband 8 $119 $ 1,897
Poehlman 10 110*
$169 $ 3,388$ 24,153
Type 4Type 4
Retraction Retraction CostCost
28*RIO provided
ResultsResults
Total Total
Cost
29
ResultsResultsInvestigator
Type 1
Investigative Cost
Type 2
Grant Award(s)
Type 3
VEA
Type 4
Retraction Cost
Total Cost
Hartzer $ 126,610 - - - $ 126,610
Duan $ 165,585 $ 544,836 - $ 678 $ 711,099
Dreyer $ 165,585 $ - $ - $ - $
165,585
Xiong $ 120,612 $ - $ 13,874 $ - $
134,486
Padgett $ 120,612 $ - $ 9,604 $ - $
130,216
Ruggiero $ 120,612 $ 48,662 $ - $ 949 $
170,223
Pangurangi $ 120,612 $ - $ - $ - $
120,612
Arnold $ 120,612 $ - $ - $ 237 $
120,849
Jacoby $ 159,587 $ 282,332 $ - $ - $
441,919
Prasad $ 120,215 $ 1,254,109 $ - $ 237 $
1,374,561
Yao $ 120,215 $ 106,652 $ - $ 237 $
227,104
Ganz $ 122,975 $ - $ - $ - $
122,975
Radolf $ 187,756 $ 447,190 $ 74,880 $ - $
709,826
Gelband $ 320,860 $ 543,217 $ - $ 1,897 $
865,974
Sultan $ 116,160 $ - $ - $ - $
116,160
Kammer $ 116,225 $ 745,346 $ - $ - $
861,571
Poehlman $ 201,626 $
1,963,453
$ - $ 27,541 $
2,192,620
Total $ 2,526,458 $ 5,935,797
$ 98,358 $ 31,776 $ 8,592,390
30
DiscussionDiscussion
Economic costs of findings of Economic costs of findings of research misconduct are research misconduct are measureablemeasureable
Research misconduct cost driversResearch misconduct cost drivers Misaligned incentivesMisaligned incentives Downstream costsDownstream costs
31
DiscussionDiscussion Economic costs of findings of research misconduct Economic costs of findings of research misconduct
are measureableare measureableRank
Investigator(Case) MD/PhD Year
MisconductType Total Cost
Professor(N= 4)
KammerPoehlmanRadolfPrasad
MDPhDMDPhD
2005200520032002
Fabrication/FalsificationFabrication/FalsificationFabrication/FalsificationFabrication/Falsification
$ 709,826$ 861,571$ 1,374,561$ 2,192,620
AssociateProfessor(N=5)
GelbandYaoDreyerHartzerGanz
PhDMD/PhDMD/PhD
PhDMD
20032002200020002002
FalsificationFalsificationFabrication/FalsificationFalsificationFabrication/Falsification
$ 122,975$ 126,610$ 165,585$ 277,104$ 865,974
AssistantProfessor(N=7)
XiongRuggieroPandurangiSultanDuanArnoldPadgett
PhDPhDPhD
MD/PhDMDPhDPhD
2001200120012004200020012001
Fabrication/PlagiarismFabricationFalsification/PlagiarismFalsification/PlagiarismFalsificationFabrication/FalsificationPlagiarism
$ 116,160$ 120,612$ 120,849$ 130,216$ 134,486$ 170,223$ 711,099
Instructor(N=1)
Jacoby MD/PhD 2001 Falsification/Plagiarism $ 441,919
32
DiscussionDiscussion
Investigative Costs Grant Award VEA Retraction Cost
Repeated misconduct Repeated misconduct Admission of misconduct
Repeated misconduct
Complexity of grant specific aims
Discovery date of misconduct Concurrent with grant application process Pre-award Concurrent with active grant period Post grant project period expiration
Tenure status
NIH award type Career development award (K series) Research training/fellowships (T/F series) Research grants (R series) Program project & center grants (P series)
Research misconduct cost drivers
33
DiscussionDiscussion Misaligned incentivesMisaligned incentives
Deterrence theoryDeterrence theory Expenditures to deter misconduct determine Expenditures to deter misconduct determine
probability of discovery probability of discovery Misconduct detection costs have been Misconduct detection costs have been
unmeasuredunmeasured Unfunded mandateUnfunded mandate Unmeasured prevention costs Unmeasured prevention costs
PunishmentPunishment Inconsistent, uncertainInconsistent, uncertain Research institutions have Research institutions have
responsibility for detection but bear responsibility for detection but bear little risklittle risk
34
DiscussionDiscussion
Citation Maps Retracted Articles by Duan..
Original Article
Most frequently-cited articles
Article A cites Article B
Downstream costs Duan case13 publications acknowledge $ 7,314,689 total grant funding
35
LimitationsLimitations
Study measures only one category of Study measures only one category of research misconductresearch misconduct
Research misconduct unmeasured ifResearch misconduct unmeasured if UndetectedUndetected UnreportedUnreported
36
LimitationsLimitations
No processes in place to document, estimate No processes in place to document, estimate or measure costs associated with research or measure costs associated with research misconductmisconduct Economic cost model relies heavily on Economic cost model relies heavily on
assumptionsassumptions Cost categories not consideredCost categories not considered
Value of a ruined careerValue of a ruined career Findings of misconduct attributable to non faculty Findings of misconduct attributable to non faculty Responsible Conduct of Research program costsResponsible Conduct of Research program costs Impact of flawed evidence base on public healthImpact of flawed evidence base on public health
37
QuestionsQuestions
38