Tufts CTSICommon Metrics Implementation

Learning Session 6February 14, 2016

All teach, all learn• Real-time IRB, CTSI of Southeast Wisconsin• CTSI Connect, NYU CTSI

Resources• Careers metric Driver Diagram & Change Package• Careers metric worksheet• Revised Operational Guidelines

Increase investigator awareness of available hub support services (faculty meetings, symposia/fairs/expo, optimize web site, partner with marketing) Provide investigators with:

Application templates Frequently Asked Questions (FAQs) Flowchart depicting the IRB process Tip sheet on how to improve an application Submission checklists Exemplar protocols and consent forms Periodic updates and tips, e.g., in a newsletter

Conduct training in the IRB application process for investigators and staff Provide support during application preparation (drop-in clinics, consultation services) Provide pre-screening / pre-review services Provide feedback on rejected submissions

Assess for staff member training needs and provide appropriate training Assign a single coordinator to support a study through the entire process Develop and follow Standard Operating Procedures for each step of the process Increase the number of review panels/committees (and/or frequency of meetings) Increase meeting frequency during high-demand periods

Post turnaround time metrics on a public-facing website Elicit feedback from investigators on their experience with the process at the time of

each IRB approval Hold focus groups with small groups of investigators Assess protocols with particularly long TAT for commonalities, potential remedies

Use quality improvement tools to clearly understand steps in the process and identify potential waste or bottlenecks (Process workflow mapping, Root cause analysis, LEAN / Six Sigma) Set targets for the duration of specific steps in the process Identify & remove redundant & non-essential questions from the IRB application Avoid process stagnation by engaging in parallel reviews

Utilize an electronic IRB submission and tracking system Improve online instructions at the time of data entry

Program electronic reminders for outstanding responses to inquiries

1. Engaged and supported investigators create high-quality applications and respond to inquiries in a timely manner

5. Processes are improved based on feedback from researchers and system metrics

3. Waste and redundancy are identified and eliminated

4. Use of appropriate technology is optimized

Common Metric AimImprove the median number of calendar days from the official IRB application receipt date to the official IRB final approval date for fully reviewed protocols

Median IRB Review Duration Driver Diagram v1.0

2. IRB staff and review committees are sufficient and appropriate with optimized workloads

Drivers

Strategies

6. Additional drivers and strategies will be identified as the Common Metrics Initiative continues

Real-time IRBDavid Clark, PhD

AnnMarie Eve, MSRyan Spellecy, PhDEmily Connors, MS

Story Behind the Curve: Factors Affecting Time to IRB Approval

• Incomplete applications• Budget delays• Languishing in study team inboxes• Scheduling of ancillary committee meetings• Post-approval back and forth• Tabling

GoalAggressively decrease time to approval without compromising regulatory or ethical standards for IRB review.

o Assign a research submission for review at a Convened IRB Committee within 14-20 days from the date of submission* to approval

o Provide a mechanism to reduce the rate of “table” decisionso Issue approval letter with few conditions on first review

Real-time IRB experiment• Two of the four IRB committees have been designated and

trained for “real-time”• Investigator eligibility for real-time review:

o Self-nominatedo 3 or more protocols within last 2 years approved or modifications required at first IRB

review meeting

• Both investigator and study coordinator must attend IRB meeting

• If study cannot be approved, investigator may enter changes into eBridge while the IRB is meeting and then re-present the study for a second review during the same meeting

Impediments • Need for sponsor approval of IRB requested changes• Scheduling commitment required of PI (clinical

schedule/emergencies)• Ancillary safety and compliance reviews• Complex questions requiring more time and thought

Results

Department / Division

Submission to Committee

review (days)

Committee review to final approval

letter (days) TOTAL DAYS

Nephrology 12 4 16Hematology & Oncology 7 3 10

Psychiatry 12 5 17Psychiatry 12 5 17Cardiology 5 2 7

11 studies as of November 22, 2016

Department/ Division

Submission to Committee

review (days)

Committee review to final approval

letter (days)TOTAL DAYS Exception

Neurosurgery14 22 36

State Variance required

Colorectal Surgery 21 1 22Hematology & Oncology

18 23 41Funding finalization

Gastroenterology 13 60 73Funding finalization

Neurosurgery 19 PendingFunding still

pendingBiophysics 21 1 22

Lessons Learned• Submission no later than 10 days in advance of the IRB

Committee meeting• Additional IRB staff member assistance is required• Only one real-time review per meeting• ONLY designed for a subset of submissions• Other reviews/approvals outside of the IRB office hold up

final approval • Collaboration

Translating Research into Better Health

Deborah Chavis-Keeling, MScExecutive DirectorAdministration and Finance

Rachel Thornton, MPHProgram CoordinatorRecruitment and Retention Unit

February 14, 2017

Today’s Presentation

• Thematic Grouping to our Programs

• Targeted Outreach

• Other Collaborative Offerings

NYU CTSI: Overview, Organization & Themes

CTSI Executive LeadershipUL1 MPI – Bruce Cronstein, MDUL1 MPI – Judith Hochman, MD

Deborah Chavis-Keeling, MS

KL2 - MCDPI – Michael Pillinger, MD

TL1 - NRSAPI – Christine Kovner, PhD, RN

Advisory BoardsInternal Advisory Committee

External Advisory Committee

Community Advisory Board

Review and ExpertiseScientific Review Committee

Voucher Review Committee

Recruitment/Retention Committee

CouncilExecutive Committee,

Program, Unit & Core Directors

Executive CommitteeExecutive Leadership &

Partner Liaisons

NRO REM RIP TSB

Executive Vice President for Health Robert Berne

New York University

Sr. VP & Vice Dean for Science, Chief Scientific OfficerDafna Bar-Sagi

New York University School of Medicine

Partner InstitutionsNew York University Schools & Colleges

Nathan Kline Institute for Psychiatric

Research

NYULMC:School of Medicine,

Multisite CareDelivery System

Community Partners

New York CityHealth and Hospitals

Corporation

Evaluation & Continuous Improvement Cross-cutting Working Groups

Workforce Development

Outreach & Integration

Early Translation

Readiness & Execution

• July 2016: 3:00-5:00 pm

• Email blast & newsletter

• 12 weeks of planning

• More than 80 attendees

• 24 poster presentations

• Bioengineering-related topics

• Modest budget for refreshments & set-up

• Expanding program to include other schools

CTSI Poster Day: Schools of Medicine & Engineering

• Goal to encourage researchers at medical center to collaborate with researchers from engineering

• Overlap in subject and topics with different approaches and perspectives

• Presenters reported forming new collaborations; several initiating discussions pursuing future funding

CTSI Poster Day: Schools of Medicine & Engineering

• Inaugural event October 2016; 12-2 pm

• Nearly 100 attendees

• 11 CTSI units highlighted

• 5 ‘ancillary programs’

• Advertised in CTSI & OSR newsletters

• Co-sponsored by CTSI and OSR

• Institution to offer like-minded event for cores

CTSI Science Café: Overview

CTSI Science Café: Highlighted Units and Programs

Outreach & IntegrationCommunity Engagement/Population Health ResearchRecruitment and Retention Unit Integrating Special PopulationsCollaboration: NYC Health + HospitalsCommunity Based Organizations via CAB

Early Translational ResearchPilot Projects ProgramVoucher ProgramStudio ProgramScience Interest GroupsConcierge Program

Workforce DevelopmentKL2 & TL1 Training ProgramsDegrees and CertificatesMentor Development ProgramTeam ScienceRegulatory Education

Readiness & ExecutionRegulatory Knowledge & SupportMultisite Studies UnitResearch Facilities (CRC)Expanded Scientific Review Committee Integrated Informatics

Ancillary ServicesLarge Scale Grants SupportResearch Coordinator BootcampDataCore & InformaticsTranslational Research Library Services Human Research Regulatory Affairs & Services

• Welcome and opening remarks by Co-Directors, Dr. Bruce Cronstein and Dr. Judith Hochman

• Overview of CTSI by Executive Director, Deborah Keeling

• 5 minute presentations by ancillary programs

• Last hour dedicated to attendees visiting tables and speaking with program facilitators

CTSI Science Café: Program of the Event

CTSI Science Café: Other Highlights

• Lunch and coffee bar supported the “café” theme • Program facilitators at tables to discuss program offerings• Tables positioned in ‘theme’ groupings (posterboards)• 4 x 6 postcards for each program/unit disseminated

CTSI Science Café: Program Postcards

Workforce Outreach & Integration Early Translation Readiness & Execution

CTSI Science Café: Giveaways

•To encourage collection of program postcards, wegave attendees either a CTSI tote bag or water bottle.

Not to scale

CTSI Science Café: Conclusion and Future Plans

• Highly successful

• Modest budget

• ~8 weeks planning

• Plan to host annually each October

• Future plans o Hold in larger room o More outreach to

medical center community

o Conduct quick exit survey of participants

Other Collaborative Offerings

Poster Day

eBook Symposium Seminars

Thank you!

Questions?deborah.keeling@nyumc.orgrachel.thornton@nyumc.org

Website

“Start with the end in mind”Common Metric Aim

Improve the number and percent of institutional scholars and trainees who completed the KL2 and TL1 programs, respectively, who are currently engaged in clinical and translational research (CTR).

Drivers for Careers in Clinical and Translational Research

1. Scholars receive high-quality mentorship

2. Networking is facilitated between current scholars, alumni, and other successful CTR researchers in their discipline

3. Resources specifically for career development are provided

4. Scholars develop research skills

5. Graduates and their career statuses are tracked over time

Host a networking seminar of incoming scholars, current scholars and alumni to share suggestions about being a KL2 scholar Provide a Networking Forum for trainees and alumni to interact Hold regular “Lunch and Learn” or other sessions for K cohort support Provide encouragement or support for scholars to attend national events for networking Assist scholars to identify networking opportunities that best fit their needs

Provide mentorship for guidance, support, resources, connections, and funding Offer mentorship training to mid-career and early senior faculty Identify and select for generic and CTR-specific mentor competencies Provide monetary support for mentors; acknowledge mentoring toward promotion Develop systems for mentor accountability Evaluate and give feedback to mentors

2. Resources specifically for career development are provided

3. Networking is facilitated between current scholars, alumni, and other successful CTR researchers in their discipline

Common Metric AimImprove the number and percent of institutional scholars and trainees who completed the KL2 and TL1 programs, respectively, who are currently engaged in clinical and translational research (CTR).

Careers in Clinical and Translational ResearchDriver Diagram v1.0

1. Scholars receive high-quality mentorship

Drivers

Strategies

Conduct a pre-KL2 application session to provide information about institutional career development, answer questions Provide career development seminars and activities that emphasize necessary pace and steps in progression to independence Provide scholars with opportunities for external experiences (externships, etc.) Partner with Research Administration and Development Office staff to create systems to identify and communicate funding opportunities Provide time management and work/life balance training and resources Survey or interview program graduates on strategies they found helpful Require scholars to develop/implement a personal professional development plan

4. Scholars develop research skills

Provide a library of funded grants to help illustrate strong grant writing Provide “pink sheets” or reviewer comments from non-funded grants Conduct grant pacing workshops Host mock study sections to simulate NIH review sessions Partner with the Business School in activities around “pitching” one’s research Conduct “Work in Progress” writing seminars Conduct exit interviews with scholars to assess satisfaction with training and skills

5. Graduates and their career statuses are tracked over time*

Annually request and review current year CVs of graduates Conduct an annual survey of graduates

*Tracking will not change the underlying rate at which they are engaged in CTR, but is necessary in order to collect the data for the Careers metric and understand the Story Behind the Curve

Hub example strategies, by driver

Some drivers don’t have as many strategies. Many strategies don’t have examples.

Careers in Clinical and Translational ResearchDriver Diagram v1.0

Some drivers don’t have as many strategies. Many strategies don’t have examples.

Careers in Clinical and Translational ResearchDriver Diagram v1.0

Revised Operational Guidelines

• Added Numerator/Denominator statements for rate-based

metrics

• Added a purpose statement to explain the rationale for each

metric

• Separation of optional “Pilot Funding & Subsequent Funding”

metric to its own Operational Guideline

• Some reordering and editing

Revised Operational Guidelines

Careers Metric Worksheet (p 2)

Careers Metric Worksheet (p 3)

Next Learning Session

Tuesday Mar. 14, 2017

3pm – 4pm ETTuesday May 9, 20173:00-4:00pm ET