You can use it when your sister is done with it!!!Challenges in intra-institutional data sharing

José Sigouin – University of TorontoRobert Dirstein – Dirstein Consulting Inc.

November 19, 2014

Show of Hands

Work with a research information system

Stand-alone RIS

RIS interacts with other systems

University Org Chart

President

VP Research & Innovation

Research Services Offices

VP & Provost

Deans

Chairs/Directors

VP Human Resources CFO VP University

Operations

Definitions – What’s a University

Derived from the Latin universitas magistrorum et scholarium, or “community of teachers and scholars”. (Wikipedia)

I have sometimes thought of the modern university as a series of individual faculty entrepreneurs held together by a common grievance over parking (Clark Kerr – President of the University of California)

What data is collected?

Data that has to be collected to support management of the institution– Who works at the institution– What are the research dollars, who has them,

what are the rules• Data that is under institutional control

Data that is not directly under institutional control– International dimension of research– Publications, citations, collaborations

U of T’s RIS exists primarily to manage research funds Most used measure: Funds Awarded (spending limits

allowed by sponsors over defined periods of time, aka Award, aka Budget)

Not the same as Research Revenue (purely FIS territory)

Not the same as Research Expenditures (purely FIS territory)

All 3 are valid measures of research input

“Data is only as good as the process that collects it.” Anonymous

RIS data exist primarily to manage research fundsMissing: Donations in support of research – tracked in DIS Service contracts In-kind

Derivative uses: Internal allocations Success rates Participation rates Demographic and gender analyses

Internal Allocations Where RIS Data Has to Be Right

1. Federal indirect costs distribution

2. Allocation of VPRI portfolio costs to divisions value of funding # applications # active funds

Success Rate

Is competition deadline always entered? Is it accurate? Is it complete? What if an applicant bypassed the research

office and ends up being unsuccessful?

Is it checked by anyone?

Does it drive a process?

Data is only as good as the processes it drives.

Participation Rates

# funded fac members with grants in Dept A# eligible faculty w/ main appointment in Dept A

Can lead to puzzling participation rates,sometimes even greater than 100%

Better approach:

1. Identify eligible faculty members with main appointments in Dept A

2. Among that group, identify who is funded (regardless of where grants are administered)

# funded fac members w/ main appointment in Dept A# eligible faculty w/ main appointment in Dept A

How many faculty members does the University of Toronto have?

Facts & Figures 2013

What is the difference?

Why does it matter?

Gender Analysis

Demographic Analysis

Does Everything Add Up?

Double-counting? Undercounting?

Considerable hidden noise in these apparently matching data

Council Reported by Councils* CAUBO**CIHR $816,385,752 $813,545,000NSERC $798,228,209 $790,737,000SSHRC $254,064,436 $249,431,000Grand Total $1,868,678,398 $1,853,713,000

* Apri l to March; l imited to Col leges , including CEGEPs , and Univers ities in Canada (U of T analys is )

** Varying fi sca l years ; excludes CEGEPs

Note, however, that SSHRC and CIHR reported-by-counci l numbers include NCEs but NSERC's don't.

2012-13

Case Study - MRA

Approval for submission of research proposals to external funders– All such (research) proposals must be approved by the

Principal Investigator and the appropriate officials in the Administering Unit and/or the academic division housing the Administering Unit, and then submitted to the appropriate unit of the office of the Vice-President, Research and Innovation for institutional review and approval before being sent to the sponsoring organization. (Research Administration Policy)

Automation of workflows to ensure adherence to the Research Administration Policy

Approval Schematic Application

Chair/Primary

Appointment

Request to Administer in

Different Unit?

Admin Chair

Hospital?

Provost?

Provost?

Dean? Dean?

Vice Dean?

Vice Dean?

Hospital?

University Approvers

University Approvers

Yes No

• Assistant may prepare and forward to PI

• PI submits goes to Chair/Director of Unit of Primary Appointment

• Request to Administer in a different unit?

• No• Yes

• Other Approvals required based on escalation rules

Data

What’s needed?– Personnel information, organizational

structure and relationships Who needs it?

– Research Services Office Who owns it?

– HR • (sort of)

What we discovered

Multiple official versions of the org structure– HRIS– FIS– ROSI– SAKIA

Why? – each addressed the business needs of the owner

Which one?

HRIS – provides the people, the type of appointment and the relationships of people and positons to units

Unfortunately no system differentiation between academic and non-academic units

Data quality issues

Why is data quality so important?

Resulting System Landscape

FISRIS

HRIS

MRA

MRHP

Plan 90

UTORauth

Resulting System Landscape

FISRIS

HRIS

MRA

MRHP

Plan 90

UTORauth

Ongoing challenges

Need to understand consequences of actions

Institutional Structure, Decision/Budgetary Authority & Culture favour Atomization

Solutions

Look past the detritus of current practice and examine objectives

To what use will the data be put?– Now– In the future

Leadership Communication