1

STAR METRICS & SciENCVJobs, Projects, and People

Nov 7, 2012George Chacko, Walter Schaffer,

and James OnkenNational Institutes of Health

2

WHAT IS STAR METRICS?

Science and Technology for America’s

Reinvestment:

Measuring the EffecTs of Research on

Innovation, Competitiveness and Science

3

STAR

METRICS

4

PARTICIPATING AGENCIES

• NIH• NSF• OSTP

• USDA• EPA• DOE

5

WHY STAR METRICS?

• Presidential Memorandum on Transparency and Open Government (Jan 21, 2009)

• Office of Management and Budget Memorandum Open Government Directive (Dec 8, 2009)

• “…agencies should publish information online in an open format that can be retrieved, downloaded, and searched.”

• “…agencies should proactively use modern technology to disseminate useful information.”

6

Genesis and History

• Jan 2010• Jul 2010• Oct 2010• Jul 2011• Oct 2011 NIH becomes host institution• Jan 2012 New Governance • Apr 2012 Goals refined

7

GOVERNANCE

Lead Entity Executive

Program Managers

Technical Project

ManagerContractors

Executive Committee

Interagency Working Group

Research Institutions

8

SM GOALS

• Level 1: Document the levels and trends in the scientific workforce supported by federal funding.

• Level 2: Develop an open automated data infrastructure and tools that will enable the documentation and analysis of a subset of the inputs, outputs, and outcomes resulting from federal investments in science.

9

Level I

Research Institution

STAR METRICS

Quarterly Data Submissions

Jobs Report

10

Level I Enrollment

11

LEVEL I

• Intermediate Goals

– Statistical study to guide enrollment– Evaluating data quality– Evaluating job estimate calculations– Developing a data policy– Solicit participation of minority serving institutions

12

LEVEL I (cont’d)

• Long-Term Goal 1: Increase participation– Establish and attain targets for the collection of

workforce data for federally funded extramural research based on the outcomes from the statistical study

– Develop an approach for including workforce data on federally funded intramural research

– Work with partner organizations such as the FDP, AAU, APLU, and AAMC

• Long-Term Goal 2: Finalize workforce report and extend the usability of the data collected

13

Level II

Trans-agency Database of Research Projects

14

Roadmap for Level II

• Policy level agreement on projects database

• Technical implementation plan• Proof of concept pilot• Testing and refinement• More testing and refinement• Production

15

Level II Concepts

• A database of federally funded research

• Make data available to federal agencies and other interested groups

• EPA, NIH, NSF, and USDA will contribute information to a proof-of-concept database.

• NIH will collect data from the four agencies and create and manage the database

16

Level II Downstream

• More research outputs– Publications– Patents– Links to researcher profiles, e.g. SciENCV

• Enabling– Metrics development– Cross Agency Analysis– Benchmarking research institutions

Ongoing Activities

• Level I– Statistical Study

• Level II– Design– Pilot

17

Data P

olicy

18

BUDGET

Agency Commitment/Year FY 2012

NSF $500,000 Received

NIH $500,000 Received

USDA $250,000 Received

EPA $250,000 Received

DOE -

Total $1,500,000

19

GOVERNANCE

Sally Rockey

(NIH)

Chacko (NIH)

Winter (NSF)

(TBD) SynthosysSapient

Executive Committee

Interagency Working Group

Research Institutions

20

WITHIN NIH

NIH Staff• Sally Rockey (OER)• George Chacko (CSR/OER)• Jim Onken (OER)• Jack Vinner (CIT)• Megan Columbus (OER)

Contractors• Synthosys • Sapient

21

Other

• NSF (Myron Gutmann, Susan Winter, Chris Pece)

• USDA (Catherine Woteki, Sharon Drumm, John King)

• EPA (Lek Kadeli, Bronda Harrison, Mya Sjogren)

• OSTP (Philip Rubin, Kei Koizumi)• DOE (Patricia Dehmer, Julie Carruthers)

22

Acknowledgments

• NIH ERA– Pete Morton, Rick Ikeda

• NIH OER– NETE– Pat Porter, Carol Kosh, Jess McKnight-Cullen

Former STAR METRICS Managers• Stefano Bertuzzi (ASCB)• Julia Lane (AIR)

23

IN SUMMARY

STAR METRICS is an open automated data infrastructure

project that will enable the documentation and analysis of a subset of the inputs, outputs, and outcomes resulting from federal

investments in science.

Science Experts Network and Curriculum VitaeProject Status Update

NCURANovember, 2012Walter Schaffer

24

Brief Background

• “Why do I have to keep reentering the same data into federal grant systems?”

• “Why does each separate federal grants system have its own format requirements for bio-sketches?”

• “Why can’t the federal grant-related reporting requirements use the same data that I’ve already entered into a federal system?”

• “I already maintain my profile data in my system of choice. Why can’t I simply point to that as a source of information for my CV?”

• “The administrative burden is far too large when dealing with the government.”

25

Enter… SciENCVCo-chaired by NSF and NIH.

Six agencies (DOD, DOE, EPA, NIH, NSF, USDA) currently working together to “make it happen”.

Working with the Federal Demonstration Partnership (FDP) to ensure that the system meets the goal of reducing the administrative burden on those involved in research who interact with the federal government.

26

Researcher View

27

Project Plan• Assemble NIH, NSF, and other Agency required data elements –done• Identify NIH Data sources and outputs - done• Develop data model - done• Develop NIH Use Cases & Wireframes - done• Establish timeline and level of effort for the Project – done• Approval by Research Business Models (RBM) - ?

Initial “deliverables”• Create sample XML records, pulling data from existing NIH data

sources (myBibliography, NIH electronic Research Administration (eRA)• Pull data from VIVO, Harvard Profiles, ORCID and other external

sources • Target HTML display pages for the NIH and NSF Biosketch and sample

CVsTest and Expand to other AgenciesSee YouTube Video

SciENCV Core Services PrototypeProject Plan (High-Level)

28

29

30

31

Homepage Dashboard

32

Profile info

Navigation

“Action” area

Data Sources

Objects

Products/History

33

34

35

36

Complex data objects

Bibliography, Funding, etc.

37

38

39

40

41

42

43

44

45

46

Putting it all together

47

48

49

50

end

Please direct all complaints toBart Trawick, Debbie Bucci, Neil

Thakur, Susan Winter and Pete Morton

51

NIH Research Portfolio Online Reporting Tools (RePORT)

Making the Most of Project and Investigator Information

53

Access to the NIH Portfolio

The NIH Reform Act of 2006

Requires establishment of electronic system to uniformly code research grants and activities of the Office of the Director and of all the national research institutes and national centers

The electronic system shall:• be searchable by a variety of codes, such as grant type, IC,

and public health area of interest (category)• when permissible, shall provide information on relevant

literature and patents that are associated with research activities of the National Institutes of Health

NIH RePORT Expenditures and Results (RePORTER)

54

Basics

Data Elements• Grant/Contract/Project Number• Project Title• Project Description• Principal Investigator• Funding Agency• Recipient Institution• Recipient Address• Project Start Date• Project End Date• Fiscal Year (of funding)• Funding Amount

Uses• Searching• Browsing• Summary Statistics• Graphics• Links to External Sources of Information• Reports (Standard and Customized)

55

The NIH Solution

56

Querying

57

Browsing

58

Using Search Results

59

Using Search Results

60

Visualizing ResultsResearch Support to Johns Hopkins

61

Visual Browsing

62

Mapping

63

Mapping

64

Third-Party ToolsTopic Modeling

65

Links to External Information Sources

66

• Save queries• Receive email

alerts• Save portfolios• Share queries

Customized Reports

67

Electronic Alerts

Electronic Alerts

A Community Resource

70

THANK YOU

NOTES:

This presentation was prepared by:James Onken, Ph.D., RePORT Program Manager (onkenj@od.nih.gov)

Sandeep Somaiya [C – NETE SOLUTIONS], RePORT Project Lead (somaiyas@od.nih.gov)