Impact Assessment Indicators for Administration of Public Transportation GrantsNCHRP 20-65 Task 49

Viktor Zhong

Scott Baker

Dec 3, 2015

Agenda

1. Background and Introduction

2. State Impact Indicators in Use• Overall program administration efficiency• Effectiveness of funding advocacy and funding programming• Timeliness of grant administration activities• Provision of grantee support

3. Indicators Reporting

4. Recommendations and Conclusions

12/3/2015

Background and Introduction

Research Background

“How effective is the state’s administration of transit programs?”

“How efficiently does the state use resources for transit program administration?”

– Two questions of interests to stakeholders of state transit administration, including state transit administrators and external stakeholders

– Indicators of state transit administration effectiveness and efficiency - focus of this study

– Indicators can be misinterpreted and misused

12/3/2015

Research Approach

12/3/2015

Surveyed all 50 states’ public transit administration

Conducted more in-depth interviews with three states- Missouri Department of Transportation, Multimodal Division

- Texas Department of Transportation, Public Transit Division

- Virginia Department of Rail and Public Transportation

Compiled performance indicators in use by state transit administration

State Administration Activities and Process

12/3/2015

Indicators reported in each of the three areas of state transit administration, as well as overall effectiveness and efficiency

Securing State Match/Appropriation

Making Federal Grant Application

•Local transit training

• State vehicle procurements

• Inspections• Local transit

planning,• Construction

management• Drug and alcohol

testing• Compliance

assistance• MAP-21 (asset

management plans, performance targets, safety plans)

Funding and Allocation

Receiving and Reviewing Grant Applications

Eligibility and Selection of Grant Applications

Making Grant Awards

Administration and Reimbursement of Grants

State Transit Program Design &

Management

1. Funding and Program Level

3. Grantee Support

2. F

undi

ng A

dmin

istr

ation

Audit

Closeout

Performance Indicators In Use

Overall Administration Effectiveness and Efficiency

Indicators State

Administration cost as % of total annual expenditureFL, MO, OR, VA,

WV

Program administration cost per trip supported by the program

MN

Actual expenditure to budget ratio MA, MO

Percent of partners indicated “Satisfied” or “Very Satisfied” with DOT in delivering transportation services

MO

12/3/2015

Administration Cost As A Percentage Of Total Annual Expenditure

– A measure of overall administration efficiency;

– A lower percentage is desired by some, but…

CAVEATS: when it is compared among states or over time:– Administration cost is not proportional to funding level– Amount and quality of services provided drive the ratio

– Consider separating grant administration costs from other program administration expenditures

12/3/2015

¿𝑺𝒕𝒂𝒕𝒆 𝑨𝒅𝒎𝒊𝒏𝒊𝒔𝒕𝒓𝒂𝒕𝒊𝒐𝒏𝑪𝒐𝒔𝒕𝒔

𝑻𝒐𝒕𝒂𝒍 𝑨𝒏𝒏𝒖𝒂𝒍 𝑬𝒙𝒑𝒆𝒏𝒅𝒊𝒕𝒖𝒓𝒆𝒐𝒇 𝒕𝒉𝒆𝑻𝒓𝒂𝒏𝒔𝒊𝒕 𝑷𝒓𝒐𝒈𝒓𝒂𝒎

Funding and Program Level

Indicators State

Total state and federal transit funding received/expended/under contract/available in all open grants/active

AZ, IL, MO, NM, VA, TX 

Percent of federal funds subject to forfeiture at the end of the fiscal year (Sept 30) committed by the department

FL

12/3/2015

Total Funding Level and Funding Lapsed

– Example from Texas DOT’s Public Transit Division

12/3/2015

Effectiveness and Efficiency of Grant Administration

Indicators State

Number of grants awarded/opened/managed AZ, IL, TX, VA

(Percent of )vendor invoices paid on time or time it takes to pay a requisition

AZ, MO, OH, IL

Operating/capital payments processed and to Finance on time

MI

• Average number of days from sponsor project selection to project award (MO)• Time from grant application until time of grant award contract (IL)

IL, MO

Staff hours per application review WI

Number of grants closed AZ

12/3/2015

Provision of Grantee Support

Indicators State

Number of subrecipient grantees served VA

Number of vehicles purchased/delivered to grantees AZ, MO

Number of compliance program field visit technical support

TX

Number of site visits for transit grantees’ compliance with federal and state requirements completed

MO

Number of participants in transit training MO

Number of grant recipient compliance reviews completed TX

Staff hours per procurement WI

Average purchased vehicle delivery time AZ

12/3/2015

Indicators Reporting

Indicators Reporting

– Purposes: internal and external communication

– Internally• Audience: state transit administrators • Purpose: to evaluate administrative performance and inform

management decisions to improve effectiveness and efficiency

– Externally• Audience: funders (state and federal), grantees, and general public• Purpose: to demonstrate to key stakeholders the state transit

administration used resources responsibly to provide quality services to their grantees

12/3/2015

Indicators Reporting

12/3/2015

– Reported in strategic and/or business plans

– Reported in performance report• Use a dashboard tool

Recommendations and Conclusions

Conclusions and Recommendations

– Success in tracking indicators in all aspects of state administration, including• Overall administration efficiency• Effectiveness of funding advocacy and funding programming• Timeliness of grant administration activities• Provision of grantee support

– Consider using a dashboard tool

– Consider separating grant administration costs from grantee support costs

12/3/2015

Thank You